Looking up Answers to Political Knowledge Questions in Web Surveys

Political knowledge can affect measures obtained in public opinion research and political science. When asked political knowledge questions, some respondents look up the answers online, inflating political knowledge scores. This response behavior is detectable in web surveys using paradata. This study investigates whether and to what extent respondents switch away from the survey to search for answers online using JavaScript OnBlur functions. Respondents were randomly assigned to device types (PC or smartphone) and response formats (open or closed) and answered a self-report question. The results reveal that about 30 percent of the respondents look up answers, depending on the device type and response format. They also reveal that switching away and correct answers are significantly associated with the response format. These findings provide new insights on looking up answers online and show that paradata are a promising new method to detect response behavior that may lead to incorrect inferences about respondents’ political knowledge. The findings also show that paradata and self-reports complement each other and that both measures should be combined in future web surveys measuring political knowledge.