Statistical Modeling Using Mouse Movements to Model Measurement Error and Improve Data Quality in Web Surveys

Research question/goal: 

Online surveys have become very prominent across many different disciplines both within the academic and private sector and also increasingly in official statistics. Despite best efforts of questionnaire designers, respondents regularly answer questions incorrectly, often because they do not understand what the question is asking. In offline surveys, interviewers could help respondents through difficulties. In web surveys this is no longer possible. However, respondents leave clues in keystrokes, response times and mouse movements on respondent difficulties and breakdowns in the measurement process. Those paradata can be used to check and improve data quality. The conventional approach in web surveys is to use response latency, where very low and very high response times are used as indication for bad data quality. The only web survey work involving mouse movements focused on the overall distance traveled by the mouse to identify questions and respondents with low data quality. However, mouse movements contain much more information than captured by either bare response times or simple summary statistics such as total distance and other predefined patterns. Despite the fast growing use of web surveys in commercial as well as official statistics, so far no large-scale research investigates the value of mouse movements in web surveys.

To fill this research gap, the proposed project will develop statistical methods to automatically analyze mouse movements in web surveys. In particular, we want to exploit the information that is contained in the mouse movements and use it to better understand measurement error and question difficulty. In the future, this work can be helpful as a basis to detect respondent difficulty and adaptively offer help in a responsive questionnaire design and to adjust for measurement error in subsequent analyses of the web survey answers.

Current stage: 

Together with our collaborators at HU Berlin, we are developing novel analysis techniques for a large set of previously collected mouse-tracking data . In addition, we are currently analysing data collected in a recent web experiment. Further studies are in preparation. The developed methods and results from the project were presented at international conferences; journal articles and book chapters are under way or have already been published.

Fact sheet

2017 to 2021
Data Sources: 
IAB survey