Statistical Modeling Using Mouse Movements to Model Measurement Error and Improve Data Quality in Web Surveys

Our project investigated the use of mouse cursor movements in online surveys as an indicator of participant difficulty and as a proxy for data quality. Based on the consistent finding in the cognitive sciences that mouse movements reflect uncertainty and conflict in decision experiments, and the association of similar paradata measures in surveys on perceived difficulty, we hypothesized that mouse trajectories also predict features of the response process.

In collaboration with Sonja Greven and Amanda Fernández-Fontelo from HU Berlin, we pursued a multipronged approach: In a field experiment embedded in a survey, we demonstrated that artificially creating difficulty by deviating from survey design best practices manifested itself in indices of mouse movements. Based on these results, we designed and applied functional analysis approaches to independently assign participants to the experimental conditions, showing that this is possible with substantial accuracy. With regard to the practical application as an indicator of structural issues within the survey or of individual difficulty on part of the participants, we further demonstrated that different issues are reflected in different features of mouse movements, albeit not in a strictly separable way. We finally tackled issues of privacy and consent, showing that participants are reluctant to consent to mouse movement data collection, and provided recommendations for assessing consent when implementing this method of paradata collection.

In sum, our results indicate that mouse-tracking is indicative of difficulty in surveys, but care must be taken to account for individual and contextual variability. It is likely that mouse trajectories contain further information about the specific origins of participants' problems. However, it remains to be tested whether the findings can be generalized across surveys and whether mouse tracking can be widely implemented in a manner that is readily usable by practitioners and acceptable to participants.