Mannheim Research Colloquium on Survey Methods (MaRCS): Response-time-based mixture models for monitoring and accounting for careless and insufficient effort responding

Careless and insufficient effort responding (C/IER) poses a major threat to the quality of questionnaire data.  Traditional indicator-based procedures for its detection (e.g., the long-string index or Mahalanobis distance) are limited in that they rely on somewhat arbitrary threshold settings and do not allow taking the uncertainty of C/IER classification into account. In this talk, I illustrate how the use of response time mixture modeling aids in overcoming these limitations.  After introducing the general idea of psychometric mixture modeling for C/IER identification, I discuss two recent developments tailored to research contexts with high vulnerability to C/IER occurrence. First, I present a C/IER monitoring tool for ecological momentary assessment (EMA) data. The presented approach equips researchers with a tool for (a) gauging the degree of C/IER contamination of their EMA data, (b) studying the trajectory of C/IER across the study, and (c) investigating occasion and person characteristics associated with increased C/IER rates. The approach is illustrated on EMA data from the German Socio-Economic Panel innovation sample. Second, I present a scalable two-step screen-time-based weighting procedure for adjusting for C/IER in the analysis of large-scale survey data. Instead of excluding response patterns from further analyses, the presented approach downweighs responses according to their probability of stemming from C/IER. The procedure has a light computational footprint and can feasibly be integrated with common analysis workflows for large-scale survey data. The procedure is illustrated on a sample of more than 400,000 respondents being administered 48 scales of the PISA 2018 background questionnaire.

Please use this link to attend the colloquium via Zoom: https://us02web.zoom.us/j/89023417510?pwd=YkZMWit5MlJ0bmRyNUR4SU94T0xkdz09 

MaRCS is a seminar series jointly organized by the Mannheim Centre for European Social Research (MZES), the University of Mannheim School of Social Sciences, and GESIS – Leibniz Institute for the Social Sciences.