Sensors included in consumer-grade devices like smartphones and smartwatches provide a rich source for learning about people and their behavior in everyday-life contexts. These sensors offer the potential for objective, large-scale, real-time measurements – either as standalone measures or linked with survey data. However, measurement quality in practice remains a critical challenge for valid analyses and predictions made with sensor data. This work investigates what can already be assessed with consumer-grade mobile and wearable sensors, which measurement quality challenges researchers encounter in practice, and how they can be mitigated. We conducted a systematic literature review of field studies using sensors that are typically available in mobile and wearable devices: motion, geolocation, and heart rate sensors. We explored the potential of these sensors, analyzed measurement quality considerations, and discussed study design aspects including participant experience. The analyzed studies primarily used smartphones for motion and geolocation, and smartwatches for heart-rate tracking to assess constructs such as mobility, physical activity, mental wellbeing, and social behavior. A number of studies also used participants’ personal devices (rather than researcher-provisioned devices), which demonstrates the feasibility of large-scale sensing. Reviewing measurement quality aspects using the Total Data Quality Framework, we note that convergent validity of target measures was usually well-motivated, while discriminant validity remained secondary. Errors in data collection stemmed from both participant behavior and technology limitations. Mitigation strategies were usually applied post-hoc and often lacked detailed processing information. Accordingly, we recommend explicitly discussing discriminant validity considerations. The real-time connectivity of the sensing devices can be leveraged for real-time quality assessments and error notifications. We also emphasize reporting standards aligned with FAIR principles, and transparent communication of data quality limitations. Thus, we contribute to enhancing the validity of sensor-based studies in social science and related fields, promoting better use of widely available consumer devices for research.
