RELEVANCE & RESEARCH QUESTION Current research on online behavior heavily relies on self-reported data, which, if flawed, can lead to inaccurate inference in subsequent analyses. Researchers examining online behavior require detailed measures beyond “time spent on a platform” to explore, for example, well-being, social media use, or online privacy, particularly to differentiate between active and passive social media use. This study investigates the extent of misreporting in questions about fine-grained Instagram behavior by comparing them to objective measures collected via data donation. We also explore to what extent the accuracy of self-reports is dependent on the response format (rating scale vs. open text field) and the reference period (“last week” vs. “typical week”). METHODS & DATA We collected survey data from over 500 Instagram users in a German probability-based online panel regarding 25 distinct behaviors, including posting, liking, and commenting. Participants first complete survey questions on these behaviors. As part of the survey, we conduct a 2x2 experiment that randomly varies the reference period and, for a subset of behaviors, the response format. Respondents are then asked and, if they agree, instructed to download their Instagram usage data for the last three months and donate them to our research. We analyze correlation coefficients between behavioral self-reports and donated data to assess the accuracy of self-reports in general and for specific behaviors. RESULTS Our study’s data collection phase ended on 12 November, and we cannot present any results yet. We have successfully collected self-reported and donated behavior data from 122 respondents. We will update this abstract with the respective findings before 1 March 2025. ADDED VALUE This study contributes in three ways. First, we inform the field of questionnaire design by offering insights into how to accurately inquire about specific online behaviors, which is particularly interesting for researchers who may not utilize data donation methods. Second, we examine the accuracy of self-reported data on individual Instagram behavior, helping researchers assess the validity of surveying self-reported online behaviors. Third, we illustrate the potential of data donation to gather detailed, fine-grained data on individual behaviors, which participants might be unable to report accurately.
