Assessing the mechanisms of misreporting to filter questions in surveys

To avoid asking respondents questions that do not apply to them, surveys often use filter questions that determine routing into follow-up items. Filter questions can be asked in an interleafed format, in which follow-up questions are asked immediately after each relevant filter, or in a grouped format, in which follow-up questions are asked only after multiple filters have been administered. Most previous investigations of filter questions have found that the grouped format collects more affirmative answers than the interleafed format. This result has been taken to mean that respondents in the interleafed format learn to shorten the questionnaire by answering the filter questions negatively. However, this is only one mechanism that could produce the observed differences between the two formats. Acquiescence, the tendency to answer yes to yes/no questions, could also explain the results. We conducted a telephone survey that linked filter question responses to high-quality administrative data to test two hypotheses about the mechanism of the format effect. We find strong support for motivated underreporting and less support for the acquiescence hypothesis. This is the first clear evidence that the grouped format results in more accurate answers to filter questions. However, we also find that the underreporting phenomenon does not always occur. These findings are relevant to all surveys that use multiple filter questions.