More and more surveys are conducted online. While web surveys are generally cheaper and
tend to have lower measurement error in comparison to other survey modes, especially for
sensitive questions, potential advantages might be offset by larger nonresponse bias. This
article compares the data quality in a web survey administration to another common mode of
survey administration, the telephone.
The unique feature of this study is the availability of administrative records for all sampled
individuals in combination with a random assignment of survey mode. This specific design
allows us to investigate and compare potential bias in survey statistics due to 1) nonresponse
error, 2) measurement error, and 3) combined bias of these two error sources and hence, an
overall assessment of data quality for two common modes of survey administration, telephone
and web.
Our results show that overall mean estimates on the web are more biased compared to the
telephone mode. Nonresponse and measurement bias tend to reinforce each other in both
modes, with nonresponse bias being somewhat more pronounced in the web mode. While
measurement error bias tends to be smaller in the web survey implementation, interestingly,
our results also show that the web does not consistently outperform the telephone mode for
sensitive questions.
