Using Propensity Scores for Nonresponse Adjustment with Covariate Measurement Error

The aim of the project was to advance knowledge about the use of propensity scores for nonresponse adjustment when measurement error is present in the covariates used for adjustment. Addressing the issue of measurement errors in nonresponse adjustment variables affects population estimates of key statistics, spanning a wide range of topics, such as welfare recipiency, reproductive behaviour, and health. The project used simulations to examine the consequences of covariate measurement error for nonresponse adjustments, performed secondary data analyses to investigate the amount and structure of measurement error present in paradata, collected through interviewers during the survey response process, and their predictive power on data quality, and developed a conceptual framework about the utility of different data sources given self-selection and other nonresponse biases beyond nonresponse adjustments via propensity scores. The Johns Hopkins team of the project, lead by Elizabeth Stuart, found that causal effect estimates are less biased when the propensity score model includes mismeasured covariates whose true underlying values are strongly correlated with each other. However, when the measurement errors are correlated with each other, additional bias is introduced. In addition, it is beneficial to include correctly measured auxiliary variables that are connected to confounders whose true underlying values are mismeasured in the propensity score model. The empirical investigation of latent class analyses of several post-survey interviewer observations from two major national surveys showed that interviewer observations are adequate indirect indicators of data quality (West et al. 2020). Our work published in 2019 in the Annual Reviews of Statistics and Its Application discussed the usability of samples with unknown selection probabilities for various research questions. It also includes a review of assumptions necessary for descriptive and causal inference as well as a discussion of research strategies developed to overcome sampling limitations.