In empirical survey studies, finding a sufficient number of people who are willing to respond can be challenging – especially in countries where the response rate is low (Engel & Schmidt 2019). For factorial survey experiments, drawing a vignette-sample (“fraction”) from a vignette-universe can reduce the minimum number of respondents needed for studies.
Vignette-samples can be drawn by means of quota designs, including D-efficient designs. In marketing research, resolution III D-efficient designs (also termed “orthogonal arrays”) are mostly used (Kuhfeld 2003). In the social sciences, on the other hand, one should also consider possible two-way interactions that might have an effect. Theoretically, resolution V designs are thus ideal. Due to reasons of practicability, however, resolution IV designs have usually been applied in empirical social research and are considered to be sufficient when it is clear up front which two-way interactions are likely to have an effect.
Two research questions, oriented towards further examining the resolutions of D-efficient designs, guide this presentation: First, in resolution IV designs, are those two-way interactions that are not orthogonalized truly not aliased with any main effects? Second, how does resolution – III, IV (with 1, 3 or 5 two-way interactions orthogonalized) and V – affect the minimum size of the vignette-sample that is necessary to still achieve an adequate level of D-efficiency?
In order to examine these questions, I use SAS-macros written by Kuhfeld (2003) for computing D-efficient samples, pre-construction assessment and post-construction evaluation. This should contribute to a deeper understanding of (factorial survey) experimental setups based on D-efficiency, which have ramifications for the analysis. Additionally, it suggests taking a second look at Kuhfeld’s (e.g. 2003) assumption that higher resolutions will always necessitate designs with larger vignette-samples (and thus larger sets or more respondents).
References:
Engel, U., & Schmidt, B. O. (2019). Unit- und Item-Nonresponse. In N. Baur & J. Blasius (Eds.), Handbuch Methoden der empirischen Sozialforschung (pp. 385-404). Springer VS, Wiesbaden.
Kuhfeld, W. F. (2003). Marketing research methods in SAS: Experimental design, choice, conjoint, and graphical techniques. Cary, NC: SAS Institute Inc.
