Quality of Survey Measures: A Structural Modeling Approach
Willard L. Rodgers, Frank M. Andrews, and A. Regula Herzog
Estimates of the quality of about one hundred survey measures, broadly representative of those commonly used in survey research, were obtained by specifying several multitrait-multimethod matrices and estimating the parameters of measurement models. Specifically, the total variance of each measure in such a matrix was allocated to (1) a concept factor (to provide an estimate of construct validity); (2) a method factor; and (3) the residual. These estimates of data quality were then analyzed to identify characteristics of survey designs that are associated with more accurate responses (e.g., with higher loadings on the concept factors). The most important measure characteristic was the number of response alternatives offered. The “unfolding” of response alternatives was also associated with a higher level of construct validity, as was the explicit provision of a frame of reference for the question. Differences in data quality between subgroups of respondents defined by age, education, and a variety of other characteristics were statistically non-significant for the measures examined. Data are from face-to-face interviews conducted with an area probability sample of about 1,500 persons.
Data quality; construct validity; response biases.