Estimation of Nonresponse Bias in the European Social Survey: Using Information from Reluctant Respondents
Jaak Billiet, Michel Philippens, Rory Fitzgerald, Ineke Stoop
Central to the methodological quality of the first round of the European Social Survey (ESS) was the principle of equivalence in cross-national measurement. The survey was therefore designed with equivalence as its driving force and included features such as the requirement for random probability samples, effective sample sizes, clear specifications for fieldwork institutes, clear rules for interviewers about the mode, number and timing of contact attempts with all sample units and the documentation of all contact attempts using standardised forms. The use of standardised, detailed contact forms has enabled equivalent cross-national comparisons of nonresponse as well as providing some indication of the potential bias in survey estimates. This article seeks to uncover “traces of bias,” that occur as a consequence of nonresponse, by comparing cooperative and more reluctant respondents on different substantive survey estimates. The analytical framework of the article is based upon the assumption that the attitudes of nonrespondents are more like those of reluctant than cooperative respondents. This is tested by analysing the Round 1 contact form data along with substantive data from the main survey questionnaire.
The data were analysed to answer the question whether nonresponse bias was likely to be affecting the parameters in substantive explanatory models for various attitudinal variables. The findings are mixed in this respect. By classifying respondents into “type of respondent,” based upon how easily they agreed to participate in the survey the potential effect of nonresponse on survey estimates was examined. In some of the explanatory models the effect of the “type of respondent” disappeared, but this was not always the case. However, the remaining (significant) effects are small, and do not have serious implications for the parameters of the explanatory variables. This rather optimistic view must be treated with some caution because of the possibility that real “hard” refusals that were not converted may differ from the converted refusals. More research across a range of surveys is needed about this issue.
Data quality assessment, cross-national surveys, measurement error, European Social Survey