From Start to Pilot: A Multi-method Approach to the Comprehensive Redesign of an Economic Survey Questionnaire
Alfred D. Tuttle, Rebecca L. Morrison, Diane K. Willimack
Survey methodologists at the U.S. Census Bureau assisted the redesign of an establishment survey questionnaire administered by the U.S. Bureau of Economic Analysis. Initial investigations indicated that the questionnaire was problematic for some respondents due to the dense, compact formatting of the questionnaire, the placement of instructions in a separate booklet, and the subtle but crucial distinctions between the survey concepts and respondents typical frames of reference. Sections of the questionnaire were redesigned with an eye toward ensuring respondents read these important instructions, and iteratively tested across five rounds of cognitive interviews, with results informing modifications that were retested in subsequent rounds. A complete pilot form was mailed to a subsample of companies, and evaluated via questions included in the form and respondent debriefings. The various methods led to findings that confirmed and built upon each other, allowing for thorough evaluation of the design strategies employed in the new form.
Establishment survey, pretesting, qualitative methods, iterative development, questionnaire evaluation