Accounting for Biases in Election Surveys: The Case of the 1998 Quebec Election
Claire Durand Andre Blais and Sebastien Vachon
During the last electoral campaign in Quebec, Canada, all the polls published in the media had a similar estimate of vote intentions, putting the Parti Quebecois (PQ), a centre-left party dedicated to Quebec sovereignty, clearly ahead, by an average of five points in the last six polls of the campaign. The PQ won the election, held on November 30, 1998, but with a smaller share of the vote (43 per cent) than the contending Liberal party (44 per cent), a centre-right federalist party. Pollsters and many observers have maintained that the discrepancy between the polls and the actual vote could be explained either by a last minute shift in favour of the Quebec Liberal party or by differential turnout.
We rely on a number of data sources to sort out the possible causes of such a discrepancy. A post-election poll was conducted among fifteen hundred respondents of pre-election polls conducted by two pollsters, CROP and CREATEC. Three surveys carried out by CROP during the four-week campaign were analyzed in order to estimate the impact of item and unit nonresponse and of adjustment using Census data. A study of voting sections with a high percentage of institutions allows us to estimate the voting behaviour of residents of such institutions. Two STATMEDIA studies conducted in 1997 and 1998 provide information on the sociodemographic characteristics of respondents from unlisted and doubly listed telephone lines. Finally, three CROP surveys carried out after the election allow us to compare the voting intentions of respondents from listed and unlisted telephone numbers.
The results of the post-election survey do not support the late shift and differential turnout hypotheses. The most likely explanations for the discrepancy between vote intentions and the actual vote are to be found in survey nonresponse, in sampling frame biases and in the adjustment scheme. Analysis of nonresponse shows that there is a consistent tendency for those who refuse to answer surveys to be supporters of the Liberal party. An analysis of sampling frames shows that Liberal supporters are undersampled because of the absence of respondents living in institutions and of households with unlisted telephone numbers. Finally, the fact that survey firms adjust on the basis of age may contribute slightly to the underestimation of Liberal support.
Electoral surveys; survey nonresponse; sampling bias; coverage error; late campaign shift.