The Effect of Multiple Weighting Steps on Variance Estimation
Multiple weight adjustments are common in surveys to account for ineligible units on a frame, nonresponse by some units, and the use of auxiliary data in estimation. A practical question is whether all of these steps need to be accounted for when estimating variances. Linearization variance estimators and related estimators in commercial software packages that use squared residuals usually account only for the last step in estimation, which is the incorporation of auxiliary data through poststratification, regression estimation, or similar methods. Replication variance estimators can explicitly account for all of the steps in estimation by repeating each adjustment separately for each replicate subsample. Through simulation, this article studies the difference in these methods for some specific sample designs, estimators of totals, and rates of ineligibility and nonresponse. In the simulations reported here, the linearization variance estimators are negatively biased and produce confidence intervals for a population total that cover at less than the nominal rate, especially at smaller sample sizes. The jackknife replication estimator generally yields confidence intervals that cover at or above the nominal rate but do so at the expense of considerably overestimating empirical mean squared errors. A leverage-adjusted variance estimator, which is related to the jackknife estimator, has small positive bias and nearly nominal coverage. The leverage-adjusted estimator is less computationally burdensome than the jackknife but works well in the situations studied here where multiple weighting steps are used.
Ineligibility, jackknife, leverage, nonresponse, replication variance estimate