Comparative Study of Confidence Intervals for Proportions in Complex Sample Surveys

Carolina Franco, Roderick J.A. Little, Thomas A. Louis, Eric V. Slud

Research output: Contribution to journalArticlepeer-review

Abstract

The most widespread method of computing confidence intervals (CIs) in complex surveys is to add and subtract the margin of error (MOE) from the point estimate, where the MOE is the estimated standard error multiplied by the suitable Gaussian quantile. This Wald-type interval is used by the American Community Survey (ACS), the largest US household sample survey. For inferences on small proportions with moderate sample sizes, this method often results in marked under-coverage and lower CI endpoint less than 0. We assess via simulation the coverage and width, in complex sample surveys, of seven alternatives to the Wald interval for a binomial proportion with sample size replaced by the 'effective sample size,' that is, the sample size divided by the design effect. Building on previous work by the present authors, our simulations address the impact of clustering, stratification, different stratum sampling fractions, and stratum-specific proportions. We show that all intervals undercover when there is clustering and design effects are computed from a simple design-based estimator of sampling variance. Coverage can be better calibrated for the alternatives to Wald by improving estimation of the effective sample size through superpopulation modeling. This approach is more effective in our simulations than previously proposed modifications of effective sample size. We recommend intervals of the Wilson or Bayes uniform prior form, with the Jeffreys prior interval not far behind.

Original languageEnglish (US)
Pages (from-to)334-364
Number of pages31
JournalJournal of Survey Statistics and Methodology
Volume7
Issue number3
DOIs
StatePublished - Sep 1 2019

Keywords

  • Bayesian formalism
  • Complex surveys
  • Confidence interval for proportion
  • Design effect
  • Effective sample size

ASJC Scopus subject areas

  • Statistics and Probability
  • Social Sciences (miscellaneous)
  • Statistics, Probability and Uncertainty
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Comparative Study of Confidence Intervals for Proportions in Complex Sample Surveys'. Together they form a unique fingerprint.

Cite this