TY - JOUR
T1 - Comparative Study of Confidence Intervals for Proportions in Complex Sample Surveys
AU - Franco, Carolina
AU - Little, Roderick J.A.
AU - Louis, Thomas A.
AU - Slud, Eric V.
N1 - Funding Information:
CAROLINA FRANCO and ERIC V. SLUD are Mathematical Statisticians with the Center for Statistical Research and Methodology (CSRM), US Census Bureau, 4600 Silver Hill Road, Washington DC 20233, USA. ERIC V. SLUD is also Professor, Mathematics Department, University of Maryland, College Park, MD 20742, USA. RODERICK J. A. LITTLE is Richard D. Remington Distinguished University Professor with the Department of Biostatistics, University of Michigan, 1415 Washington Heights, Ann Arbor, MI 48109, USA. THOMAS A. LOUIS is Professor Emeritus with the Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Baltimore, MD 02215, USA. This work was supported partially by the funding provided by International Centers of Excellence for Malaria Research: “Malaria Transmission and the Impact of Control Efforts in Southern Africa.” NIH-NIAID, U19-AI089680. *Address correspondence to Carolina Franco; E-mail: carolina.franco@census.gov.
Publisher Copyright:
© 2019 The Author(s). Published by Oxford University Press on behalf of the American Association for Public Opinion Research. All rights reserved.
PY - 2019/9/1
Y1 - 2019/9/1
N2 - The most widespread method of computing confidence intervals (CIs) in complex surveys is to add and subtract the margin of error (MOE) from the point estimate, where the MOE is the estimated standard error multiplied by the suitable Gaussian quantile. This Wald-type interval is used by the American Community Survey (ACS), the largest US household sample survey. For inferences on small proportions with moderate sample sizes, this method often results in marked under-coverage and lower CI endpoint less than 0. We assess via simulation the coverage and width, in complex sample surveys, of seven alternatives to the Wald interval for a binomial proportion with sample size replaced by the 'effective sample size,' that is, the sample size divided by the design effect. Building on previous work by the present authors, our simulations address the impact of clustering, stratification, different stratum sampling fractions, and stratum-specific proportions. We show that all intervals undercover when there is clustering and design effects are computed from a simple design-based estimator of sampling variance. Coverage can be better calibrated for the alternatives to Wald by improving estimation of the effective sample size through superpopulation modeling. This approach is more effective in our simulations than previously proposed modifications of effective sample size. We recommend intervals of the Wilson or Bayes uniform prior form, with the Jeffreys prior interval not far behind.
AB - The most widespread method of computing confidence intervals (CIs) in complex surveys is to add and subtract the margin of error (MOE) from the point estimate, where the MOE is the estimated standard error multiplied by the suitable Gaussian quantile. This Wald-type interval is used by the American Community Survey (ACS), the largest US household sample survey. For inferences on small proportions with moderate sample sizes, this method often results in marked under-coverage and lower CI endpoint less than 0. We assess via simulation the coverage and width, in complex sample surveys, of seven alternatives to the Wald interval for a binomial proportion with sample size replaced by the 'effective sample size,' that is, the sample size divided by the design effect. Building on previous work by the present authors, our simulations address the impact of clustering, stratification, different stratum sampling fractions, and stratum-specific proportions. We show that all intervals undercover when there is clustering and design effects are computed from a simple design-based estimator of sampling variance. Coverage can be better calibrated for the alternatives to Wald by improving estimation of the effective sample size through superpopulation modeling. This approach is more effective in our simulations than previously proposed modifications of effective sample size. We recommend intervals of the Wilson or Bayes uniform prior form, with the Jeffreys prior interval not far behind.
KW - Bayesian formalism
KW - Complex surveys
KW - Confidence interval for proportion
KW - Design effect
KW - Effective sample size
UR - http://www.scopus.com/inward/record.url?scp=85084352766&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85084352766&partnerID=8YFLogxK
U2 - 10.1093/jssam/smy019
DO - 10.1093/jssam/smy019
M3 - Article
C2 - 31428658
AN - SCOPUS:85084352766
SN - 2325-0984
VL - 7
SP - 334
EP - 364
JO - Journal of Survey Statistics and Methodology
JF - Journal of Survey Statistics and Methodology
IS - 3
ER -