TY - JOUR
T1 - Validation studies for population-based intervention coverage indicators
T2 - Design, analysis, and interpretation
AU - for the Improving Coverage Measurement Group
AU - Munos, Melinda K.
AU - Blanc, Ann K.
AU - Carter, Emily D.
AU - Eisele, Thomas P.
AU - Gesuale, Steve
AU - Katz, Joanne
AU - Marchant, Tanya
AU - Stanton, Cynthia K.
AU - Campbell, Harry
N1 - Funding Information:
Funding: This work was supported by the Improving Coverage Measurement (ICM) grant from the Bill & Me-linda Gates Foundation (OPP1084442). Authorship contributions: Conceptualized the paper: HC, AKB, TPE, CKS, MKM, JK, TM, FA, SMK. Wrote or critically revised the paper: MKM, AKB, TPE, JK, TM, CKS, EC, SG, SMK, AH, FA and HC. Agree with manuscript and conclusions: MKM, AKB, TPE, JK, TM, CKS, EC, SG, FA, SMK, AH, JG and HC. Competing interests: Harry Campbell is the Co-Editor in Chief of the Journal of Global Health. To ensure that any possible conflict of interest relevant to the journal has been addressed, this article was reviewed according to best practice guidelines of international editorial organisations. All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf (available upon request from the corresponding author) and declare no conflicts of interest.
Publisher Copyright:
© 2018, Journal of Health Global.
PY - 2018
Y1 - 2018
N2 - Background Population-based intervention coverage indicators are widely used to track country and program progress in improving health and to evaluate health programs. Indicator validation studies that compare survey responses to a "gold standard" measure are useful to understand whether the indicator provides accurate information. The Improving Coverage Measurement (ICM) Core Group has developed and implemented a standard approach to validating coverage indicators measured in household surveys, described in this paper. Methods The general design of these studies includes measurement of true health status and intervention receipt (gold standard), followed by interviews with the individuals observed, and a comparison of the observations (gold standard) to the responses to survey questions. The gold standard should use a data source external to the respondent to document need for and receipt of an intervention. Most frequently, this is accomplished through direct observation of clinical care, and/or use of a study-trained clinician to obtain a gold standard diagnosis. Follow-up interviews with respondents should employ standard survey questions, where they exist, as well as alternative or additional questions that can be compared against the standard household survey questions. Results Indicator validation studies should report on participation at every stage, and provide data on reasons for non-participation. Metrics of individual validity (sensitivity, specificity, area under the receiver operating characteristic curve) and population-level validity (inflation factor) should be reported, as well as the percent of survey responses that are "don't know" or missing. Associations between interviewer and participant characteristics and measures of validity should be assessed and reported. Conclusions These methods allow respondent-reported coverage measures to be validated against more objective measures of need for and receipt of an intervention, and should be considered together with cognitive interviewing, discriminative validity, or reliability testing to inform decisions about which indicators to include in household surveys. Public health researchers should assess the evidence for validity of existing and proposed household survey coverage indicators and consider validation studies to fill evidence gaps.
AB - Background Population-based intervention coverage indicators are widely used to track country and program progress in improving health and to evaluate health programs. Indicator validation studies that compare survey responses to a "gold standard" measure are useful to understand whether the indicator provides accurate information. The Improving Coverage Measurement (ICM) Core Group has developed and implemented a standard approach to validating coverage indicators measured in household surveys, described in this paper. Methods The general design of these studies includes measurement of true health status and intervention receipt (gold standard), followed by interviews with the individuals observed, and a comparison of the observations (gold standard) to the responses to survey questions. The gold standard should use a data source external to the respondent to document need for and receipt of an intervention. Most frequently, this is accomplished through direct observation of clinical care, and/or use of a study-trained clinician to obtain a gold standard diagnosis. Follow-up interviews with respondents should employ standard survey questions, where they exist, as well as alternative or additional questions that can be compared against the standard household survey questions. Results Indicator validation studies should report on participation at every stage, and provide data on reasons for non-participation. Metrics of individual validity (sensitivity, specificity, area under the receiver operating characteristic curve) and population-level validity (inflation factor) should be reported, as well as the percent of survey responses that are "don't know" or missing. Associations between interviewer and participant characteristics and measures of validity should be assessed and reported. Conclusions These methods allow respondent-reported coverage measures to be validated against more objective measures of need for and receipt of an intervention, and should be considered together with cognitive interviewing, discriminative validity, or reliability testing to inform decisions about which indicators to include in household surveys. Public health researchers should assess the evidence for validity of existing and proposed household survey coverage indicators and consider validation studies to fill evidence gaps.
UR - http://www.scopus.com/inward/record.url?scp=85053115531&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85053115531&partnerID=8YFLogxK
U2 - 10.7189/jogh.08.020804
DO - 10.7189/jogh.08.020804
M3 - Article
C2 - 30202519
AN - SCOPUS:85053115531
SN - 2047-2978
VL - 8
JO - Journal of global health
JF - Journal of global health
IS - 2
M1 - 020804
ER -