A proposed model to conduct process and outcome evaluations and implementation research of child health programs in Africa using integrated community case management as an example

The iCCM Symposium impact outcome evaluation thematic group

Research output: Contribution to journalArticle

Abstract

Aim To use a newly devised set of criteria to review the study design and scope of collection of process, outcomes and contextual data for evaluations and implementation research of integrated community case management (iCCM) in Sub-Saharan African. Methods We examined 24 program evaluations and implementation research studies of iCCM in sub-Saharan Africa conducted in the last 5 years (2008-2013), assessed the design used and categorized them according to whether or not they collected sufficient information to conduct process and outcome evaluations. Results Five of the 24 studies used a stepped wedge design and two were randomized control trials. The remaining 17 were quasi-experimental of which 10 had comparison areas; however, not all comparison areas had a pre and post household survey. With regard to process data, 22 of the studies collected sufficient information to report on implementation strength, and all, except one, could report on program implementation. The most common missing data elements were health facility treatments, service costs, and qualitative data to assess demand. For the measurement of program outcomes, 7 of the 24 studies had a year or less of implementation at scale before the endline survey, 6 of the household surveys did not collect point of service, 10 did not collect timeliness (care seeking within 24 hours of symptoms) and 12 did not have socioeconomic (SES) information. Among the 16 studies with comparison areas, only 5 randomly selected comparison areas, while 10 had appropriate comparison areas. Conclusions Several evaluations were done too soon after implementation, lacked information on health facility treatments, costs, demand, timeliness or SES and/or did not have a counterfactual. We propose several study designs and minimal data elements to be collected to provide sufficient information to assess whether iCCM increased timely coverage of treatment for the neediest children in a cost-efficient manner.

Original languageEnglish (US)
Article number020409
JournalJournal of global health
Volume4
Issue number2
DOIs
StatePublished - Jan 1 2014

Fingerprint

Case Management
Health Facilities
Health Care Costs
Africa South of the Sahara
Program Evaluation
Costs and Cost Analysis
Child Health
Surveys and Questionnaires
Therapeutics

ASJC Scopus subject areas

  • Health Policy
  • Public Health, Environmental and Occupational Health

Cite this

A proposed model to conduct process and outcome evaluations and implementation research of child health programs in Africa using integrated community case management as an example. / The iCCM Symposium impact outcome evaluation thematic group.

In: Journal of global health, Vol. 4, No. 2, 020409, 01.01.2014.

Research output: Contribution to journalArticle

@article{164cec236f514636a81f14b622ad7cc4,
title = "A proposed model to conduct process and outcome evaluations and implementation research of child health programs in Africa using integrated community case management as an example",
abstract = "Aim To use a newly devised set of criteria to review the study design and scope of collection of process, outcomes and contextual data for evaluations and implementation research of integrated community case management (iCCM) in Sub-Saharan African. Methods We examined 24 program evaluations and implementation research studies of iCCM in sub-Saharan Africa conducted in the last 5 years (2008-2013), assessed the design used and categorized them according to whether or not they collected sufficient information to conduct process and outcome evaluations. Results Five of the 24 studies used a stepped wedge design and two were randomized control trials. The remaining 17 were quasi-experimental of which 10 had comparison areas; however, not all comparison areas had a pre and post household survey. With regard to process data, 22 of the studies collected sufficient information to report on implementation strength, and all, except one, could report on program implementation. The most common missing data elements were health facility treatments, service costs, and qualitative data to assess demand. For the measurement of program outcomes, 7 of the 24 studies had a year or less of implementation at scale before the endline survey, 6 of the household surveys did not collect point of service, 10 did not collect timeliness (care seeking within 24 hours of symptoms) and 12 did not have socioeconomic (SES) information. Among the 16 studies with comparison areas, only 5 randomly selected comparison areas, while 10 had appropriate comparison areas. Conclusions Several evaluations were done too soon after implementation, lacked information on health facility treatments, costs, demand, timeliness or SES and/or did not have a counterfactual. We propose several study designs and minimal data elements to be collected to provide sufficient information to assess whether iCCM increased timely coverage of treatment for the neediest children in a cost-efficient manner.",
author = "{The iCCM Symposium impact outcome evaluation thematic group} and Theresa Diaz and Tanya Guenther and Oliphant, {Nicholas P.} and Maria Mu{\~n}iz and Yolanda Barbera and Agbessi Amouzou and Franco Pagnoni and Abigail Pratt and Saul Morris and Helen Counihan and David Collins and Zina Jarrah and Daniel Kadobera and Elizeus Rutebemberwa and Mohamadou Siribie and Sirima, {Sodiomon Bienvenu} and Chinbuah, {Margaret Amanua} and Gyapong, {John O.} and Paulin Basinga and Tanya Doherty and Hamer, {Davidson H.} and Kojo Yeboah-Antwi",
year = "2014",
month = "1",
day = "1",
doi = "10.7189/jogh.04.020409",
language = "English (US)",
volume = "4",
journal = "Journal of Global Health",
issn = "2047-2978",
publisher = "Edinburgh University Global Health Society",
number = "2",

}

TY - JOUR

T1 - A proposed model to conduct process and outcome evaluations and implementation research of child health programs in Africa using integrated community case management as an example

AU - The iCCM Symposium impact outcome evaluation thematic group

AU - Diaz, Theresa

AU - Guenther, Tanya

AU - Oliphant, Nicholas P.

AU - Muñiz, Maria

AU - Barbera, Yolanda

AU - Amouzou, Agbessi

AU - Pagnoni, Franco

AU - Pratt, Abigail

AU - Morris, Saul

AU - Counihan, Helen

AU - Collins, David

AU - Jarrah, Zina

AU - Kadobera, Daniel

AU - Rutebemberwa, Elizeus

AU - Siribie, Mohamadou

AU - Sirima, Sodiomon Bienvenu

AU - Chinbuah, Margaret Amanua

AU - Gyapong, John O.

AU - Basinga, Paulin

AU - Doherty, Tanya

AU - Hamer, Davidson H.

AU - Yeboah-Antwi, Kojo

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Aim To use a newly devised set of criteria to review the study design and scope of collection of process, outcomes and contextual data for evaluations and implementation research of integrated community case management (iCCM) in Sub-Saharan African. Methods We examined 24 program evaluations and implementation research studies of iCCM in sub-Saharan Africa conducted in the last 5 years (2008-2013), assessed the design used and categorized them according to whether or not they collected sufficient information to conduct process and outcome evaluations. Results Five of the 24 studies used a stepped wedge design and two were randomized control trials. The remaining 17 were quasi-experimental of which 10 had comparison areas; however, not all comparison areas had a pre and post household survey. With regard to process data, 22 of the studies collected sufficient information to report on implementation strength, and all, except one, could report on program implementation. The most common missing data elements were health facility treatments, service costs, and qualitative data to assess demand. For the measurement of program outcomes, 7 of the 24 studies had a year or less of implementation at scale before the endline survey, 6 of the household surveys did not collect point of service, 10 did not collect timeliness (care seeking within 24 hours of symptoms) and 12 did not have socioeconomic (SES) information. Among the 16 studies with comparison areas, only 5 randomly selected comparison areas, while 10 had appropriate comparison areas. Conclusions Several evaluations were done too soon after implementation, lacked information on health facility treatments, costs, demand, timeliness or SES and/or did not have a counterfactual. We propose several study designs and minimal data elements to be collected to provide sufficient information to assess whether iCCM increased timely coverage of treatment for the neediest children in a cost-efficient manner.

AB - Aim To use a newly devised set of criteria to review the study design and scope of collection of process, outcomes and contextual data for evaluations and implementation research of integrated community case management (iCCM) in Sub-Saharan African. Methods We examined 24 program evaluations and implementation research studies of iCCM in sub-Saharan Africa conducted in the last 5 years (2008-2013), assessed the design used and categorized them according to whether or not they collected sufficient information to conduct process and outcome evaluations. Results Five of the 24 studies used a stepped wedge design and two were randomized control trials. The remaining 17 were quasi-experimental of which 10 had comparison areas; however, not all comparison areas had a pre and post household survey. With regard to process data, 22 of the studies collected sufficient information to report on implementation strength, and all, except one, could report on program implementation. The most common missing data elements were health facility treatments, service costs, and qualitative data to assess demand. For the measurement of program outcomes, 7 of the 24 studies had a year or less of implementation at scale before the endline survey, 6 of the household surveys did not collect point of service, 10 did not collect timeliness (care seeking within 24 hours of symptoms) and 12 did not have socioeconomic (SES) information. Among the 16 studies with comparison areas, only 5 randomly selected comparison areas, while 10 had appropriate comparison areas. Conclusions Several evaluations were done too soon after implementation, lacked information on health facility treatments, costs, demand, timeliness or SES and/or did not have a counterfactual. We propose several study designs and minimal data elements to be collected to provide sufficient information to assess whether iCCM increased timely coverage of treatment for the neediest children in a cost-efficient manner.

UR - http://www.scopus.com/inward/record.url?scp=84960888784&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84960888784&partnerID=8YFLogxK

U2 - 10.7189/jogh.04.020409

DO - 10.7189/jogh.04.020409

M3 - Article

AN - SCOPUS:84960888784

VL - 4

JO - Journal of Global Health

JF - Journal of Global Health

SN - 2047-2978

IS - 2

M1 - 020409

ER -