Hierarchical statistical models to represent and visualize survey evidence for program evaluation: ICCM in Malawi

Policy and Program evaluation for maternal, newborn and child health is becoming increasingly complex due to changing contexts. Monitoring and evaluation efforts in this area can take advantage of large nationally representative household surveys such as DHS or MICS that are increasing in size and frequency, however, this analysis presents challenges on several fronts. We propose an approach with hierarchical models for cross-sectional survey data to describe evidence relating to program evaluation, and apply this approach to the recent scale up of iCCM in Malawi. We describe careseeking for children sick with diarrhea, pneumonia, or malaria with empirical Bayes estimates for each district of Malawi at two time points, both for careseeking from any source, and for careseeking only from health surveillance assistants (HSA). We do not find evidence that children in areas with more HSA trained in iCCM are more likely to seek care for pneumonia, diarrhea, or malaria, despite evidence that many indeed are seeking care from HSA. Children in areas with more HSA trained in iCCM are more likely to seek care from a HSA, with 100 additional trained health workers in a district corresponding to a 2% average increase in careseeking from HSA. The hierarchical models presented here provide a flexible set of methods that describe the primary evidence for evaluating iCCM in Malawi and which could be extended to formal causal analyses, and to analysis for other similar evaluations with national survey data.