Methods and challenges in measuring the impact of national pneumococcal and rotavirus vaccine introduction on morbidity and mortality in Malawi

Background: Pneumonia and gastroenteritis are leading causes of vaccine-preventable childhood morbidity and mortality. Malawi introduced pneumococcal conjugate and rotavirus vaccines to the immunisation programme in 2011 and 2012, respectively. Evaluating their effectiveness is vital to ensure optimal implementation and justify sustained investment. Methods/Design: A national evaluation platform was established to determine vaccine effectiveness and impact in Malawi. Impact and effectiveness against vaccine-type invasive pneumococcal disease, radiological pneumonia and rotavirus gastroenteritis are investigated using before-after incidence comparisons and case-control designs, respectively. Mortality is assessed using a prospective population cohort. Cost-effectiveness evaluation is nested within the case-control studies. We describe platform characteristics including strengths and weaknesses for conducting vaccine evaluations. Discussion: Integrating data from individual level and ecological methods across multiple sites provides comprehensive information for policymakers on programme impact and vaccine effectiveness including changes in serotype/genotype distribution over time. Challenges to robust vaccine evaluation in real-world conditions include: vaccination ascertainment; pre-existing rapid decline in mortality and pneumococcal disease in the context of non-vaccine interventions; and the maintenance of completeness and quality of reporting at scale and over time. In observational non-randomised designs ascertainment of vaccine status may be biased particularly in infants with fatal outcomes. In the context of multiple population level interventions targeting study endpoints attribution of reduced incidence to vaccine impact may be flawed. Providing evidence from several independent but complementary studies will provide the greatest confidence in assigning impact. Welcome declines in disease incidence and in child mortality make accrual of required sample sizes difficult, necessitating large studies to detect the relatively small but potentially significant contribution of vaccines to mortality prevention. Careful evaluation of vaccine effectiveness and impact in such settings is critical to sustaining support for vaccine programmes. Our evaluation platform covers a large population with a high prevalence of HIV and malnutrition and its findings will be relevant to other settings in sub-Saharan Africa.