Key Design Considerations When Calculating Cost Savings for Population Health Management Programs in an Observational Setting

Objective: To illustrate the impact of key quasi-experimental design elements on cost savings measurement for population health management (PHM) programs. Data Sources: Population health management program records and Medicaid claims and enrollment data from December 2011 through March 2016. Study Design: The study uses a difference-in-difference design to compare changes in cost and utilization outcomes between program participants and propensity score-matched nonparticipants. Comparisons of measured savings are made based on (1) stable versus dynamic population enrollment and (2) all eligible versus enrolled-only participant definitions. Options for the operationalization of time are also discussed. Data Collection/Extraction Methods: Individual-level Medicaid administrative and claims data and PHM program records are used to match study groups on baseline risk factors and assess changes in costs and utilization. Principal Findings: Savings estimates are statistically similar but smaller in magnitude when eliminating variability based on duration of population enrollment and when evaluating program impact on the entire target population. Measurement in calendar time, when possible, simplifies interpretability. Conclusion: Program evaluation design elements, including population stability and participant definitions, can influence the estimated magnitude of program savings for the payer and should be considered carefully. Time specifications can also affect interpretability and usefulness.