Methods for assessing quality of provider performance in developing countries

Objective. To compare the effectiveness of methods for assessing the quality of pediatric outpatient health provider performance in developing countries. Design. Exit interviews, record reviews, and provider interview results were compared with those of direct observation of pediatric patient care. Thirty health care providers in 14 facilities in Lilongwe District, Malawi were interviewed and observed, treating 436 children in August 1994. Caretakers for 426 of the patients were interviewed, and 362 pediatric outpatient entries in the health center patient register were located and reviewed. Main measures. Kappa statistics measuring the level of agreement on the same sample were used for three methods (record reviews, provider interviews, and exit interviews) in comparison with the fourth method, direct observation. Results. All three methods had strengths and weaknesses. Exit interviews with caretakers provided reliable responses for many history-taking tasks, easily discernible physical exam tasks, and many counseling tasks. Record review took little time, but provided limited information: however, the results were reliable for treatments. Provider interviews had much lower reliability, but were usable for assessing more rare events (treating severely ill children). Conclusions. Although exit interviews and direct observation provide the 'best' data, they are most resource-intensive. Depending on the purpose of the assessment, various combinations of methods might be more effective.