Learner feedback and educational outcomes with an internet-based ambulatory curriculum: A qualitative and quantitative analysis

Background: Online medical education curricula offer new tools to teach and evaluate learners. The effect on educational outcomes of using learner feedback to guide curricular revision for online learning is unknown. Methods. In this study, qualitative analysis of learner feedback gathered from an online curriculum was used to identify themes of learner feedback, and changes to the online curriculum in response to this feedback were tracked. Learner satisfaction and knowledge gains were then compared from before and after implementation of learner feedback. Results: 37,755 learners from 122 internal medicine residency training programs were studied, including 9437 postgraduate year (PGY)1 residents (24.4% of learners), 9864 PGY2 residents (25.5%), 9653 PGY3 residents (25.0%), and 6605 attending physicians (17.0%). Qualitative analysis of learner feedback on how to improve the curriculum showed that learners commented most on the overall quality of the educational content, followed by specific comments on the content. When learner feedback was incorporated into curricular revision, learner satisfaction with the instructive value of the curriculum (1=not instructive; 5=highly instructive) increased from 3.8 to 4.1 (p<0.001), and knowledge gains (i.e., post test scores minus pretest scores) increased from 17.0% to 20.2% (p<0.001). Conclusions: Learners give more feedback on the factual content of a curriculum than on other areas such as interactivity or website design. Incorporating learner feedback into curricular revision was associated with improved educational outcomes. Online curricula should be designed to include a mechanism for learner feedback and that feedback should be used for future curricular revision.