Milestone Implementation’s Impact on Narrative Comments and Perception of Feedback for Internal Medicine Residents: a Mixed Methods Study

Sonja E. Raaum, Katie Lappe, Jorie M. Colbert-Getz, Caroline K. Milne

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


Background: Feedback is a critical element of graduate medical education. Narrative comments on evaluation forms are a source of feedback for residents. As a shared mental model for performance, milestone-based evaluations may impact narrative comments and resident perception of feedback. Objective: To determine if milestone-based evaluations impacted the quality of faculty members’ narrative comments on evaluations and, as an extension, residents’ perception of feedback. Design: Concurrent mixed methods study, including qualitative analysis of narrative comments and survey of resident perception of feedback. Participants: Seventy internal medicine residents and their faculty evaluators at the University of Utah. Approach: Faculty narrative comments from 248 evaluations pre- and post-milestone implementation were analyzed for quality and Accreditation Council for Graduate Medical Education competency by area of strength and area for improvement. Seventy residents were surveyed regarding quality of feedback pre- and post-milestone implementation. Key Results: Qualitative analysis of narrative comments revealed nearly all evaluations pre- and post-milestone implementation included comments about areas of strength but were frequently vague and not related to competencies. Few evaluations included narrative comments on areas for improvement, but these were of higher quality compared to areas of strength (p < 0.001). Overall resident perception of quality of narrative comments was low and did not change following milestone implementation (p = 0.562) for the 86% of residents (N = 60/70) who completed the pre- and post-surveys. Conclusions: The quality of narrative comments was poor, and there was no evidence of improved quality following introduction of milestone-based evaluations. Comments on areas for improvement were of higher quality than areas of strength, suggesting an area for targeted intervention. Residents’ perception of feedback quality did not change following implementation of milestone-based evaluations, suggesting that in the post-milestone era, internal medicine educators need to utilize additional interventions to improve quality of feedback.

Original languageEnglish (US)
Pages (from-to)929-935
Number of pages7
JournalJournal of general internal medicine
Issue number6
StatePublished - Jun 15 2019
Externally publishedYes


  • graduate medical education
  • medical education qualitative methods
  • milestones

ASJC Scopus subject areas

  • Internal Medicine


Dive into the research topics of 'Milestone Implementation’s Impact on Narrative Comments and Perception of Feedback for Internal Medicine Residents: a Mixed Methods Study'. Together they form a unique fingerprint.

Cite this