Leveraging Natural Language Processing: Toward Computer-Assisted Scoring of Patient Notes in the USMLE Step 2 Clinical Skills Exam

Jessica Salt, Polina Harik, Michael A. Barone

Research output: Contribution to journalReview articlepeer-review

Abstract

The United States Medical Licensing Examination Step 2 Clinical Skills (CS) exam uses physician raters to evaluate patient notes written by examinees. In this Invited Commentary, the authors describe the ways in which the Step 2 CS exam could benefit from adopting a computer-assisted scoring approach that combines physician raters' judgments with computer-generated scores based on natural language processing (NLP). Since 2003, the National Board of Medical Examiners has researched NLP technology to determine whether it offers the opportunity to mitigate challenges associated with human raters while continuing to capitalize on the judgment of physician experts. The authors discuss factors to consider before computer-assisted scoring is introduced into a high-stakes licensure exam context. They suggest that combining physician judgments and computer-assisted scoring can enhance and improve performance-based assessments in medical education and medical regulation.

Original languageEnglish (US)
Pages (from-to)314-316
Number of pages3
JournalAcademic Medicine
Volume94
Issue number3
DOIs
StatePublished - Mar 1 2019
Externally publishedYes

ASJC Scopus subject areas

  • Education

Fingerprint Dive into the research topics of 'Leveraging Natural Language Processing: Toward Computer-Assisted Scoring of Patient Notes in the USMLE Step 2 Clinical Skills Exam'. Together they form a unique fingerprint.

Cite this