Does the NBME Surgery Shelf exam constitute a “double jeopardy” of USMLE Step 1 performance?

Background Scores from the NBME Subject Examination in Surgery (Surgery Shelf) positively correlate with United States Medical Licensing Examination Step 1 (Step 1). Based on this relationship, the authors evaluated the predictive value of Step 1 on the Surgery Shelf. Methods Surgery Shelf standard scores were substituted for Step 1 standard scores for 395 students in 2012–2014 at one medical school. Linear regression was used to determine how well Step 1 scores predicted Surgery Shelf scores. Percent match between original (with Shelf) and modified (with Step 1) clerkship grades were computed. Results Step 1 scores significantly predicted Surgery Shelf scores, R² = 0.42, P < 0.001. For every point increase in Step 1, a Surgery Shelf score increased by 0.30 points. Seventy-seven percent of original grades matched the modified grades. Conclusion Replacing Surgery Shelf scores with Step 1 scores did not have an effect on the majority of final clerkship grades. This observation raises concern over use of Surgery Shelf scores as a measure of knowledge obtained during the Surgery clerkship.