Validation of an Objective Assessment Tool for Tonsillectomy in Otolaryngology Resident Training

Objective: Create and validate an objective structured assessment of technical skills (OSATS) for otolaryngology residents learning how to perform a tonsillectomy. Study Design: Multicenter prospective longitudinal validation study. Methods: A multi-institutional study at six tertiary academic otolaryngology residency programs from July 2009 to May 2012. Using the modified Delphi technique, a panel of pediatric otolaryngologists created a tonsillectomy task-based checklist (TBC) for a tonsil OSATS using a 5-point Likert-type scale. Residents were assessed by pediatric otolaryngology staff at the time of surgery with the TBC and a global rating scale. Procedure time, patient age, number of previously performed tonsillectomies, and surgical technique were also collected. Results: One hundred sixty-seven tonsil OSATS were completed for 38 residents, and competency was recorded for 99 (59.2%). Residents scored as competent had performed significantly more previous tonsillectomies than those deemed noncompetent, 44.4 ± 35.6 and 13.5 ± 11.6, respectively (P <.001). The mean overall score on the tonsil TBC was 4.0 ± 0.8 and 2.6 ± 1.0 for competent and noncompetent, respectively (P <.001). Higher number of tonsillectomies performed and mean tonsil TBC score significantly increased the likelihood of competency (P <.001). Each additional tonsillectomy performed increased the likelihood of achieving competency by 6.3% (P =.006, 95% confidence interval (CI): 1.330–1.110), and each 1.0 point increase in mean tonsil TBC score increased the likelihood of competency by a factor of 2.71 (P =.006, 95% CI:1.330–5.513). There is a 95% likelihood of competency at 48 tonsillectomies or a tonsil TBC score of 4.91. Conclusion: The tonsil OSATS is a valid and feasible instrument to assess resident competency with tonsillectomy and provides timely objective feedback. Level of Evidence: 4. Laryngoscope, 131:E359–E366, 2021.