Development and pilot testing of an operative competency assessment tool for pediatric direct laryngoscopy and rigid bronchoscopy

Stacey L. Ishman, David J. Brown, Emily F. Boss, Margaret L. Skinner, David E. Tunkel, Rose Stavinoha, Sandra Y. Lin

Research output: Contribution to journalArticle

Abstract

Objectives: To develop and pilot test an objective assessment of technical skills instrument for evaluation of residents' surgical performance of pediatric direct laryngoscopy and rigid bronchoscopy, with emphasis on feasibility, validity, and interrater agreement. Study Design: Prospective, unblinded educational quality-improvement project combining paired and unpaired observations. Methods: Objective Structured Assessment of Technical Skills (OSATS) evaluation instruments were developed from detailed faculty input. Two instruments were created: a surgical checklist and a global assessment of surgical performance. The instruments were tested in a simulation bronchoscopy course and in the operative suite. Paired observations were used to determine interrater agreement, whereas multiple evaluations were used to calculate construct validity and internal consistency. Results: Forty-four assessments were completed for 19 residents. Seven faculty members evaluated residents as they performed laryngoscopy and bronchoscopy in an animal simulation laboratory and in the operating room. The evaluation tool was found to be feasible. The interrater agreement for both instruments was significant at 80.4% to 84.6% (P ≤.0001). Construct validity was confirmed with increasing mean global and task specific scores by postgraduate year (P ;lt&.0001). Internal consistency, measured with Cronbach alpha, was high at 0.968. Conclusions: This pilot study suggests that a reliable and valid instrument for objective evaluation of surgical competency can be developed for pediatric direct laryngoscopy and rigid bronchoscopy. This instrument can be used for formative and summative feedback of operative performance. In addition, it was easy to use and valid in a limited evaluation; however, larger studies are required to validate its utility.

Original languageEnglish (US)
Pages (from-to)2294-2300
Number of pages7
JournalLaryngoscope
Volume120
Issue number11
DOIs
StatePublished - Nov 1 2010

Keywords

  • Bronchoscopy
  • OSAT
  • education
  • educational measurement
  • laryngoscopy
  • pediatric airway
  • residency Laryngoscope
  • surgery

ASJC Scopus subject areas

  • Otorhinolaryngology

Fingerprint Dive into the research topics of 'Development and pilot testing of an operative competency assessment tool for pediatric direct laryngoscopy and rigid bronchoscopy'. Together they form a unique fingerprint.

  • Cite this