TY - JOUR
T1 - Using Audience Response Systems for Real-Time Learning Assessments During Surgical Morbidity and Mortality Conference
AU - Leeds, Ira L.
AU - DiBrito, Sandra R.
AU - Jones, Christian D.
AU - Higgins, Robert S.D.
AU - Haut, Elliott R.
N1 - Funding Information:
Funding: I.L. received salary support from a National Cancer Institute T32 training grant (T32CA126607) while completing this study. The authors gratefully acknowledge the Johns Hopkins University School of Medicine Institute for Educational Excellence's Small Grants Program for providing the funding for the educational intervention described here. During the period of the study reported here, E.H. received research grant support from AHRQ (R01HS024547), the National Institutes of Health National Heart, Lung, and Blood Institute (NIH-NHLBI) (1R21HL129028-01A1), and research contract support from the Patient Centered Outcomes Research Institute (PCORI; CE-12-11-4489, PCS-1511-32745, and DI-1603-34596). E.H. receives consulting and speaking fees for the “Preventing Avoidable Venous Thromboembolism - Every Patient, Every Time” Vizient IMPERATIV Advantage Performance Improvement Collaborative; book royalties from Lippincott Williams & Wilkins; fees as an author of a paper that was commissioned by the National Academies of Medicine titled “Military Trauma Care's Learning Health System: The Importance of Data Driven Decision Making,” which was used to support the report titled “A National Trauma Care System: Integrating Military and Civilian Trauma Systems to Achieve Zero Preventable Deaths After Injury.”
Funding Information:
Funding: I.L. received salary support from a National Cancer Institute T32 training grant (T32CA126607) while completing this study. The authors gratefully acknowledge the Johns Hopkins University School of Medicine Institute for Educational Excellence's Small Grants Program for providing the funding for the educational intervention described here. During the period of the study reported here, E.H. received research grant support from AHRQ (R01HS024547), the National Institutes of Health National Heart, Lung, and Blood Institute (NIH-NHLBI) (1R21HL129028-01A1), and research contract support from the Patient Centered Outcomes Research Institute (PCORI; CE-12-11-4489, PCS-1511-32745, and DI-1603-34596). E.H. receives consulting and speaking fees for the ?Preventing Avoidable Venous Thromboembolism - Every Patient, Every Time? Vizient IMPERATIV Advantage Performance Improvement Collaborative; book royalties from Lippincott Williams & Wilkins; fees as an author of a paper that was commissioned by the National Academies of Medicine titled ?Military Trauma Care's Learning Health System: The Importance of Data Driven Decision Making,? which was used to support the report titled ?A National Trauma Care System: Integrating Military and Civilian Trauma Systems to Achieve Zero Preventable Deaths After Injury.?
Publisher Copyright:
© 2018
PY - 2018/11
Y1 - 2018/11
N2 - OBJECTIVE: Morbidity and mortality (M&M) conference is a mainstay of surgical education. However, its effectiveness is poorly described. The purpose of this study was to demonstrate the feasibility of a real-time audience response system for learner assessment during M&M. DESIGN: We integrated a web-based audience response system into weekly M&M conference. First, this platform collected qualitative responses about the role of M&M. Then, we used the platform to direct questions to attendees in real time. Questions focused on surgical risk estimation and classifying root causes. Responses were grouped by training and compared to a validated risk tool's prediction. Root cause assignment concordance was statistically compared using Cohen's kappa between the pluralities of faculty responses to that of trainees. SETTING: General surgical residency program based at a tertiary academic medical center. PARTICIPANTS: Affiliated categorical residents, preliminary residents, and clinical fellows. RESULTS: We enrolled 110 participants (38 faculty, 31 senior trainees, and 41 trainees). The majority of respondents (75.9%) cited education as the purpose of M&M, and all of respondents stated education as their personal motivation. Audience response questions were integrated into 34 unique case presentations. Mean absolute differences between predicted complication rates and attendees’ predictions were highest for faculty (−9.4%, p = 0.009) and lowest for junior residents (−1.8%, p = 0.385). When assigning root cause of each morbidity, concordance between faculty and trainees was low to moderate (K = 0.41). CONCLUSIONS: Assessment of learning during M&M can be performed in real time with discrimination observed by learner experience level. These data support development of this response platform to trend learner performance over time and to monitor targeted educational interventions at future M&Ms.
AB - OBJECTIVE: Morbidity and mortality (M&M) conference is a mainstay of surgical education. However, its effectiveness is poorly described. The purpose of this study was to demonstrate the feasibility of a real-time audience response system for learner assessment during M&M. DESIGN: We integrated a web-based audience response system into weekly M&M conference. First, this platform collected qualitative responses about the role of M&M. Then, we used the platform to direct questions to attendees in real time. Questions focused on surgical risk estimation and classifying root causes. Responses were grouped by training and compared to a validated risk tool's prediction. Root cause assignment concordance was statistically compared using Cohen's kappa between the pluralities of faculty responses to that of trainees. SETTING: General surgical residency program based at a tertiary academic medical center. PARTICIPANTS: Affiliated categorical residents, preliminary residents, and clinical fellows. RESULTS: We enrolled 110 participants (38 faculty, 31 senior trainees, and 41 trainees). The majority of respondents (75.9%) cited education as the purpose of M&M, and all of respondents stated education as their personal motivation. Audience response questions were integrated into 34 unique case presentations. Mean absolute differences between predicted complication rates and attendees’ predictions were highest for faculty (−9.4%, p = 0.009) and lowest for junior residents (−1.8%, p = 0.385). When assigning root cause of each morbidity, concordance between faculty and trainees was low to moderate (K = 0.41). CONCLUSIONS: Assessment of learning during M&M can be performed in real time with discrimination observed by learner experience level. These data support development of this response platform to trend learner performance over time and to monitor targeted educational interventions at future M&Ms.
KW - Educational technology
KW - Learning
KW - Morbidity and mortality conference
KW - Practice-Based Learning and Improvement
KW - Professional education
KW - Surgical education
UR - http://www.scopus.com/inward/record.url?scp=85049318890&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85049318890&partnerID=8YFLogxK
U2 - 10.1016/j.jsurg.2018.05.010
DO - 10.1016/j.jsurg.2018.05.010
M3 - Article
C2 - 30523799
AN - SCOPUS:85049318890
VL - 75
SP - 1535
EP - 1543
JO - Journal of Surgical Education
JF - Journal of Surgical Education
SN - 1931-7204
IS - 6
ER -