Mode of Information Delivery Does Not Effect Anesthesia Trainee Performance During Simulated Perioperative Pediatric Critical Events: A Trial of Paper Versus Electronic Cognitive Aids

Scott Watkins, Shilo Anders, Anna Clebone, Elisabeth Hughes, Vikram Patel, Laura Zeigler, Yaping Shi, Matthew S. Shotwell, Matthew D. McEvoy, Matthew B. Weinger

Research output: Contribution to journalArticle

Abstract

Introduction: Cognitive aids (CAs), including emergency manuals and checklists, have been recommended as a means to address the failure of healthcare providers to adhere to evidence-based standards of treatment during crisis situations. Unfortunately, users of CAs still commit errors, omit critical steps, fail to achieve perfect adherence to guidelines, and frequently choose to not use CA during both simulated and real crisis events. We sought to evaluate whether the mode in which a CA presents information (ie, paper vs. electronic) affects clinician performance during simulated critical events. Methods: In a prospective, randomized, controlled trial, anesthesia trainees managed simulated events under 1 of the following 3 conditions: (1) from memory alone (control), (2) with a paper CA, or (3) with an electronic version of the same CA. Management of the events was assessed using scenario-specific checklists. Mixed-effect regression models were used for analysis of overall checklist score and for elapsed time. Results: One hundred thirty-nine simulated events were observed and rated. Approximately, 1 of 3 trainees assigned to use a CA (electronic 29%, paper 36%) chose not to use it during the scenario. Compared with the control group (52%), the overall score was 6% higher in the paper CA group and 8% higher (95% confidence interval, 0.914.5; P = 0.03) in the electronic CA group. The difference between paper and electronic CA was not significant. There was a wide range in time to first use of the CA, but the time to task completion was not affected by CA use, nor did the time to CA use impact CA effectiveness as measured by performance. Conclusions: The format (paper or electronic) of the CA did not affect the impact of the CA on clinician performance in this study. Clinician compliance with the use of the CA was unaffected by format, suggesting that other factors may determine whether clinicians choose to use a CA or not. Time to use of the CA did not affect clinical performance, suggesting that it may not be when CAs are used but how they are used that determines their impact. The current study highlights the importance of not just familiarizing clinicians with the content of CA but also training clinicians in when and how to use an emergency CA.

Original languageEnglish (US)
Pages (from-to)385-393
Number of pages9
JournalSimulation in Healthcare
Volume11
Issue number6
DOIs
StatePublished - Dec 1 2016
Externally publishedYes

Fingerprint

Pediatrics
Anesthesia
trainee
Electronic paper
Electronics
electronics
Checklist
event
performance
Choose
Data storage equipment
Emergency
Emergencies
Guideline Adherence
scenario
Randomized Controlled Trial
Mixed Effects
Scenarios
Group
Health Personnel

Keywords

  • Anesthesia
  • Checklists
  • Cognitive AIDS
  • Resuscitation
  • Simulation

ASJC Scopus subject areas

  • Epidemiology
  • Medicine (miscellaneous)
  • Education
  • Modeling and Simulation

Cite this

Mode of Information Delivery Does Not Effect Anesthesia Trainee Performance During Simulated Perioperative Pediatric Critical Events : A Trial of Paper Versus Electronic Cognitive Aids. / Watkins, Scott; Anders, Shilo; Clebone, Anna; Hughes, Elisabeth; Patel, Vikram; Zeigler, Laura; Shi, Yaping; Shotwell, Matthew S.; McEvoy, Matthew D.; Weinger, Matthew B.

In: Simulation in Healthcare, Vol. 11, No. 6, 01.12.2016, p. 385-393.

Research output: Contribution to journalArticle

Watkins, Scott ; Anders, Shilo ; Clebone, Anna ; Hughes, Elisabeth ; Patel, Vikram ; Zeigler, Laura ; Shi, Yaping ; Shotwell, Matthew S. ; McEvoy, Matthew D. ; Weinger, Matthew B. / Mode of Information Delivery Does Not Effect Anesthesia Trainee Performance During Simulated Perioperative Pediatric Critical Events : A Trial of Paper Versus Electronic Cognitive Aids. In: Simulation in Healthcare. 2016 ; Vol. 11, No. 6. pp. 385-393.
@article{b6940d884fb24df3b7465351547b46d7,
title = "Mode of Information Delivery Does Not Effect Anesthesia Trainee Performance During Simulated Perioperative Pediatric Critical Events: A Trial of Paper Versus Electronic Cognitive Aids",
abstract = "Introduction: Cognitive aids (CAs), including emergency manuals and checklists, have been recommended as a means to address the failure of healthcare providers to adhere to evidence-based standards of treatment during crisis situations. Unfortunately, users of CAs still commit errors, omit critical steps, fail to achieve perfect adherence to guidelines, and frequently choose to not use CA during both simulated and real crisis events. We sought to evaluate whether the mode in which a CA presents information (ie, paper vs. electronic) affects clinician performance during simulated critical events. Methods: In a prospective, randomized, controlled trial, anesthesia trainees managed simulated events under 1 of the following 3 conditions: (1) from memory alone (control), (2) with a paper CA, or (3) with an electronic version of the same CA. Management of the events was assessed using scenario-specific checklists. Mixed-effect regression models were used for analysis of overall checklist score and for elapsed time. Results: One hundred thirty-nine simulated events were observed and rated. Approximately, 1 of 3 trainees assigned to use a CA (electronic 29{\%}, paper 36{\%}) chose not to use it during the scenario. Compared with the control group (52{\%}), the overall score was 6{\%} higher in the paper CA group and 8{\%} higher (95{\%} confidence interval, 0.914.5; P = 0.03) in the electronic CA group. The difference between paper and electronic CA was not significant. There was a wide range in time to first use of the CA, but the time to task completion was not affected by CA use, nor did the time to CA use impact CA effectiveness as measured by performance. Conclusions: The format (paper or electronic) of the CA did not affect the impact of the CA on clinician performance in this study. Clinician compliance with the use of the CA was unaffected by format, suggesting that other factors may determine whether clinicians choose to use a CA or not. Time to use of the CA did not affect clinical performance, suggesting that it may not be when CAs are used but how they are used that determines their impact. The current study highlights the importance of not just familiarizing clinicians with the content of CA but also training clinicians in when and how to use an emergency CA.",
keywords = "Anesthesia, Checklists, Cognitive AIDS, Resuscitation, Simulation",
author = "Scott Watkins and Shilo Anders and Anna Clebone and Elisabeth Hughes and Vikram Patel and Laura Zeigler and Yaping Shi and Shotwell, {Matthew S.} and McEvoy, {Matthew D.} and Weinger, {Matthew B.}",
year = "2016",
month = "12",
day = "1",
doi = "10.1097/SIH.0000000000000191",
language = "English (US)",
volume = "11",
pages = "385--393",
journal = "Simulation in Healthcare",
issn = "1559-2332",
publisher = "Lippincott Williams and Wilkins",
number = "6",

}

TY - JOUR

T1 - Mode of Information Delivery Does Not Effect Anesthesia Trainee Performance During Simulated Perioperative Pediatric Critical Events

T2 - A Trial of Paper Versus Electronic Cognitive Aids

AU - Watkins, Scott

AU - Anders, Shilo

AU - Clebone, Anna

AU - Hughes, Elisabeth

AU - Patel, Vikram

AU - Zeigler, Laura

AU - Shi, Yaping

AU - Shotwell, Matthew S.

AU - McEvoy, Matthew D.

AU - Weinger, Matthew B.

PY - 2016/12/1

Y1 - 2016/12/1

N2 - Introduction: Cognitive aids (CAs), including emergency manuals and checklists, have been recommended as a means to address the failure of healthcare providers to adhere to evidence-based standards of treatment during crisis situations. Unfortunately, users of CAs still commit errors, omit critical steps, fail to achieve perfect adherence to guidelines, and frequently choose to not use CA during both simulated and real crisis events. We sought to evaluate whether the mode in which a CA presents information (ie, paper vs. electronic) affects clinician performance during simulated critical events. Methods: In a prospective, randomized, controlled trial, anesthesia trainees managed simulated events under 1 of the following 3 conditions: (1) from memory alone (control), (2) with a paper CA, or (3) with an electronic version of the same CA. Management of the events was assessed using scenario-specific checklists. Mixed-effect regression models were used for analysis of overall checklist score and for elapsed time. Results: One hundred thirty-nine simulated events were observed and rated. Approximately, 1 of 3 trainees assigned to use a CA (electronic 29%, paper 36%) chose not to use it during the scenario. Compared with the control group (52%), the overall score was 6% higher in the paper CA group and 8% higher (95% confidence interval, 0.914.5; P = 0.03) in the electronic CA group. The difference between paper and electronic CA was not significant. There was a wide range in time to first use of the CA, but the time to task completion was not affected by CA use, nor did the time to CA use impact CA effectiveness as measured by performance. Conclusions: The format (paper or electronic) of the CA did not affect the impact of the CA on clinician performance in this study. Clinician compliance with the use of the CA was unaffected by format, suggesting that other factors may determine whether clinicians choose to use a CA or not. Time to use of the CA did not affect clinical performance, suggesting that it may not be when CAs are used but how they are used that determines their impact. The current study highlights the importance of not just familiarizing clinicians with the content of CA but also training clinicians in when and how to use an emergency CA.

AB - Introduction: Cognitive aids (CAs), including emergency manuals and checklists, have been recommended as a means to address the failure of healthcare providers to adhere to evidence-based standards of treatment during crisis situations. Unfortunately, users of CAs still commit errors, omit critical steps, fail to achieve perfect adherence to guidelines, and frequently choose to not use CA during both simulated and real crisis events. We sought to evaluate whether the mode in which a CA presents information (ie, paper vs. electronic) affects clinician performance during simulated critical events. Methods: In a prospective, randomized, controlled trial, anesthesia trainees managed simulated events under 1 of the following 3 conditions: (1) from memory alone (control), (2) with a paper CA, or (3) with an electronic version of the same CA. Management of the events was assessed using scenario-specific checklists. Mixed-effect regression models were used for analysis of overall checklist score and for elapsed time. Results: One hundred thirty-nine simulated events were observed and rated. Approximately, 1 of 3 trainees assigned to use a CA (electronic 29%, paper 36%) chose not to use it during the scenario. Compared with the control group (52%), the overall score was 6% higher in the paper CA group and 8% higher (95% confidence interval, 0.914.5; P = 0.03) in the electronic CA group. The difference between paper and electronic CA was not significant. There was a wide range in time to first use of the CA, but the time to task completion was not affected by CA use, nor did the time to CA use impact CA effectiveness as measured by performance. Conclusions: The format (paper or electronic) of the CA did not affect the impact of the CA on clinician performance in this study. Clinician compliance with the use of the CA was unaffected by format, suggesting that other factors may determine whether clinicians choose to use a CA or not. Time to use of the CA did not affect clinical performance, suggesting that it may not be when CAs are used but how they are used that determines their impact. The current study highlights the importance of not just familiarizing clinicians with the content of CA but also training clinicians in when and how to use an emergency CA.

KW - Anesthesia

KW - Checklists

KW - Cognitive AIDS

KW - Resuscitation

KW - Simulation

UR - http://www.scopus.com/inward/record.url?scp=85006265581&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85006265581&partnerID=8YFLogxK

U2 - 10.1097/SIH.0000000000000191

DO - 10.1097/SIH.0000000000000191

M3 - Article

C2 - 27922569

AN - SCOPUS:85006265581

VL - 11

SP - 385

EP - 393

JO - Simulation in Healthcare

JF - Simulation in Healthcare

SN - 1559-2332

IS - 6

ER -