Human Factors Evaluation of the Universal Anaesthesia Machine: Assessing Equipment with High-Fidelity Simulation Prior to Deployment in a Resource-Constrained Environment

John Sampson, Benjamin H. Lee, Rahul Koka, Adaora M. Chima, Eric V. Jackson, Onyebuchi O. Ogbuagu, Tina Tran, Michael Rosen

Research output: Contribution to journalArticle

Abstract

Background: Anesthesia providers in low- and middle-income countries face many challenges, including poor availability of functioning equipment designed to meet their environmental, organizational, and resource constraints. These are serious global health disparities which threaten access to care and patient safety for those who receive surgical care. In this study, we conducted a simulation-based human factors analysis of the Universal Anaesthesia Machine (UAM ® ), a device designed to support anesthesia providers in austere medical settings. Our team anticipated the introduction of the UAM ® to the two major referral hospitals in Freetown, Sierra Leone. A prior observational study had identified these two hospitals as having environmental conditions consistent with an austere environment: an unstable electrical grid, as well as limited access to compressed oxygen, biomedical support, and consumables. Although the Baltimore simulation environment cannot reproduce all of the challenges present in a resource-constrained environment such as Sierra Leone, the major impediments to standard anesthesia machine functionality and human factors-associated use can be reproduced with the use of high-fidelity simulation. Using anesthesia care providers who have limited UAM ® familiarity, this study allowed for the examination of machine-user issues in a controlled environment in preparation for further field studies concerning equipment introduction, training and device deployment in Sierra Leone. The goals of this study were: 1. to assess the usability of the UAM ® (machine-user interface, simulated patient use, symbology, etc.)across different provider user groups during simulation of use in scenarios depicting routine use in healthy patients, use in clinically challenging patients and use in environmentally-challenging scenarios in a controlled setting devoid of patient risk, and 2. To gather feedback on available UAM manuals and cognitive aides and UAM usability issues in order to guide development of curricula for training providers on use of the UAM ® in the intended austere clinical environments. Methods: Residents, fellows, attending physician anesthesiologists, student nurse anesthetists, and nurse anesthetists participated in a variety of simulations involving the Universal Anaesthesia Machine ® at the Johns Hopkins Medicine Simulation Center between September 2012 and July 2013. Data collected included participant demographics, performance during simulation scenarios captured with critical action checklists, workload ratings captured with the National Aeronautics and Space Administration Task Load Index (NASA TLX), and participant reactions to UAM ® use captured through a post-session survey and semi-structured usability debriefing. The scenarios were: 1. normal use (machine check, induction, and maintenance of an uneventful case), 2. use in a challenging clinical condition (acute onset of bronchospasm)and 3.use in an adverse environmental event (power failure). Critical action checklists and workload ratings were analyzed by Analysis of Covariance (ANCOVA)to control for participant demographics. Usability debriefings were analyzed qualitatively. Results: Thirty-five anesthesia providers participated in the study. Overall participant ratings, observations of performance in simulation scenarios, and usability debriefings indicated a high level of usability for the UAM ® . Mean participant ratings were high for ease of use (5.4 ± 0.96)and clarity of instruction (6.2 ± 0.87)on a 7-point scale in which higher ratings indicate more positive perceptions. After adjusting for clinical experience, workload ratings were significantly higher in the bronchospasm scenario than in the normal/routine use (P = 0.046; 95% CI, 0.33–34.7)or power failure scenarios (P = 0.012; 95% CI, 5.24–37.9). Thirty-two specific usability issues were identified and grouped into five themes: device design and labeling, machine use during simulation scenarios, user-anticipated errors or hazards, curriculum issues, and overall impressions of the UAM ® . Conclusions: The UAM ® design addresses many of the key challenges facing anesthesia providers in resource-constrained settings. The simulation-based human factors evaluation described here successfully identified opportunities for continued refinement of the initial device design as well as issues to be addressed in future curricula and cognitive aides.

Original languageEnglish (US)
JournalJournal of the National Medical Association
DOIs
StatePublished - Jan 1 2019

Fingerprint

Anesthesia
Equipment and Supplies
Sierra Leone
Workload
Nurse Anesthetists
Curriculum
Equipment Design
Bronchial Spasm
Checklist
Demography
United States National Aeronautics and Space Administration
Controlled Environment
Baltimore
Patient Safety
Statistical Factor Analysis
Observational Studies
Referral and Consultation
Maintenance
Medicine

Keywords

  • Africa
  • Anesthesia
  • Global health
  • Human factors
  • Simulation training

ASJC Scopus subject areas

  • Medicine(all)

Cite this

@article{7c3f256df2c84a4585d08320a6b8dc11,
title = "Human Factors Evaluation of the Universal Anaesthesia Machine: Assessing Equipment with High-Fidelity Simulation Prior to Deployment in a Resource-Constrained Environment",
abstract = "Background: Anesthesia providers in low- and middle-income countries face many challenges, including poor availability of functioning equipment designed to meet their environmental, organizational, and resource constraints. These are serious global health disparities which threaten access to care and patient safety for those who receive surgical care. In this study, we conducted a simulation-based human factors analysis of the Universal Anaesthesia Machine (UAM {\circledR} ), a device designed to support anesthesia providers in austere medical settings. Our team anticipated the introduction of the UAM {\circledR} to the two major referral hospitals in Freetown, Sierra Leone. A prior observational study had identified these two hospitals as having environmental conditions consistent with an austere environment: an unstable electrical grid, as well as limited access to compressed oxygen, biomedical support, and consumables. Although the Baltimore simulation environment cannot reproduce all of the challenges present in a resource-constrained environment such as Sierra Leone, the major impediments to standard anesthesia machine functionality and human factors-associated use can be reproduced with the use of high-fidelity simulation. Using anesthesia care providers who have limited UAM {\circledR} familiarity, this study allowed for the examination of machine-user issues in a controlled environment in preparation for further field studies concerning equipment introduction, training and device deployment in Sierra Leone. The goals of this study were: 1. to assess the usability of the UAM {\circledR} (machine-user interface, simulated patient use, symbology, etc.)across different provider user groups during simulation of use in scenarios depicting routine use in healthy patients, use in clinically challenging patients and use in environmentally-challenging scenarios in a controlled setting devoid of patient risk, and 2. To gather feedback on available UAM manuals and cognitive aides and UAM usability issues in order to guide development of curricula for training providers on use of the UAM {\circledR} in the intended austere clinical environments. Methods: Residents, fellows, attending physician anesthesiologists, student nurse anesthetists, and nurse anesthetists participated in a variety of simulations involving the Universal Anaesthesia Machine {\circledR} at the Johns Hopkins Medicine Simulation Center between September 2012 and July 2013. Data collected included participant demographics, performance during simulation scenarios captured with critical action checklists, workload ratings captured with the National Aeronautics and Space Administration Task Load Index (NASA TLX), and participant reactions to UAM {\circledR} use captured through a post-session survey and semi-structured usability debriefing. The scenarios were: 1. normal use (machine check, induction, and maintenance of an uneventful case), 2. use in a challenging clinical condition (acute onset of bronchospasm)and 3.use in an adverse environmental event (power failure). Critical action checklists and workload ratings were analyzed by Analysis of Covariance (ANCOVA)to control for participant demographics. Usability debriefings were analyzed qualitatively. Results: Thirty-five anesthesia providers participated in the study. Overall participant ratings, observations of performance in simulation scenarios, and usability debriefings indicated a high level of usability for the UAM {\circledR} . Mean participant ratings were high for ease of use (5.4 ± 0.96)and clarity of instruction (6.2 ± 0.87)on a 7-point scale in which higher ratings indicate more positive perceptions. After adjusting for clinical experience, workload ratings were significantly higher in the bronchospasm scenario than in the normal/routine use (P = 0.046; 95{\%} CI, 0.33–34.7)or power failure scenarios (P = 0.012; 95{\%} CI, 5.24–37.9). Thirty-two specific usability issues were identified and grouped into five themes: device design and labeling, machine use during simulation scenarios, user-anticipated errors or hazards, curriculum issues, and overall impressions of the UAM {\circledR} . Conclusions: The UAM {\circledR} design addresses many of the key challenges facing anesthesia providers in resource-constrained settings. The simulation-based human factors evaluation described here successfully identified opportunities for continued refinement of the initial device design as well as issues to be addressed in future curricula and cognitive aides.",
keywords = "Africa, Anesthesia, Global health, Human factors, Simulation training",
author = "John Sampson and Lee, {Benjamin H.} and Rahul Koka and Chima, {Adaora M.} and Jackson, {Eric V.} and Ogbuagu, {Onyebuchi O.} and Tina Tran and Michael Rosen",
year = "2019",
month = "1",
day = "1",
doi = "10.1016/j.jnma.2019.03.005",
language = "English (US)",
journal = "Journal of the National Medical Association",
issn = "0027-9684",
publisher = "National Medical Association",

}

TY - JOUR

T1 - Human Factors Evaluation of the Universal Anaesthesia Machine

T2 - Assessing Equipment with High-Fidelity Simulation Prior to Deployment in a Resource-Constrained Environment

AU - Sampson, John

AU - Lee, Benjamin H.

AU - Koka, Rahul

AU - Chima, Adaora M.

AU - Jackson, Eric V.

AU - Ogbuagu, Onyebuchi O.

AU - Tran, Tina

AU - Rosen, Michael

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Background: Anesthesia providers in low- and middle-income countries face many challenges, including poor availability of functioning equipment designed to meet their environmental, organizational, and resource constraints. These are serious global health disparities which threaten access to care and patient safety for those who receive surgical care. In this study, we conducted a simulation-based human factors analysis of the Universal Anaesthesia Machine (UAM ® ), a device designed to support anesthesia providers in austere medical settings. Our team anticipated the introduction of the UAM ® to the two major referral hospitals in Freetown, Sierra Leone. A prior observational study had identified these two hospitals as having environmental conditions consistent with an austere environment: an unstable electrical grid, as well as limited access to compressed oxygen, biomedical support, and consumables. Although the Baltimore simulation environment cannot reproduce all of the challenges present in a resource-constrained environment such as Sierra Leone, the major impediments to standard anesthesia machine functionality and human factors-associated use can be reproduced with the use of high-fidelity simulation. Using anesthesia care providers who have limited UAM ® familiarity, this study allowed for the examination of machine-user issues in a controlled environment in preparation for further field studies concerning equipment introduction, training and device deployment in Sierra Leone. The goals of this study were: 1. to assess the usability of the UAM ® (machine-user interface, simulated patient use, symbology, etc.)across different provider user groups during simulation of use in scenarios depicting routine use in healthy patients, use in clinically challenging patients and use in environmentally-challenging scenarios in a controlled setting devoid of patient risk, and 2. To gather feedback on available UAM manuals and cognitive aides and UAM usability issues in order to guide development of curricula for training providers on use of the UAM ® in the intended austere clinical environments. Methods: Residents, fellows, attending physician anesthesiologists, student nurse anesthetists, and nurse anesthetists participated in a variety of simulations involving the Universal Anaesthesia Machine ® at the Johns Hopkins Medicine Simulation Center between September 2012 and July 2013. Data collected included participant demographics, performance during simulation scenarios captured with critical action checklists, workload ratings captured with the National Aeronautics and Space Administration Task Load Index (NASA TLX), and participant reactions to UAM ® use captured through a post-session survey and semi-structured usability debriefing. The scenarios were: 1. normal use (machine check, induction, and maintenance of an uneventful case), 2. use in a challenging clinical condition (acute onset of bronchospasm)and 3.use in an adverse environmental event (power failure). Critical action checklists and workload ratings were analyzed by Analysis of Covariance (ANCOVA)to control for participant demographics. Usability debriefings were analyzed qualitatively. Results: Thirty-five anesthesia providers participated in the study. Overall participant ratings, observations of performance in simulation scenarios, and usability debriefings indicated a high level of usability for the UAM ® . Mean participant ratings were high for ease of use (5.4 ± 0.96)and clarity of instruction (6.2 ± 0.87)on a 7-point scale in which higher ratings indicate more positive perceptions. After adjusting for clinical experience, workload ratings were significantly higher in the bronchospasm scenario than in the normal/routine use (P = 0.046; 95% CI, 0.33–34.7)or power failure scenarios (P = 0.012; 95% CI, 5.24–37.9). Thirty-two specific usability issues were identified and grouped into five themes: device design and labeling, machine use during simulation scenarios, user-anticipated errors or hazards, curriculum issues, and overall impressions of the UAM ® . Conclusions: The UAM ® design addresses many of the key challenges facing anesthesia providers in resource-constrained settings. The simulation-based human factors evaluation described here successfully identified opportunities for continued refinement of the initial device design as well as issues to be addressed in future curricula and cognitive aides.

AB - Background: Anesthesia providers in low- and middle-income countries face many challenges, including poor availability of functioning equipment designed to meet their environmental, organizational, and resource constraints. These are serious global health disparities which threaten access to care and patient safety for those who receive surgical care. In this study, we conducted a simulation-based human factors analysis of the Universal Anaesthesia Machine (UAM ® ), a device designed to support anesthesia providers in austere medical settings. Our team anticipated the introduction of the UAM ® to the two major referral hospitals in Freetown, Sierra Leone. A prior observational study had identified these two hospitals as having environmental conditions consistent with an austere environment: an unstable electrical grid, as well as limited access to compressed oxygen, biomedical support, and consumables. Although the Baltimore simulation environment cannot reproduce all of the challenges present in a resource-constrained environment such as Sierra Leone, the major impediments to standard anesthesia machine functionality and human factors-associated use can be reproduced with the use of high-fidelity simulation. Using anesthesia care providers who have limited UAM ® familiarity, this study allowed for the examination of machine-user issues in a controlled environment in preparation for further field studies concerning equipment introduction, training and device deployment in Sierra Leone. The goals of this study were: 1. to assess the usability of the UAM ® (machine-user interface, simulated patient use, symbology, etc.)across different provider user groups during simulation of use in scenarios depicting routine use in healthy patients, use in clinically challenging patients and use in environmentally-challenging scenarios in a controlled setting devoid of patient risk, and 2. To gather feedback on available UAM manuals and cognitive aides and UAM usability issues in order to guide development of curricula for training providers on use of the UAM ® in the intended austere clinical environments. Methods: Residents, fellows, attending physician anesthesiologists, student nurse anesthetists, and nurse anesthetists participated in a variety of simulations involving the Universal Anaesthesia Machine ® at the Johns Hopkins Medicine Simulation Center between September 2012 and July 2013. Data collected included participant demographics, performance during simulation scenarios captured with critical action checklists, workload ratings captured with the National Aeronautics and Space Administration Task Load Index (NASA TLX), and participant reactions to UAM ® use captured through a post-session survey and semi-structured usability debriefing. The scenarios were: 1. normal use (machine check, induction, and maintenance of an uneventful case), 2. use in a challenging clinical condition (acute onset of bronchospasm)and 3.use in an adverse environmental event (power failure). Critical action checklists and workload ratings were analyzed by Analysis of Covariance (ANCOVA)to control for participant demographics. Usability debriefings were analyzed qualitatively. Results: Thirty-five anesthesia providers participated in the study. Overall participant ratings, observations of performance in simulation scenarios, and usability debriefings indicated a high level of usability for the UAM ® . Mean participant ratings were high for ease of use (5.4 ± 0.96)and clarity of instruction (6.2 ± 0.87)on a 7-point scale in which higher ratings indicate more positive perceptions. After adjusting for clinical experience, workload ratings were significantly higher in the bronchospasm scenario than in the normal/routine use (P = 0.046; 95% CI, 0.33–34.7)or power failure scenarios (P = 0.012; 95% CI, 5.24–37.9). Thirty-two specific usability issues were identified and grouped into five themes: device design and labeling, machine use during simulation scenarios, user-anticipated errors or hazards, curriculum issues, and overall impressions of the UAM ® . Conclusions: The UAM ® design addresses many of the key challenges facing anesthesia providers in resource-constrained settings. The simulation-based human factors evaluation described here successfully identified opportunities for continued refinement of the initial device design as well as issues to be addressed in future curricula and cognitive aides.

KW - Africa

KW - Anesthesia

KW - Global health

KW - Human factors

KW - Simulation training

UR - http://www.scopus.com/inward/record.url?scp=85065235055&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065235055&partnerID=8YFLogxK

U2 - 10.1016/j.jnma.2019.03.005

DO - 10.1016/j.jnma.2019.03.005

M3 - Article

C2 - 31078287

AN - SCOPUS:85065235055

JO - Journal of the National Medical Association

JF - Journal of the National Medical Association

SN - 0027-9684

ER -