Evaluation of a Training Program to Improve Organizational Capacity for Health Systems Analytics

Steven Miller, Phillip Stablein, Jay Syed, Valerie Smothers, Emily Marx, Peter Schuyler Greene, Harold P Lehmann, Paul G. Nagy

Research output: Contribution to journalArticle

Abstract

OBJECTIVE:  The Leadership in Analytics and Data Science (LEADS) course was evaluated for effectiveness. LEADS was a 6-month program for working biomedical and health informatics (BMHI) professionals designed to improve analytics skills, knowledge of enterprise applications, data stewardship, and to foster an analytics community of practice through lectures, hands-on skill building workshops, networking events, and small group projects. METHODS:  The effectiveness of the LEADS course was evaluated using the Kirkpatrick Model by assessing pre- and postcourse knowledge, analytics capabilities, goals, practice, class lecture reaction, and change in the size of participant professional networks. Differences in pre- and postcourse responses were analyzed with a Wilcoxon signed rank test to determine significance, and effect sizes were computed using a z-statistic. RESULTS:  Twenty-nine students completed the course with 96% of respondents reporting that they were "very" or "extremely" likely to recommend the course. Participants reported improvement in several analytics capabilities including Epic data warehousing (p = 0.017), institutional review board policy (p = 0.005), and data stewardship (p = 0.007). Changes in practice patterns mirrored those in self-reported capability. On average, the participant professional network doubled. CONCLUSION:  LEADS was the first course targeted to working BMHI professional at a large academic medical center to have a formal effectiveness evaluation be published in the literature. The course achieved the goals of expansion of BMHI knowledge, skills, and professional networks. The LEADS course provides a template for continuing education of working BMHI professionals.

Original languageEnglish (US)
Pages (from-to)634-642
Number of pages9
JournalApplied clinical informatics
Volume10
Issue number4
DOIs
StatePublished - Aug 1 2019

Fingerprint

Informatics
Health
Education
Data warehouses
Research Ethics Committees
Continuing Education
Nonparametric Statistics
Statistics
Students
Industry

ASJC Scopus subject areas

  • Health Informatics
  • Computer Science Applications
  • Health Information Management

Cite this

Evaluation of a Training Program to Improve Organizational Capacity for Health Systems Analytics. / Miller, Steven; Stablein, Phillip; Syed, Jay; Smothers, Valerie; Marx, Emily; Greene, Peter Schuyler; Lehmann, Harold P; Nagy, Paul G.

In: Applied clinical informatics, Vol. 10, No. 4, 01.08.2019, p. 634-642.

Research output: Contribution to journalArticle

Miller, Steven ; Stablein, Phillip ; Syed, Jay ; Smothers, Valerie ; Marx, Emily ; Greene, Peter Schuyler ; Lehmann, Harold P ; Nagy, Paul G. / Evaluation of a Training Program to Improve Organizational Capacity for Health Systems Analytics. In: Applied clinical informatics. 2019 ; Vol. 10, No. 4. pp. 634-642.
@article{0ccb5e91b7f34e35b133c6fbaeb1622f,
title = "Evaluation of a Training Program to Improve Organizational Capacity for Health Systems Analytics",
abstract = "OBJECTIVE:  The Leadership in Analytics and Data Science (LEADS) course was evaluated for effectiveness. LEADS was a 6-month program for working biomedical and health informatics (BMHI) professionals designed to improve analytics skills, knowledge of enterprise applications, data stewardship, and to foster an analytics community of practice through lectures, hands-on skill building workshops, networking events, and small group projects. METHODS:  The effectiveness of the LEADS course was evaluated using the Kirkpatrick Model by assessing pre- and postcourse knowledge, analytics capabilities, goals, practice, class lecture reaction, and change in the size of participant professional networks. Differences in pre- and postcourse responses were analyzed with a Wilcoxon signed rank test to determine significance, and effect sizes were computed using a z-statistic. RESULTS:  Twenty-nine students completed the course with 96{\%} of respondents reporting that they were {"}very{"} or {"}extremely{"} likely to recommend the course. Participants reported improvement in several analytics capabilities including Epic data warehousing (p = 0.017), institutional review board policy (p = 0.005), and data stewardship (p = 0.007). Changes in practice patterns mirrored those in self-reported capability. On average, the participant professional network doubled. CONCLUSION:  LEADS was the first course targeted to working BMHI professional at a large academic medical center to have a formal effectiveness evaluation be published in the literature. The course achieved the goals of expansion of BMHI knowledge, skills, and professional networks. The LEADS course provides a template for continuing education of working BMHI professionals.",
author = "Steven Miller and Phillip Stablein and Jay Syed and Valerie Smothers and Emily Marx and Greene, {Peter Schuyler} and Lehmann, {Harold P} and Nagy, {Paul G.}",
year = "2019",
month = "8",
day = "1",
doi = "10.1055/s-0039-1694965",
language = "English (US)",
volume = "10",
pages = "634--642",
journal = "Applied Clinical Informatics",
issn = "1869-0327",
publisher = "Schattauer GmbH",
number = "4",

}

TY - JOUR

T1 - Evaluation of a Training Program to Improve Organizational Capacity for Health Systems Analytics

AU - Miller, Steven

AU - Stablein, Phillip

AU - Syed, Jay

AU - Smothers, Valerie

AU - Marx, Emily

AU - Greene, Peter Schuyler

AU - Lehmann, Harold P

AU - Nagy, Paul G.

PY - 2019/8/1

Y1 - 2019/8/1

N2 - OBJECTIVE:  The Leadership in Analytics and Data Science (LEADS) course was evaluated for effectiveness. LEADS was a 6-month program for working biomedical and health informatics (BMHI) professionals designed to improve analytics skills, knowledge of enterprise applications, data stewardship, and to foster an analytics community of practice through lectures, hands-on skill building workshops, networking events, and small group projects. METHODS:  The effectiveness of the LEADS course was evaluated using the Kirkpatrick Model by assessing pre- and postcourse knowledge, analytics capabilities, goals, practice, class lecture reaction, and change in the size of participant professional networks. Differences in pre- and postcourse responses were analyzed with a Wilcoxon signed rank test to determine significance, and effect sizes were computed using a z-statistic. RESULTS:  Twenty-nine students completed the course with 96% of respondents reporting that they were "very" or "extremely" likely to recommend the course. Participants reported improvement in several analytics capabilities including Epic data warehousing (p = 0.017), institutional review board policy (p = 0.005), and data stewardship (p = 0.007). Changes in practice patterns mirrored those in self-reported capability. On average, the participant professional network doubled. CONCLUSION:  LEADS was the first course targeted to working BMHI professional at a large academic medical center to have a formal effectiveness evaluation be published in the literature. The course achieved the goals of expansion of BMHI knowledge, skills, and professional networks. The LEADS course provides a template for continuing education of working BMHI professionals.

AB - OBJECTIVE:  The Leadership in Analytics and Data Science (LEADS) course was evaluated for effectiveness. LEADS was a 6-month program for working biomedical and health informatics (BMHI) professionals designed to improve analytics skills, knowledge of enterprise applications, data stewardship, and to foster an analytics community of practice through lectures, hands-on skill building workshops, networking events, and small group projects. METHODS:  The effectiveness of the LEADS course was evaluated using the Kirkpatrick Model by assessing pre- and postcourse knowledge, analytics capabilities, goals, practice, class lecture reaction, and change in the size of participant professional networks. Differences in pre- and postcourse responses were analyzed with a Wilcoxon signed rank test to determine significance, and effect sizes were computed using a z-statistic. RESULTS:  Twenty-nine students completed the course with 96% of respondents reporting that they were "very" or "extremely" likely to recommend the course. Participants reported improvement in several analytics capabilities including Epic data warehousing (p = 0.017), institutional review board policy (p = 0.005), and data stewardship (p = 0.007). Changes in practice patterns mirrored those in self-reported capability. On average, the participant professional network doubled. CONCLUSION:  LEADS was the first course targeted to working BMHI professional at a large academic medical center to have a formal effectiveness evaluation be published in the literature. The course achieved the goals of expansion of BMHI knowledge, skills, and professional networks. The LEADS course provides a template for continuing education of working BMHI professionals.

UR - http://www.scopus.com/inward/record.url?scp=85071633203&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85071633203&partnerID=8YFLogxK

U2 - 10.1055/s-0039-1694965

DO - 10.1055/s-0039-1694965

M3 - Article

C2 - 31461754

AN - SCOPUS:85071633203

VL - 10

SP - 634

EP - 642

JO - Applied Clinical Informatics

JF - Applied Clinical Informatics

SN - 1869-0327

IS - 4

ER -