Teaching Receiver Operating Characteristic Analysis. An Interactive Laboratory Exercise

Research output: Contribution to journalArticle

Abstract

Rationale and Objectives: Despite its fundamental importance in the evaluation of diagnostic tests, receiver operating characteristic (ROC) analysis is not easily understood. The purpose of this project was to create a learning experience that resulted in an intuitive understanding of the basic principles of ROC analysis. Materials and Methods: An interactive laboratory exercise was developed for a class about radiology testing taught within a clinical epidemiology course between 2000 and 2009. The physician students in the course were clinical fellows from various medical specialties who were enrolled in a graduate degree program in clinical investigation. For the exercise, the class was divided into six groups. Each group interpreted radiographs from a set of 50 exams of the peripheral skeleton to determine the presence or absence of an acute fracture. Data from the class were pooled and given to each student. Students calculated the area under the ROC curve (AUC) corresponding to overall class performance. A binormal ROC curve was also fitted to the data from each class year. Results: The laboratory exercise was conducted for 8 years with approximately 20-30 students per year. The mean AUC over the eight laboratory classes was 0.72 with a standard deviation of 0.08 (range, 0.60-0.85). Conclusion: With some simplifications in design, an observer study can be conducted in a laboratory classroom setting. Participatory data collection promotes the intuitive understanding of ROC analysis principles.

Original languageEnglish (US)
Pages (from-to)1452-1456
Number of pages5
JournalAcademic Radiology
Volume19
Issue number12
DOIs
StatePublished - Dec 2012

Fingerprint

ROC Curve
Teaching
Exercise
Students
Routine Diagnostic Tests
Radiology
Skeleton
Area Under Curve
Epidemiology
Medicine
Learning
Physicians

Keywords

  • Diagnostic accuracy
  • Education
  • Observer study
  • Receiver operating characteristic analysis
  • ROC analysis

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

Teaching Receiver Operating Characteristic Analysis. An Interactive Laboratory Exercise. / Eng, John.

In: Academic Radiology, Vol. 19, No. 12, 12.2012, p. 1452-1456.

Research output: Contribution to journalArticle

@article{94373f2fe9a14c82801c3dd73ce89421,
title = "Teaching Receiver Operating Characteristic Analysis. An Interactive Laboratory Exercise",
abstract = "Rationale and Objectives: Despite its fundamental importance in the evaluation of diagnostic tests, receiver operating characteristic (ROC) analysis is not easily understood. The purpose of this project was to create a learning experience that resulted in an intuitive understanding of the basic principles of ROC analysis. Materials and Methods: An interactive laboratory exercise was developed for a class about radiology testing taught within a clinical epidemiology course between 2000 and 2009. The physician students in the course were clinical fellows from various medical specialties who were enrolled in a graduate degree program in clinical investigation. For the exercise, the class was divided into six groups. Each group interpreted radiographs from a set of 50 exams of the peripheral skeleton to determine the presence or absence of an acute fracture. Data from the class were pooled and given to each student. Students calculated the area under the ROC curve (AUC) corresponding to overall class performance. A binormal ROC curve was also fitted to the data from each class year. Results: The laboratory exercise was conducted for 8 years with approximately 20-30 students per year. The mean AUC over the eight laboratory classes was 0.72 with a standard deviation of 0.08 (range, 0.60-0.85). Conclusion: With some simplifications in design, an observer study can be conducted in a laboratory classroom setting. Participatory data collection promotes the intuitive understanding of ROC analysis principles.",
keywords = "Diagnostic accuracy, Education, Observer study, Receiver operating characteristic analysis, ROC analysis",
author = "John Eng",
year = "2012",
month = "12",
doi = "10.1016/j.acra.2012.09.003",
language = "English (US)",
volume = "19",
pages = "1452--1456",
journal = "Academic Radiology",
issn = "1076-6332",
publisher = "Elsevier USA",
number = "12",

}

TY - JOUR

T1 - Teaching Receiver Operating Characteristic Analysis. An Interactive Laboratory Exercise

AU - Eng, John

PY - 2012/12

Y1 - 2012/12

N2 - Rationale and Objectives: Despite its fundamental importance in the evaluation of diagnostic tests, receiver operating characteristic (ROC) analysis is not easily understood. The purpose of this project was to create a learning experience that resulted in an intuitive understanding of the basic principles of ROC analysis. Materials and Methods: An interactive laboratory exercise was developed for a class about radiology testing taught within a clinical epidemiology course between 2000 and 2009. The physician students in the course were clinical fellows from various medical specialties who were enrolled in a graduate degree program in clinical investigation. For the exercise, the class was divided into six groups. Each group interpreted radiographs from a set of 50 exams of the peripheral skeleton to determine the presence or absence of an acute fracture. Data from the class were pooled and given to each student. Students calculated the area under the ROC curve (AUC) corresponding to overall class performance. A binormal ROC curve was also fitted to the data from each class year. Results: The laboratory exercise was conducted for 8 years with approximately 20-30 students per year. The mean AUC over the eight laboratory classes was 0.72 with a standard deviation of 0.08 (range, 0.60-0.85). Conclusion: With some simplifications in design, an observer study can be conducted in a laboratory classroom setting. Participatory data collection promotes the intuitive understanding of ROC analysis principles.

AB - Rationale and Objectives: Despite its fundamental importance in the evaluation of diagnostic tests, receiver operating characteristic (ROC) analysis is not easily understood. The purpose of this project was to create a learning experience that resulted in an intuitive understanding of the basic principles of ROC analysis. Materials and Methods: An interactive laboratory exercise was developed for a class about radiology testing taught within a clinical epidemiology course between 2000 and 2009. The physician students in the course were clinical fellows from various medical specialties who were enrolled in a graduate degree program in clinical investigation. For the exercise, the class was divided into six groups. Each group interpreted radiographs from a set of 50 exams of the peripheral skeleton to determine the presence or absence of an acute fracture. Data from the class were pooled and given to each student. Students calculated the area under the ROC curve (AUC) corresponding to overall class performance. A binormal ROC curve was also fitted to the data from each class year. Results: The laboratory exercise was conducted for 8 years with approximately 20-30 students per year. The mean AUC over the eight laboratory classes was 0.72 with a standard deviation of 0.08 (range, 0.60-0.85). Conclusion: With some simplifications in design, an observer study can be conducted in a laboratory classroom setting. Participatory data collection promotes the intuitive understanding of ROC analysis principles.

KW - Diagnostic accuracy

KW - Education

KW - Observer study

KW - Receiver operating characteristic analysis

KW - ROC analysis

UR - http://www.scopus.com/inward/record.url?scp=84868300152&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84868300152&partnerID=8YFLogxK

U2 - 10.1016/j.acra.2012.09.003

DO - 10.1016/j.acra.2012.09.003

M3 - Article

C2 - 23040502

AN - SCOPUS:84868300152

VL - 19

SP - 1452

EP - 1456

JO - Academic Radiology

JF - Academic Radiology

SN - 1076-6332

IS - 12

ER -