Optimal observer framework and categorization observer framework for three-class ROC analysis

X. He, E. C. Frey

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

ROC analysis has been an important tool for system evaluation and optimization in medical imaging. Despite its success in evaluating binary classification tasks, ROC analysis does not provide a direct way for evaluating performance on classification tasks that involve more than two diagnostic alternatives. We have previously developed a three-class ROC analysis method that provides a practical way to evaluate three-class task performance. Based on two-class ROC analysis and the proposed three-class ROC analysis method, this work proposes two frameworks, the optimal observer framework and the categorization observer framework, for three-class ROC analysis. The optimal observer framework seeks three-class decision rules and decision variables based on a formal decision strategy; it provides a ROC surface for system comparison on the basis of optimal performance with respect to this strategy. A categorization procedure is the generalization to 3-D of a 2-alternative forced choice procedure and is an important concept in the categorization observer framework. The categorization observer framework seeks three-class decision rules, decision variables and ROC surface such that task performance as measured by volume under the ROC surface (VUS) and the percent correct on the categorization procedure are equal. We then show that how our previously-proposed three-class ROC method fits into both frameworks.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2006
Subtitle of host publicationImage Perception, Observer Performance, and Technology Assessment
DOIs
StatePublished - Jun 22 2006
EventMedical Imaging 2006: Image Perception, Observer Performance, and Technology Assessment - San Diego, CA, United States
Duration: Feb 14 2006Feb 16 2006

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume6146
ISSN (Print)1605-7422

Other

OtherMedical Imaging 2006: Image Perception, Observer Performance, and Technology Assessment
CountryUnited States
CitySan Diego, CA
Period2/14/062/16/06

Keywords

  • ROC analysis
  • Three-class classification

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Biomaterials
  • Atomic and Molecular Physics, and Optics
  • Radiology Nuclear Medicine and imaging

Fingerprint Dive into the research topics of 'Optimal observer framework and categorization observer framework for three-class ROC analysis'. Together they form a unique fingerprint.

Cite this