Optimal observer framework and categorization observer framework for three-class ROC analysis

X. He, Eric Frey

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

ROC analysis has been an important tool for system evaluation and optimization in medical imaging. Despite its success in evaluating binary classification tasks, ROC analysis does not provide a direct way for evaluating performance on classification tasks that involve more than two diagnostic alternatives. We have previously developed a three-class ROC analysis method that provides a practical way to evaluate three-class task performance. Based on two-class ROC analysis and the proposed three-class ROC analysis method, this work proposes two frameworks, the optimal observer framework and the categorization observer framework, for three-class ROC analysis. The optimal observer framework seeks three-class decision rules and decision variables based on a formal decision strategy; it provides a ROC surface for system comparison on the basis of optimal performance with respect to this strategy. A categorization procedure is the generalization to 3-D of a 2-alternative forced choice procedure and is an important concept in the categorization observer framework. The categorization observer framework seeks three-class decision rules, decision variables and ROC surface such that task performance as measured by volume under the ROC surface (VUS) and the percent correct on the categorization procedure are equal. We then show that how our previously-proposed three-class ROC method fits into both frameworks.

Original languageEnglish (US)
Title of host publicationProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume6146
DOIs
Publication statusPublished - 2006
EventMedical Imaging 2006: Image Perception, Observer Performance, and Technology Assessment - San Diego, CA, United States
Duration: Feb 14 2006Feb 16 2006

Other

OtherMedical Imaging 2006: Image Perception, Observer Performance, and Technology Assessment
CountryUnited States
CitySan Diego, CA
Period2/14/062/16/06

    Fingerprint

Keywords

  • ROC analysis
  • Three-class classification

ASJC Scopus subject areas

  • Engineering(all)

Cite this

He, X., & Frey, E. (2006). Optimal observer framework and categorization observer framework for three-class ROC analysis. In Progress in Biomedical Optics and Imaging - Proceedings of SPIE (Vol. 6146). [61460B] https://doi.org/10.1117/12.654172