Three-class ROC analysistoward a general decision theoretic solution

Xin He, Brandon D. Gallas, Eric Frey

Research output: Contribution to journalArticle

Abstract

Multiclass receiver operating characteristic (ROC) analysis has remained an open theoretical problem since the introduction of binary ROC analysis in the 1950s. Previously, we have developed a paradigm for three-class ROC analysis that extends and unifies decision theoretic, linear discriminant analysis, and probabilistic foundations of binary ROC analysis in a three-class paradigm. One critical element in this paradigm is the equal error utility (EEU) assumption. This assumption allows us to reduce the intrinsic space of the three-class ROC analysis (5-D hypersurface in 6-D hyperspace) to a 2-D surface in the 3-D space of true positive fractions (sensitivity space). In this work, we show that this 2-D ROC surface fully and uniquely provides a complete descriptor for the optimal performance of a system for a three-class classification task, i.e., the triplet of likelihood ratio distributions, assuming such a triplet exists. To be specific, we consider two classifiers that utilize likelihood ratios, and we assumed each classifier has a continuous and differentiable 2-D sensitivity-space ROC surface. Under these conditions, we proved that the classifiers have the same triplet of likelihood ratio distributions if and only if they have the same 2-D sensitivity-space ROC surfaces. As a result, the 2-D sensitivity surface contains complete information on the optimal three-class task performance for the corresponding likelihood ratio classifier.

Original languageEnglish (US)
Article number5306177
Pages (from-to)206-215
Number of pages10
JournalIEEE Transactions on Medical Imaging
Volume29
Issue number1
DOIs
StatePublished - Jan 2010

Fingerprint

ROC Curve
Classifiers
Discriminant analysis
Task Performance and Analysis
Discriminant Analysis

Keywords

  • Extended receiver operating characteristic (ROC) analysis
  • ROC analysis
  • Three-class ROC analysis

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Computer Science Applications
  • Radiological and Ultrasound Technology
  • Software

Cite this

Three-class ROC analysistoward a general decision theoretic solution. / He, Xin; Gallas, Brandon D.; Frey, Eric.

In: IEEE Transactions on Medical Imaging, Vol. 29, No. 1, 5306177, 01.2010, p. 206-215.

Research output: Contribution to journalArticle

@article{53729f0ab89b4b4cbea4f1f4417e7e2e,
title = "Three-class ROC analysistoward a general decision theoretic solution",
abstract = "Multiclass receiver operating characteristic (ROC) analysis has remained an open theoretical problem since the introduction of binary ROC analysis in the 1950s. Previously, we have developed a paradigm for three-class ROC analysis that extends and unifies decision theoretic, linear discriminant analysis, and probabilistic foundations of binary ROC analysis in a three-class paradigm. One critical element in this paradigm is the equal error utility (EEU) assumption. This assumption allows us to reduce the intrinsic space of the three-class ROC analysis (5-D hypersurface in 6-D hyperspace) to a 2-D surface in the 3-D space of true positive fractions (sensitivity space). In this work, we show that this 2-D ROC surface fully and uniquely provides a complete descriptor for the optimal performance of a system for a three-class classification task, i.e., the triplet of likelihood ratio distributions, assuming such a triplet exists. To be specific, we consider two classifiers that utilize likelihood ratios, and we assumed each classifier has a continuous and differentiable 2-D sensitivity-space ROC surface. Under these conditions, we proved that the classifiers have the same triplet of likelihood ratio distributions if and only if they have the same 2-D sensitivity-space ROC surfaces. As a result, the 2-D sensitivity surface contains complete information on the optimal three-class task performance for the corresponding likelihood ratio classifier.",
keywords = "Extended receiver operating characteristic (ROC) analysis, ROC analysis, Three-class ROC analysis",
author = "Xin He and Gallas, {Brandon D.} and Eric Frey",
year = "2010",
month = "1",
doi = "10.1109/TMI.2009.2034516",
language = "English (US)",
volume = "29",
pages = "206--215",
journal = "IEEE Transactions on Medical Imaging",
issn = "0278-0062",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "1",

}

TY - JOUR

T1 - Three-class ROC analysistoward a general decision theoretic solution

AU - He, Xin

AU - Gallas, Brandon D.

AU - Frey, Eric

PY - 2010/1

Y1 - 2010/1

N2 - Multiclass receiver operating characteristic (ROC) analysis has remained an open theoretical problem since the introduction of binary ROC analysis in the 1950s. Previously, we have developed a paradigm for three-class ROC analysis that extends and unifies decision theoretic, linear discriminant analysis, and probabilistic foundations of binary ROC analysis in a three-class paradigm. One critical element in this paradigm is the equal error utility (EEU) assumption. This assumption allows us to reduce the intrinsic space of the three-class ROC analysis (5-D hypersurface in 6-D hyperspace) to a 2-D surface in the 3-D space of true positive fractions (sensitivity space). In this work, we show that this 2-D ROC surface fully and uniquely provides a complete descriptor for the optimal performance of a system for a three-class classification task, i.e., the triplet of likelihood ratio distributions, assuming such a triplet exists. To be specific, we consider two classifiers that utilize likelihood ratios, and we assumed each classifier has a continuous and differentiable 2-D sensitivity-space ROC surface. Under these conditions, we proved that the classifiers have the same triplet of likelihood ratio distributions if and only if they have the same 2-D sensitivity-space ROC surfaces. As a result, the 2-D sensitivity surface contains complete information on the optimal three-class task performance for the corresponding likelihood ratio classifier.

AB - Multiclass receiver operating characteristic (ROC) analysis has remained an open theoretical problem since the introduction of binary ROC analysis in the 1950s. Previously, we have developed a paradigm for three-class ROC analysis that extends and unifies decision theoretic, linear discriminant analysis, and probabilistic foundations of binary ROC analysis in a three-class paradigm. One critical element in this paradigm is the equal error utility (EEU) assumption. This assumption allows us to reduce the intrinsic space of the three-class ROC analysis (5-D hypersurface in 6-D hyperspace) to a 2-D surface in the 3-D space of true positive fractions (sensitivity space). In this work, we show that this 2-D ROC surface fully and uniquely provides a complete descriptor for the optimal performance of a system for a three-class classification task, i.e., the triplet of likelihood ratio distributions, assuming such a triplet exists. To be specific, we consider two classifiers that utilize likelihood ratios, and we assumed each classifier has a continuous and differentiable 2-D sensitivity-space ROC surface. Under these conditions, we proved that the classifiers have the same triplet of likelihood ratio distributions if and only if they have the same 2-D sensitivity-space ROC surfaces. As a result, the 2-D sensitivity surface contains complete information on the optimal three-class task performance for the corresponding likelihood ratio classifier.

KW - Extended receiver operating characteristic (ROC) analysis

KW - ROC analysis

KW - Three-class ROC analysis

UR - http://www.scopus.com/inward/record.url?scp=73849120367&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=73849120367&partnerID=8YFLogxK

U2 - 10.1109/TMI.2009.2034516

DO - 10.1109/TMI.2009.2034516

M3 - Article

C2 - 19884079

AN - SCOPUS:73849120367

VL - 29

SP - 206

EP - 215

JO - IEEE Transactions on Medical Imaging

JF - IEEE Transactions on Medical Imaging

SN - 0278-0062

IS - 1

M1 - 5306177

ER -