Simultaneous truth and performance level estimation with incomplete, over-complete, and ancillary data

Bennett A. Landman, John A. Bogovic, Jerry Ladd Prince

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Image labeling and parcellation are critical tasks for the assessment of volumetric and morphometric features in medical imaging data. The process of image labeling is inherently error prone as images are corrupted by noise and artifact. Even expert interpretations are subject to subjectivity and the precision of the individual raters. Hence, all labels must be considered imperfect with some degree of inherent variability. One may seek multiple independent assessments to both reduce this variability as well as quantify the degree of uncertainty. Existing techniques exploit maximum a posteriori statistics to combine data from multiple raters. A current limitation with these approaches is that they require each rater to generate a complete dataset, which is often impossible given both human foibles and the typical turnover rate of raters in a research or clinical environment. Herein, we propose a robust set of extensions that allow for missing data, account for repeated label sets, and utilize training/catch trial data. With these extensions, numerous raters can label small, overlapping portions of a large dataset, and rater heterogeneity can be robustly controlled while simultaneously estimating a single, reliable label set and characterizing uncertainty. The proposed approach enables parallel processing of labeling tasks and reduces the otherwise detrimental impact of rater unavailability.

Original languageEnglish (US)
Title of host publicationProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume7623
EditionPART 1
DOIs
StatePublished - 2010
EventMedical Imaging 2010: Image Processing - San Diego, CA, United States
Duration: Feb 14 2010Feb 16 2010

Other

OtherMedical Imaging 2010: Image Processing
CountryUnited States
CitySan Diego, CA
Period2/14/102/16/10

Fingerprint

marking
Uncertainty
Labels
Labeling
Diagnostic Imaging
Artifacts
Noise
artifacts
Medical imaging
education
estimating
Research
statistics
Statistics
Datasets
Processing

Keywords

  • analysis
  • data fusion
  • delineation
  • labeling
  • Parcellation
  • STAPLE
  • statistics

ASJC Scopus subject areas

  • Atomic and Molecular Physics, and Optics
  • Electronic, Optical and Magnetic Materials
  • Biomaterials
  • Radiology Nuclear Medicine and imaging

Cite this

Landman, B. A., Bogovic, J. A., & Prince, J. L. (2010). Simultaneous truth and performance level estimation with incomplete, over-complete, and ancillary data. In Progress in Biomedical Optics and Imaging - Proceedings of SPIE (PART 1 ed., Vol. 7623). [76231N] https://doi.org/10.1117/12.844182

Simultaneous truth and performance level estimation with incomplete, over-complete, and ancillary data. / Landman, Bennett A.; Bogovic, John A.; Prince, Jerry Ladd.

Progress in Biomedical Optics and Imaging - Proceedings of SPIE. Vol. 7623 PART 1. ed. 2010. 76231N.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Landman, BA, Bogovic, JA & Prince, JL 2010, Simultaneous truth and performance level estimation with incomplete, over-complete, and ancillary data. in Progress in Biomedical Optics and Imaging - Proceedings of SPIE. PART 1 edn, vol. 7623, 76231N, Medical Imaging 2010: Image Processing, San Diego, CA, United States, 2/14/10. https://doi.org/10.1117/12.844182
Landman BA, Bogovic JA, Prince JL. Simultaneous truth and performance level estimation with incomplete, over-complete, and ancillary data. In Progress in Biomedical Optics and Imaging - Proceedings of SPIE. PART 1 ed. Vol. 7623. 2010. 76231N https://doi.org/10.1117/12.844182
Landman, Bennett A. ; Bogovic, John A. ; Prince, Jerry Ladd. / Simultaneous truth and performance level estimation with incomplete, over-complete, and ancillary data. Progress in Biomedical Optics and Imaging - Proceedings of SPIE. Vol. 7623 PART 1. ed. 2010.
@inproceedings{7261eeb7767e48aa9d6964192d3725f8,
title = "Simultaneous truth and performance level estimation with incomplete, over-complete, and ancillary data",
abstract = "Image labeling and parcellation are critical tasks for the assessment of volumetric and morphometric features in medical imaging data. The process of image labeling is inherently error prone as images are corrupted by noise and artifact. Even expert interpretations are subject to subjectivity and the precision of the individual raters. Hence, all labels must be considered imperfect with some degree of inherent variability. One may seek multiple independent assessments to both reduce this variability as well as quantify the degree of uncertainty. Existing techniques exploit maximum a posteriori statistics to combine data from multiple raters. A current limitation with these approaches is that they require each rater to generate a complete dataset, which is often impossible given both human foibles and the typical turnover rate of raters in a research or clinical environment. Herein, we propose a robust set of extensions that allow for missing data, account for repeated label sets, and utilize training/catch trial data. With these extensions, numerous raters can label small, overlapping portions of a large dataset, and rater heterogeneity can be robustly controlled while simultaneously estimating a single, reliable label set and characterizing uncertainty. The proposed approach enables parallel processing of labeling tasks and reduces the otherwise detrimental impact of rater unavailability.",
keywords = "analysis, data fusion, delineation, labeling, Parcellation, STAPLE, statistics",
author = "Landman, {Bennett A.} and Bogovic, {John A.} and Prince, {Jerry Ladd}",
year = "2010",
doi = "10.1117/12.844182",
language = "English (US)",
isbn = "9780819480248",
volume = "7623",
booktitle = "Progress in Biomedical Optics and Imaging - Proceedings of SPIE",
edition = "PART 1",

}

TY - GEN

T1 - Simultaneous truth and performance level estimation with incomplete, over-complete, and ancillary data

AU - Landman, Bennett A.

AU - Bogovic, John A.

AU - Prince, Jerry Ladd

PY - 2010

Y1 - 2010

N2 - Image labeling and parcellation are critical tasks for the assessment of volumetric and morphometric features in medical imaging data. The process of image labeling is inherently error prone as images are corrupted by noise and artifact. Even expert interpretations are subject to subjectivity and the precision of the individual raters. Hence, all labels must be considered imperfect with some degree of inherent variability. One may seek multiple independent assessments to both reduce this variability as well as quantify the degree of uncertainty. Existing techniques exploit maximum a posteriori statistics to combine data from multiple raters. A current limitation with these approaches is that they require each rater to generate a complete dataset, which is often impossible given both human foibles and the typical turnover rate of raters in a research or clinical environment. Herein, we propose a robust set of extensions that allow for missing data, account for repeated label sets, and utilize training/catch trial data. With these extensions, numerous raters can label small, overlapping portions of a large dataset, and rater heterogeneity can be robustly controlled while simultaneously estimating a single, reliable label set and characterizing uncertainty. The proposed approach enables parallel processing of labeling tasks and reduces the otherwise detrimental impact of rater unavailability.

AB - Image labeling and parcellation are critical tasks for the assessment of volumetric and morphometric features in medical imaging data. The process of image labeling is inherently error prone as images are corrupted by noise and artifact. Even expert interpretations are subject to subjectivity and the precision of the individual raters. Hence, all labels must be considered imperfect with some degree of inherent variability. One may seek multiple independent assessments to both reduce this variability as well as quantify the degree of uncertainty. Existing techniques exploit maximum a posteriori statistics to combine data from multiple raters. A current limitation with these approaches is that they require each rater to generate a complete dataset, which is often impossible given both human foibles and the typical turnover rate of raters in a research or clinical environment. Herein, we propose a robust set of extensions that allow for missing data, account for repeated label sets, and utilize training/catch trial data. With these extensions, numerous raters can label small, overlapping portions of a large dataset, and rater heterogeneity can be robustly controlled while simultaneously estimating a single, reliable label set and characterizing uncertainty. The proposed approach enables parallel processing of labeling tasks and reduces the otherwise detrimental impact of rater unavailability.

KW - analysis

KW - data fusion

KW - delineation

KW - labeling

KW - Parcellation

KW - STAPLE

KW - statistics

UR - http://www.scopus.com/inward/record.url?scp=79751523494&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79751523494&partnerID=8YFLogxK

U2 - 10.1117/12.844182

DO - 10.1117/12.844182

M3 - Conference contribution

C2 - 20694058

AN - SCOPUS:79751523494

SN - 9780819480248

VL - 7623

BT - Progress in Biomedical Optics and Imaging - Proceedings of SPIE

ER -