Semi-supervised 3D abdominal multi-organ segmentation via deep multi-planar co-training

Yuyin Zhou, Yan Wang, Peng Tang, Song Bai, Wei Shen, Elliot K Fishman, Alan Yuille

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In multi-organ segmentation of abdominal CT scans, most existing fully supervised deep learning algorithms require lots of voxel-wise annotations, which are usually difficult, expensive, and slow to obtain. In comparison, massive unlabeled 3D CT volumes are usually easily accessible. Current mainstream works to address semi-supervised biomedical image segmentation problem are mostly graph-based. By contrast, deep network based semi-supervised learning methods have not drawn much attention in this field. In this work, we propose Deep Multi-Planar Co-Training (DMPCT), whose contributions can be divided into two folds: 1) The deep model is learned in a co-training style which can mine consensus information from multiple planes like the sagittal, coronal, and axial planes; 2) Multi-planar fusion is applied to generate more reliable pseudo-labels, which alleviates the errors occurring in the pseudo-labels and thus can help to train better segmentation networks. Experiments are done on our newly collected large dataset with 100 unlabeled cases as well as 210 labeled cases where 16 anatomical structures are manually annotated by four radiologists and confirmed by a senior expert. The results suggest that DMPCT significantly outperforms the fully supervised method by more than 4% especially when only a small set of annotations is used.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages121-140
Number of pages20
ISBN (Electronic)9781728119755
DOIs
StatePublished - Mar 4 2019
Event19th IEEE Winter Conference on Applications of Computer Vision, WACV 2019 - Waikoloa Village, United States
Duration: Jan 7 2019Jan 11 2019

Publication series

NameProceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019

Conference

Conference19th IEEE Winter Conference on Applications of Computer Vision, WACV 2019
CountryUnited States
CityWaikoloa Village
Period1/7/191/11/19

Fingerprint

Labels
Computerized tomography
Supervised learning
Image segmentation
Learning algorithms
Fusion reactions
Experiments
Deep learning

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Computer Science Applications

Cite this

Zhou, Y., Wang, Y., Tang, P., Bai, S., Shen, W., Fishman, E. K., & Yuille, A. (2019). Semi-supervised 3D abdominal multi-organ segmentation via deep multi-planar co-training. In Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019 (pp. 121-140). [8658899] (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACV.2019.00020

Semi-supervised 3D abdominal multi-organ segmentation via deep multi-planar co-training. / Zhou, Yuyin; Wang, Yan; Tang, Peng; Bai, Song; Shen, Wei; Fishman, Elliot K; Yuille, Alan.

Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 121-140 8658899 (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhou, Y, Wang, Y, Tang, P, Bai, S, Shen, W, Fishman, EK & Yuille, A 2019, Semi-supervised 3D abdominal multi-organ segmentation via deep multi-planar co-training. in Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019., 8658899, Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019, Institute of Electrical and Electronics Engineers Inc., pp. 121-140, 19th IEEE Winter Conference on Applications of Computer Vision, WACV 2019, Waikoloa Village, United States, 1/7/19. https://doi.org/10.1109/WACV.2019.00020
Zhou Y, Wang Y, Tang P, Bai S, Shen W, Fishman EK et al. Semi-supervised 3D abdominal multi-organ segmentation via deep multi-planar co-training. In Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 121-140. 8658899. (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019). https://doi.org/10.1109/WACV.2019.00020
Zhou, Yuyin ; Wang, Yan ; Tang, Peng ; Bai, Song ; Shen, Wei ; Fishman, Elliot K ; Yuille, Alan. / Semi-supervised 3D abdominal multi-organ segmentation via deep multi-planar co-training. Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 121-140 (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019).
@inproceedings{35d70593accd4687afb4161c4bd954d7,
title = "Semi-supervised 3D abdominal multi-organ segmentation via deep multi-planar co-training",
abstract = "In multi-organ segmentation of abdominal CT scans, most existing fully supervised deep learning algorithms require lots of voxel-wise annotations, which are usually difficult, expensive, and slow to obtain. In comparison, massive unlabeled 3D CT volumes are usually easily accessible. Current mainstream works to address semi-supervised biomedical image segmentation problem are mostly graph-based. By contrast, deep network based semi-supervised learning methods have not drawn much attention in this field. In this work, we propose Deep Multi-Planar Co-Training (DMPCT), whose contributions can be divided into two folds: 1) The deep model is learned in a co-training style which can mine consensus information from multiple planes like the sagittal, coronal, and axial planes; 2) Multi-planar fusion is applied to generate more reliable pseudo-labels, which alleviates the errors occurring in the pseudo-labels and thus can help to train better segmentation networks. Experiments are done on our newly collected large dataset with 100 unlabeled cases as well as 210 labeled cases where 16 anatomical structures are manually annotated by four radiologists and confirmed by a senior expert. The results suggest that DMPCT significantly outperforms the fully supervised method by more than 4{\%} especially when only a small set of annotations is used.",
author = "Yuyin Zhou and Yan Wang and Peng Tang and Song Bai and Wei Shen and Fishman, {Elliot K} and Alan Yuille",
year = "2019",
month = "3",
day = "4",
doi = "10.1109/WACV.2019.00020",
language = "English (US)",
series = "Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "121--140",
booktitle = "Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019",

}

TY - GEN

T1 - Semi-supervised 3D abdominal multi-organ segmentation via deep multi-planar co-training

AU - Zhou, Yuyin

AU - Wang, Yan

AU - Tang, Peng

AU - Bai, Song

AU - Shen, Wei

AU - Fishman, Elliot K

AU - Yuille, Alan

PY - 2019/3/4

Y1 - 2019/3/4

N2 - In multi-organ segmentation of abdominal CT scans, most existing fully supervised deep learning algorithms require lots of voxel-wise annotations, which are usually difficult, expensive, and slow to obtain. In comparison, massive unlabeled 3D CT volumes are usually easily accessible. Current mainstream works to address semi-supervised biomedical image segmentation problem are mostly graph-based. By contrast, deep network based semi-supervised learning methods have not drawn much attention in this field. In this work, we propose Deep Multi-Planar Co-Training (DMPCT), whose contributions can be divided into two folds: 1) The deep model is learned in a co-training style which can mine consensus information from multiple planes like the sagittal, coronal, and axial planes; 2) Multi-planar fusion is applied to generate more reliable pseudo-labels, which alleviates the errors occurring in the pseudo-labels and thus can help to train better segmentation networks. Experiments are done on our newly collected large dataset with 100 unlabeled cases as well as 210 labeled cases where 16 anatomical structures are manually annotated by four radiologists and confirmed by a senior expert. The results suggest that DMPCT significantly outperforms the fully supervised method by more than 4% especially when only a small set of annotations is used.

AB - In multi-organ segmentation of abdominal CT scans, most existing fully supervised deep learning algorithms require lots of voxel-wise annotations, which are usually difficult, expensive, and slow to obtain. In comparison, massive unlabeled 3D CT volumes are usually easily accessible. Current mainstream works to address semi-supervised biomedical image segmentation problem are mostly graph-based. By contrast, deep network based semi-supervised learning methods have not drawn much attention in this field. In this work, we propose Deep Multi-Planar Co-Training (DMPCT), whose contributions can be divided into two folds: 1) The deep model is learned in a co-training style which can mine consensus information from multiple planes like the sagittal, coronal, and axial planes; 2) Multi-planar fusion is applied to generate more reliable pseudo-labels, which alleviates the errors occurring in the pseudo-labels and thus can help to train better segmentation networks. Experiments are done on our newly collected large dataset with 100 unlabeled cases as well as 210 labeled cases where 16 anatomical structures are manually annotated by four radiologists and confirmed by a senior expert. The results suggest that DMPCT significantly outperforms the fully supervised method by more than 4% especially when only a small set of annotations is used.

UR - http://www.scopus.com/inward/record.url?scp=85063592457&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063592457&partnerID=8YFLogxK

U2 - 10.1109/WACV.2019.00020

DO - 10.1109/WACV.2019.00020

M3 - Conference contribution

AN - SCOPUS:85063592457

T3 - Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019

SP - 121

EP - 140

BT - Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -