Adversarial U-net with spectral normalization for histopathology image segmentation using synthetic data

Faisal Mahmood, Richard Chen, Daniel Borders, Gregory N. McKay, Kevan Salimian, Alexander Baras, Nicholas J. Durr

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Automated segmentation of tissue and cellular structure in H&E images is an important first step towards automated histopathology slide analysis. For example, nuclei segmentation can aid with detecting pleomorphism and epithelium segmentation can aid in identification of tumor infiltrating lymphocytes etc. Existing deep learning-based approaches are often trained organ-wise and lack diversity of training data for multi-organ segmentation networks. In this work, we propose to augment existing nuclei segmentation datasets using cycleGANs. We learn an unpaired mapping from perturbed randomized polygon masks to pseudo-H&E images. We generate over synthetic H&E patches from several different organs for nuclei segmentation. We then use an adversarial U-Net with spectral normalization for increased training stability for segmentation. This paired image-to-image translation style network not only learns the mapping form H&E patches to segmentation masks but also learns an optimal loss function. Such an approach eliminates the need for a hand-crafted loss which has been explored significantly for nuclei segmentation. We demonstrate that the average accuracy for multi-organ nuclei segmentation increases to 94.43% using the proposed synthetic data generation and adversarial U-Net-based segmentation pipeline as compared to 79.81% when no synthetic data and adversarial loss was used.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2019
Subtitle of host publicationDigital Pathology
EditorsJohn E. Tomaszewski, Aaron D. Ward
PublisherSPIE
ISBN (Electronic)9781510625594
DOIs
StatePublished - Jan 1 2019
EventMedical Imaging 2019: Digital Pathology - San Diego, United States
Duration: Feb 20 2019Feb 21 2019

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume10956
ISSN (Print)1605-7422

Conference

ConferenceMedical Imaging 2019: Digital Pathology
CountryUnited States
CitySan Diego
Period2/20/192/21/19

Fingerprint

Masks
Image segmentation
organs
Tumor-Infiltrating Lymphocytes
nuclei
Cellular Structures
Lymphocytes
Epithelium
Hand
Learning
education
masks
Tumors
Pipelines
epithelium
Tissue
polygons
lymphocytes
chutes
learning

Keywords

  • Computational Pathology
  • Neuclei Segmentation
  • Pathology Segmentation

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging

Cite this

Mahmood, F., Chen, R., Borders, D., McKay, G. N., Salimian, K., Baras, A., & Durr, N. J. (2019). Adversarial U-net with spectral normalization for histopathology image segmentation using synthetic data. In J. E. Tomaszewski, & A. D. Ward (Eds.), Medical Imaging 2019: Digital Pathology [109560N] (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 10956). SPIE. https://doi.org/10.1117/12.2512918

Adversarial U-net with spectral normalization for histopathology image segmentation using synthetic data. / Mahmood, Faisal; Chen, Richard; Borders, Daniel; McKay, Gregory N.; Salimian, Kevan; Baras, Alexander; Durr, Nicholas J.

Medical Imaging 2019: Digital Pathology. ed. / John E. Tomaszewski; Aaron D. Ward. SPIE, 2019. 109560N (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 10956).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mahmood, F, Chen, R, Borders, D, McKay, GN, Salimian, K, Baras, A & Durr, NJ 2019, Adversarial U-net with spectral normalization for histopathology image segmentation using synthetic data. in JE Tomaszewski & AD Ward (eds), Medical Imaging 2019: Digital Pathology., 109560N, Progress in Biomedical Optics and Imaging - Proceedings of SPIE, vol. 10956, SPIE, Medical Imaging 2019: Digital Pathology, San Diego, United States, 2/20/19. https://doi.org/10.1117/12.2512918
Mahmood F, Chen R, Borders D, McKay GN, Salimian K, Baras A et al. Adversarial U-net with spectral normalization for histopathology image segmentation using synthetic data. In Tomaszewski JE, Ward AD, editors, Medical Imaging 2019: Digital Pathology. SPIE. 2019. 109560N. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE). https://doi.org/10.1117/12.2512918
Mahmood, Faisal ; Chen, Richard ; Borders, Daniel ; McKay, Gregory N. ; Salimian, Kevan ; Baras, Alexander ; Durr, Nicholas J. / Adversarial U-net with spectral normalization for histopathology image segmentation using synthetic data. Medical Imaging 2019: Digital Pathology. editor / John E. Tomaszewski ; Aaron D. Ward. SPIE, 2019. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE).
@inproceedings{523b84e224bb400e94e6ffdb1c34d85a,
title = "Adversarial U-net with spectral normalization for histopathology image segmentation using synthetic data",
abstract = "Automated segmentation of tissue and cellular structure in H&E images is an important first step towards automated histopathology slide analysis. For example, nuclei segmentation can aid with detecting pleomorphism and epithelium segmentation can aid in identification of tumor infiltrating lymphocytes etc. Existing deep learning-based approaches are often trained organ-wise and lack diversity of training data for multi-organ segmentation networks. In this work, we propose to augment existing nuclei segmentation datasets using cycleGANs. We learn an unpaired mapping from perturbed randomized polygon masks to pseudo-H&E images. We generate over synthetic H&E patches from several different organs for nuclei segmentation. We then use an adversarial U-Net with spectral normalization for increased training stability for segmentation. This paired image-to-image translation style network not only learns the mapping form H&E patches to segmentation masks but also learns an optimal loss function. Such an approach eliminates the need for a hand-crafted loss which has been explored significantly for nuclei segmentation. We demonstrate that the average accuracy for multi-organ nuclei segmentation increases to 94.43{\%} using the proposed synthetic data generation and adversarial U-Net-based segmentation pipeline as compared to 79.81{\%} when no synthetic data and adversarial loss was used.",
keywords = "Computational Pathology, Neuclei Segmentation, Pathology Segmentation",
author = "Faisal Mahmood and Richard Chen and Daniel Borders and McKay, {Gregory N.} and Kevan Salimian and Alexander Baras and Durr, {Nicholas J.}",
year = "2019",
month = "1",
day = "1",
doi = "10.1117/12.2512918",
language = "English (US)",
series = "Progress in Biomedical Optics and Imaging - Proceedings of SPIE",
publisher = "SPIE",
editor = "Tomaszewski, {John E.} and Ward, {Aaron D.}",
booktitle = "Medical Imaging 2019",

}

TY - GEN

T1 - Adversarial U-net with spectral normalization for histopathology image segmentation using synthetic data

AU - Mahmood, Faisal

AU - Chen, Richard

AU - Borders, Daniel

AU - McKay, Gregory N.

AU - Salimian, Kevan

AU - Baras, Alexander

AU - Durr, Nicholas J.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Automated segmentation of tissue and cellular structure in H&E images is an important first step towards automated histopathology slide analysis. For example, nuclei segmentation can aid with detecting pleomorphism and epithelium segmentation can aid in identification of tumor infiltrating lymphocytes etc. Existing deep learning-based approaches are often trained organ-wise and lack diversity of training data for multi-organ segmentation networks. In this work, we propose to augment existing nuclei segmentation datasets using cycleGANs. We learn an unpaired mapping from perturbed randomized polygon masks to pseudo-H&E images. We generate over synthetic H&E patches from several different organs for nuclei segmentation. We then use an adversarial U-Net with spectral normalization for increased training stability for segmentation. This paired image-to-image translation style network not only learns the mapping form H&E patches to segmentation masks but also learns an optimal loss function. Such an approach eliminates the need for a hand-crafted loss which has been explored significantly for nuclei segmentation. We demonstrate that the average accuracy for multi-organ nuclei segmentation increases to 94.43% using the proposed synthetic data generation and adversarial U-Net-based segmentation pipeline as compared to 79.81% when no synthetic data and adversarial loss was used.

AB - Automated segmentation of tissue and cellular structure in H&E images is an important first step towards automated histopathology slide analysis. For example, nuclei segmentation can aid with detecting pleomorphism and epithelium segmentation can aid in identification of tumor infiltrating lymphocytes etc. Existing deep learning-based approaches are often trained organ-wise and lack diversity of training data for multi-organ segmentation networks. In this work, we propose to augment existing nuclei segmentation datasets using cycleGANs. We learn an unpaired mapping from perturbed randomized polygon masks to pseudo-H&E images. We generate over synthetic H&E patches from several different organs for nuclei segmentation. We then use an adversarial U-Net with spectral normalization for increased training stability for segmentation. This paired image-to-image translation style network not only learns the mapping form H&E patches to segmentation masks but also learns an optimal loss function. Such an approach eliminates the need for a hand-crafted loss which has been explored significantly for nuclei segmentation. We demonstrate that the average accuracy for multi-organ nuclei segmentation increases to 94.43% using the proposed synthetic data generation and adversarial U-Net-based segmentation pipeline as compared to 79.81% when no synthetic data and adversarial loss was used.

KW - Computational Pathology

KW - Neuclei Segmentation

KW - Pathology Segmentation

UR - http://www.scopus.com/inward/record.url?scp=85068711750&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068711750&partnerID=8YFLogxK

U2 - 10.1117/12.2512918

DO - 10.1117/12.2512918

M3 - Conference contribution

AN - SCOPUS:85068711750

T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE

BT - Medical Imaging 2019

A2 - Tomaszewski, John E.

A2 - Ward, Aaron D.

PB - SPIE

ER -