Unpaired brain mr-to-ct synthesis using a structure-constrained cyclegan

Heran Yang, Jian Sun, Aaron Carass, Can Zhao, Junghoon Lee, Zongben Xu, Jerry Ladd Prince

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The cycleGAN is becoming an influential method in medical image synthesis. However, due to a lack of direct constraints between input and synthetic images, the cycleGAN cannot guarantee structural consistency between these two images, and such consistency is of extreme importance in medical imaging. To overcome this, we propose a structure-constrained cycleGAN for brain MR-to-CT synthesis using unpaired data that defines an extra structure-consistency loss based on the modality independent neighborhood descriptor to constrain structural consistency. Additionally, we use a position-based selection strategy for selecting training images instead of a completely random selection scheme. Experimental results on synthesizing CT images from brain MR images demonstrate that our method is better than the conventional cycleGAN and approximates the cycleGAN trained with paired data.

Original languageEnglish (US)
Title of host publicationDeep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 4th International Workshop, DLMIA 2018 and 8th International Workshop, ML-CDS 2018 Held in Conjunction with MICCAI 2018
EditorsLena Maier-Hein, Tanveer Syeda-Mahmood, Zeike Taylor, Zhi Lu, Danail Stoyanov, Anant Madabhushi, João Manuel R.S. Tavares, Jacinto C. Nascimento, Mehdi Moradi, Anne Martel, Joao Paulo Papa, Sailesh Conjeti, Vasileios Belagiannis, Hayit Greenspan, Gustavo Carneiro, Andrew Bradley
PublisherSpringer Verlag
Pages174-182
Number of pages9
ISBN (Print)9783030008888
DOIs
StatePublished - Jan 1 2018
Event4th International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2018 and 8th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2018 Held in Conjunction with MICCAI 2018 - Granada, Spain
Duration: Sep 20 2018Sep 20 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11045 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other4th International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2018 and 8th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2018 Held in Conjunction with MICCAI 2018
CountrySpain
CityGranada
Period9/20/189/20/18

Fingerprint

Brain
Synthesis
Medical imaging
Paired Data
CT Image
Medical Imaging
Medical Image
Modality
Descriptors
Extremes
Experimental Results
Demonstrate

Keywords

  • CycleGAN
  • Deep learning
  • MIND
  • MR-to-CT synthesis

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Yang, H., Sun, J., Carass, A., Zhao, C., Lee, J., Xu, Z., & Prince, J. L. (2018). Unpaired brain mr-to-ct synthesis using a structure-constrained cyclegan. In L. Maier-Hein, T. Syeda-Mahmood, Z. Taylor, Z. Lu, D. Stoyanov, A. Madabhushi, J. M. R. S. Tavares, J. C. Nascimento, M. Moradi, A. Martel, J. P. Papa, S. Conjeti, V. Belagiannis, H. Greenspan, G. Carneiro, ... A. Bradley (Eds.), Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 4th International Workshop, DLMIA 2018 and 8th International Workshop, ML-CDS 2018 Held in Conjunction with MICCAI 2018 (pp. 174-182). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11045 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-00889-5_20

Unpaired brain mr-to-ct synthesis using a structure-constrained cyclegan. / Yang, Heran; Sun, Jian; Carass, Aaron; Zhao, Can; Lee, Junghoon; Xu, Zongben; Prince, Jerry Ladd.

Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 4th International Workshop, DLMIA 2018 and 8th International Workshop, ML-CDS 2018 Held in Conjunction with MICCAI 2018. ed. / Lena Maier-Hein; Tanveer Syeda-Mahmood; Zeike Taylor; Zhi Lu; Danail Stoyanov; Anant Madabhushi; João Manuel R.S. Tavares; Jacinto C. Nascimento; Mehdi Moradi; Anne Martel; Joao Paulo Papa; Sailesh Conjeti; Vasileios Belagiannis; Hayit Greenspan; Gustavo Carneiro; Andrew Bradley. Springer Verlag, 2018. p. 174-182 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11045 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yang, H, Sun, J, Carass, A, Zhao, C, Lee, J, Xu, Z & Prince, JL 2018, Unpaired brain mr-to-ct synthesis using a structure-constrained cyclegan. in L Maier-Hein, T Syeda-Mahmood, Z Taylor, Z Lu, D Stoyanov, A Madabhushi, JMRS Tavares, JC Nascimento, M Moradi, A Martel, JP Papa, S Conjeti, V Belagiannis, H Greenspan, G Carneiro & A Bradley (eds), Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 4th International Workshop, DLMIA 2018 and 8th International Workshop, ML-CDS 2018 Held in Conjunction with MICCAI 2018. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11045 LNCS, Springer Verlag, pp. 174-182, 4th International Workshop on Deep Learning in Medical Image Analysis, DLMIA 2018 and 8th International Workshop on Multimodal Learning for Clinical Decision Support, ML-CDS 2018 Held in Conjunction with MICCAI 2018, Granada, Spain, 9/20/18. https://doi.org/10.1007/978-3-030-00889-5_20
Yang H, Sun J, Carass A, Zhao C, Lee J, Xu Z et al. Unpaired brain mr-to-ct synthesis using a structure-constrained cyclegan. In Maier-Hein L, Syeda-Mahmood T, Taylor Z, Lu Z, Stoyanov D, Madabhushi A, Tavares JMRS, Nascimento JC, Moradi M, Martel A, Papa JP, Conjeti S, Belagiannis V, Greenspan H, Carneiro G, Bradley A, editors, Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 4th International Workshop, DLMIA 2018 and 8th International Workshop, ML-CDS 2018 Held in Conjunction with MICCAI 2018. Springer Verlag. 2018. p. 174-182. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-00889-5_20
Yang, Heran ; Sun, Jian ; Carass, Aaron ; Zhao, Can ; Lee, Junghoon ; Xu, Zongben ; Prince, Jerry Ladd. / Unpaired brain mr-to-ct synthesis using a structure-constrained cyclegan. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 4th International Workshop, DLMIA 2018 and 8th International Workshop, ML-CDS 2018 Held in Conjunction with MICCAI 2018. editor / Lena Maier-Hein ; Tanveer Syeda-Mahmood ; Zeike Taylor ; Zhi Lu ; Danail Stoyanov ; Anant Madabhushi ; João Manuel R.S. Tavares ; Jacinto C. Nascimento ; Mehdi Moradi ; Anne Martel ; Joao Paulo Papa ; Sailesh Conjeti ; Vasileios Belagiannis ; Hayit Greenspan ; Gustavo Carneiro ; Andrew Bradley. Springer Verlag, 2018. pp. 174-182 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{2db8d14cd37341e8a74d1b6db9529d68,
title = "Unpaired brain mr-to-ct synthesis using a structure-constrained cyclegan",
abstract = "The cycleGAN is becoming an influential method in medical image synthesis. However, due to a lack of direct constraints between input and synthetic images, the cycleGAN cannot guarantee structural consistency between these two images, and such consistency is of extreme importance in medical imaging. To overcome this, we propose a structure-constrained cycleGAN for brain MR-to-CT synthesis using unpaired data that defines an extra structure-consistency loss based on the modality independent neighborhood descriptor to constrain structural consistency. Additionally, we use a position-based selection strategy for selecting training images instead of a completely random selection scheme. Experimental results on synthesizing CT images from brain MR images demonstrate that our method is better than the conventional cycleGAN and approximates the cycleGAN trained with paired data.",
keywords = "CycleGAN, Deep learning, MIND, MR-to-CT synthesis",
author = "Heran Yang and Jian Sun and Aaron Carass and Can Zhao and Junghoon Lee and Zongben Xu and Prince, {Jerry Ladd}",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-030-00889-5_20",
language = "English (US)",
isbn = "9783030008888",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "174--182",
editor = "Lena Maier-Hein and Tanveer Syeda-Mahmood and Zeike Taylor and Zhi Lu and Danail Stoyanov and Anant Madabhushi and Tavares, {Jo{\~a}o Manuel R.S.} and Nascimento, {Jacinto C.} and Mehdi Moradi and Anne Martel and Papa, {Joao Paulo} and Sailesh Conjeti and Vasileios Belagiannis and Hayit Greenspan and Gustavo Carneiro and Andrew Bradley",
booktitle = "Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 4th International Workshop, DLMIA 2018 and 8th International Workshop, ML-CDS 2018 Held in Conjunction with MICCAI 2018",

}

TY - GEN

T1 - Unpaired brain mr-to-ct synthesis using a structure-constrained cyclegan

AU - Yang, Heran

AU - Sun, Jian

AU - Carass, Aaron

AU - Zhao, Can

AU - Lee, Junghoon

AU - Xu, Zongben

AU - Prince, Jerry Ladd

PY - 2018/1/1

Y1 - 2018/1/1

N2 - The cycleGAN is becoming an influential method in medical image synthesis. However, due to a lack of direct constraints between input and synthetic images, the cycleGAN cannot guarantee structural consistency between these two images, and such consistency is of extreme importance in medical imaging. To overcome this, we propose a structure-constrained cycleGAN for brain MR-to-CT synthesis using unpaired data that defines an extra structure-consistency loss based on the modality independent neighborhood descriptor to constrain structural consistency. Additionally, we use a position-based selection strategy for selecting training images instead of a completely random selection scheme. Experimental results on synthesizing CT images from brain MR images demonstrate that our method is better than the conventional cycleGAN and approximates the cycleGAN trained with paired data.

AB - The cycleGAN is becoming an influential method in medical image synthesis. However, due to a lack of direct constraints between input and synthetic images, the cycleGAN cannot guarantee structural consistency between these two images, and such consistency is of extreme importance in medical imaging. To overcome this, we propose a structure-constrained cycleGAN for brain MR-to-CT synthesis using unpaired data that defines an extra structure-consistency loss based on the modality independent neighborhood descriptor to constrain structural consistency. Additionally, we use a position-based selection strategy for selecting training images instead of a completely random selection scheme. Experimental results on synthesizing CT images from brain MR images demonstrate that our method is better than the conventional cycleGAN and approximates the cycleGAN trained with paired data.

KW - CycleGAN

KW - Deep learning

KW - MIND

KW - MR-to-CT synthesis

UR - http://www.scopus.com/inward/record.url?scp=85057230081&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057230081&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-00889-5_20

DO - 10.1007/978-3-030-00889-5_20

M3 - Conference contribution

AN - SCOPUS:85057230081

SN - 9783030008888

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 174

EP - 182

BT - Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support - 4th International Workshop, DLMIA 2018 and 8th International Workshop, ML-CDS 2018 Held in Conjunction with MICCAI 2018

A2 - Maier-Hein, Lena

A2 - Syeda-Mahmood, Tanveer

A2 - Taylor, Zeike

A2 - Lu, Zhi

A2 - Stoyanov, Danail

A2 - Madabhushi, Anant

A2 - Tavares, João Manuel R.S.

A2 - Nascimento, Jacinto C.

A2 - Moradi, Mehdi

A2 - Martel, Anne

A2 - Papa, Joao Paulo

A2 - Conjeti, Sailesh

A2 - Belagiannis, Vasileios

A2 - Greenspan, Hayit

A2 - Carneiro, Gustavo

A2 - Bradley, Andrew

PB - Springer Verlag

ER -