Cross-modality image synthesis from unpaired data using cyclegan: Effects of gradient consistency loss and training data size

Yuta Hiasa; Yoshito Otake; Masaki Takao; Takumi Matsuoka; Kazuma Takashima; Aaron Carass; Jerry L. Prince; Nobuhiko Sugano; Yoshinobu Sato

doi:10.1007/978-3-030-00536-8_4

Cross-modality image synthesis from unpaired data using cyclegan: Effects of gradient consistency loss and training data size

Yuta Hiasa, Yoshito Otake, Masaki Takao, Takumi Matsuoka, Kazuma Takashima, Aaron Carass, Jerry L. Prince, Nobuhiko Sugano, Yoshinobu Sato

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

45 Scopus citations

Abstract

CT is commonly used in orthopedic procedures. MRI is used along with CT to identify muscle structures and diagnose osteonecrosis due to its superior soft tissue contrast. However, MRI has poor contrast for bone structures. Clearly, it would be helpful if a corresponding CT were available, as bone boundaries are more clearly seen and CT has a standardized (i.e., Hounsfield) unit. Therefore, we aim at MR-to-CT synthesis. While the CycleGAN was successfully applied to unpaired CT and MR images of the head, these images do not have as much variation of intensity pairs as do images in the pelvic region due to the presence of joints and muscles. In this paper, we extended the CycleGAN approach by adding the gradient consistency loss to improve the accuracy at the boundaries. We conducted two experiments. To evaluate image synthesis, we investigated dependency of image synthesis accuracy on (1) the number of training data and (2) incorporation of the gradient consistency loss. To demonstrate the applicability of our method, we also investigated segmentation accuracy on synthesized images.

Original language	English (US)
Title of host publication	Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings
Editors	Orcun Goksel, Ipek Oguz, Ali Gooya, Ninon Burgos
Publisher	Springer Verlag
Pages	31-41
Number of pages	11
ISBN (Print)	9783030005351
DOIs	https://doi.org/10.1007/978-3-030-00536-8_4
State	Published - 2018
Externally published	Yes
Event	3rd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2018 Held in Conjunction with 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2018 - Granada, Spain Duration: Sep 16 2018 → Sep 16 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11037 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	3rd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2018 Held in Conjunction with 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2018
Country/Territory	Spain
City	Granada
Period	9/16/18 → 9/16/18

Keywords

CT
CycleGAN
Image synthesis
MR
Musculoskeletal image
Segmentation

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-00536-8_4

Cite this

Hiasa, Y., Otake, Y., Takao, M., Matsuoka, T., Takashima, K., Carass, A., Prince, J. L., Sugano, N., & Sato, Y. (2018). Cross-modality image synthesis from unpaired data using cyclegan: Effects of gradient consistency loss and training data size. In O. Goksel, I. Oguz, A. Gooya, & N. Burgos (Eds.), Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings (pp. 31-41). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11037 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-00536-8_4

Cross-modality image synthesis from unpaired data using cyclegan: Effects of gradient consistency loss and training data size. / Hiasa, Yuta; Otake, Yoshito; Takao, Masaki et al.
Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings. ed. / Orcun Goksel; Ipek Oguz; Ali Gooya; Ninon Burgos. Springer Verlag, 2018. p. 31-41 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11037 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Hiasa, Y, Otake, Y, Takao, M, Matsuoka, T, Takashima, K, Carass, A, Prince, JL, Sugano, N & Sato, Y 2018, Cross-modality image synthesis from unpaired data using cyclegan: Effects of gradient consistency loss and training data size. in O Goksel, I Oguz, A Gooya & N Burgos (eds), Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11037 LNCS, Springer Verlag, pp. 31-41, 3rd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2018 Held in Conjunction with 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2018, Granada, Spain, 9/16/18. https://doi.org/10.1007/978-3-030-00536-8_4

Hiasa Y, Otake Y, Takao M, Matsuoka T, Takashima K, Carass A et al. Cross-modality image synthesis from unpaired data using cyclegan: Effects of gradient consistency loss and training data size. In Goksel O, Oguz I, Gooya A, Burgos N, editors, Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings. Springer Verlag. 2018. p. 31-41. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-00536-8_4

Hiasa, Yuta ; Otake, Yoshito ; Takao, Masaki et al. / Cross-modality image synthesis from unpaired data using cyclegan : Effects of gradient consistency loss and training data size. Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings. editor / Orcun Goksel ; Ipek Oguz ; Ali Gooya ; Ninon Burgos. Springer Verlag, 2018. pp. 31-41 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{301c889c2fb74be89d5afc1e3efbd4fb,

title = "Cross-modality image synthesis from unpaired data using cyclegan: Effects of gradient consistency loss and training data size",

abstract = "CT is commonly used in orthopedic procedures. MRI is used along with CT to identify muscle structures and diagnose osteonecrosis due to its superior soft tissue contrast. However, MRI has poor contrast for bone structures. Clearly, it would be helpful if a corresponding CT were available, as bone boundaries are more clearly seen and CT has a standardized (i.e., Hounsfield) unit. Therefore, we aim at MR-to-CT synthesis. While the CycleGAN was successfully applied to unpaired CT and MR images of the head, these images do not have as much variation of intensity pairs as do images in the pelvic region due to the presence of joints and muscles. In this paper, we extended the CycleGAN approach by adding the gradient consistency loss to improve the accuracy at the boundaries. We conducted two experiments. To evaluate image synthesis, we investigated dependency of image synthesis accuracy on (1) the number of training data and (2) incorporation of the gradient consistency loss. To demonstrate the applicability of our method, we also investigated segmentation accuracy on synthesized images.",

keywords = "CT, CycleGAN, Image synthesis, MR, Musculoskeletal image, Segmentation",

author = "Yuta Hiasa and Yoshito Otake and Masaki Takao and Takumi Matsuoka and Kazuma Takashima and Aaron Carass and Prince, {Jerry L.} and Nobuhiko Sugano and Yoshinobu Sato",

note = "Publisher Copyright: {\textcopyright} 2018, Springer Nature Switzerland AG.; 3rd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2018 Held in Conjunction with 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2018 ; Conference date: 16-09-2018 Through 16-09-2018",

year = "2018",

doi = "10.1007/978-3-030-00536-8_4",

language = "English (US)",

isbn = "9783030005351",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "31--41",

editor = "Orcun Goksel and Ipek Oguz and Ali Gooya and Ninon Burgos",

booktitle = "Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings",

}

TY - GEN

T1 - Cross-modality image synthesis from unpaired data using cyclegan

T2 - 3rd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2018 Held in Conjunction with 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2018

AU - Hiasa, Yuta

AU - Otake, Yoshito

AU - Takao, Masaki

AU - Matsuoka, Takumi

AU - Takashima, Kazuma

AU - Carass, Aaron

AU - Prince, Jerry L.

AU - Sugano, Nobuhiko

AU - Sato, Yoshinobu

PY - 2018

Y1 - 2018

N2 - CT is commonly used in orthopedic procedures. MRI is used along with CT to identify muscle structures and diagnose osteonecrosis due to its superior soft tissue contrast. However, MRI has poor contrast for bone structures. Clearly, it would be helpful if a corresponding CT were available, as bone boundaries are more clearly seen and CT has a standardized (i.e., Hounsfield) unit. Therefore, we aim at MR-to-CT synthesis. While the CycleGAN was successfully applied to unpaired CT and MR images of the head, these images do not have as much variation of intensity pairs as do images in the pelvic region due to the presence of joints and muscles. In this paper, we extended the CycleGAN approach by adding the gradient consistency loss to improve the accuracy at the boundaries. We conducted two experiments. To evaluate image synthesis, we investigated dependency of image synthesis accuracy on (1) the number of training data and (2) incorporation of the gradient consistency loss. To demonstrate the applicability of our method, we also investigated segmentation accuracy on synthesized images.

AB - CT is commonly used in orthopedic procedures. MRI is used along with CT to identify muscle structures and diagnose osteonecrosis due to its superior soft tissue contrast. However, MRI has poor contrast for bone structures. Clearly, it would be helpful if a corresponding CT were available, as bone boundaries are more clearly seen and CT has a standardized (i.e., Hounsfield) unit. Therefore, we aim at MR-to-CT synthesis. While the CycleGAN was successfully applied to unpaired CT and MR images of the head, these images do not have as much variation of intensity pairs as do images in the pelvic region due to the presence of joints and muscles. In this paper, we extended the CycleGAN approach by adding the gradient consistency loss to improve the accuracy at the boundaries. We conducted two experiments. To evaluate image synthesis, we investigated dependency of image synthesis accuracy on (1) the number of training data and (2) incorporation of the gradient consistency loss. To demonstrate the applicability of our method, we also investigated segmentation accuracy on synthesized images.

KW - CT

KW - CycleGAN

KW - Image synthesis

KW - MR

KW - Musculoskeletal image

KW - Segmentation

UR - http://www.scopus.com/inward/record.url?scp=85053912661&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053912661&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-00536-8_4

DO - 10.1007/978-3-030-00536-8_4

M3 - Conference contribution

AN - SCOPUS:85053912661

SN - 9783030005351

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 31

EP - 41

BT - Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings

A2 - Goksel, Orcun

A2 - Oguz, Ipek

A2 - Gooya, Ali

A2 - Burgos, Ninon

PB - Springer Verlag

Y2 - 16 September 2018 through 16 September 2018

ER -

Cross-modality image synthesis from unpaired data using cyclegan: Effects of gradient consistency loss and training data size

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this