Unsupervised MR-to-CT Synthesis Using Structure-Constrained CycleGAN

Heran Yang, Jian Sun, Aaron Carass, Can Zhao, Junghoon Lee, Jerry L. Prince, Zongben Xu

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


Synthesizing a CT image from an available MR image has recently emerged as a key goal in radiotherapy treatment planning for cancer patients. CycleGANs have achieved promising results on unsupervised MR-to-CT image synthesis; however, because they have no direct constraints between input and synthetic images, cycleGANs do not guarantee structural consistency between these two images. This means that anatomical geometry can be shifted in the synthetic CT images, clearly a highly undesirable outcome in the given application. In this paper, we propose a structure-constrained cycleGAN for unsupervised MR-to-CT synthesis by defining an extra structure-consistency loss based on the modality independent neighborhood descriptor. We also utilize a spectral normalization technique to stabilize the training process and a self-attention module to model the long-range spatial dependencies in the synthetic images. Results on unpaired brain and abdomen MR-to-CT image synthesis show that our method produces better synthetic CT images in both accuracy and visual quality as compared to other unsupervised synthesis methods. We also show that an approximate affine pre-registration for unpaired training data can improve synthesis results.

Original languageEnglish (US)
Article number9164889
Pages (from-to)4249-4261
Number of pages13
JournalIEEE transactions on medical imaging
Issue number12
StatePublished - Dec 2020


  • CycleGAN
  • MIND
  • MR-to-CT synthesis
  • deep learning

ASJC Scopus subject areas

  • Software
  • Radiological and Ultrasound Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Unsupervised MR-to-CT Synthesis Using Structure-Constrained CycleGAN'. Together they form a unique fingerprint.

Cite this