Deep harmonization of inconsistent mr data for consistent volume segmentation

Blake E. Dewey, Can Zhao, Aaron Carass, Jiwon Oh, Peter Calabresi, Peter C Van Zijl, Jerry Ladd Prince

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Magnetic resonance image analysis is often hampered by inconsistent data due to upgrades or changes to the scanner platform or modification of scanning protocols. These changes can manifest in three main sources of image inconsistency: contrast, resolution, and noise. Modern analysis techniques that use supervised machine learning can be especially susceptible to these inconsistencies, as existing training data may not be valid after an upgrade or protocol change. In previous work, differences in contrast and resolution have been addressed in isolation. We propose a novel method of image intensity harmonization that addresses each of the three sources of inconsistency. We formulate our method around a multi-planar, multi-contrast U-net, where all of the available contrasts are used as input channels in a single modified U-Net to produce all of the output contrasts simultaneously. The multi-contrast nature of the deep network allows for harmonization of contrast as information can be shared between contrasts. In addition, coherent, biological features are highlighted and matched to the target, while noise, which differs between matched inputs and outputs, is not reinforced. This process also normalizes small differences in resolution due to the influence of the high resolution channels. To combat larger differences in resolution, which would not be recovered by the neural network alone, we use self super-resolution (SSR) on all images with thick (>2 mm) slices before harmonization. To generate consistent images, the target images are also processed in a similar manner so that all resulting images have consistent qualities. Our harmonization process eliminates significant volume bias of multiple brain compartments and lesion estimation. In addition, absolute volume difference and Dice similarity of segmentation volumes were significantly improved (p < 0.005). SSR alone did not affect the statistical significance of the difference, even though the absolute volume difference was reduced.

Original languageEnglish (US)
Title of host publicationSimulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings
EditorsOrcun Goksel, Ipek Oguz, Ali Gooya, Ninon Burgos
PublisherSpringer Verlag
Pages20-30
Number of pages11
ISBN (Print)9783030005351
DOIs
Publication statusPublished - Jan 1 2018
Event3rd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2018 Held in Conjunction with 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2018 - Granada, Spain
Duration: Sep 16 2018Sep 16 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11037 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other3rd International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2018 Held in Conjunction with 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2018
CountrySpain
CityGranada
Period9/16/189/16/18

    Fingerprint

Keywords

  • Deep learning
  • MR harmonization
  • Multiple sclerosis
  • Super-resolution

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Dewey, B. E., Zhao, C., Carass, A., Oh, J., Calabresi, P., Van Zijl, P. C., & Prince, J. L. (2018). Deep harmonization of inconsistent mr data for consistent volume segmentation. In O. Goksel, I. Oguz, A. Gooya, & N. Burgos (Eds.), Simulation and Synthesis in Medical Imaging - Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Proceedings (pp. 20-30). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11037 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-00536-8_3