DeepDRR – A Catalyst for Machine Learning in Fluoroscopy-Guided Procedures

Mathias Unberath, Jan Nico Zaech, Sing Chun Lee, Bastian Bier, Javad Fotouhi, Mehran Armand, Nassir Navab

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Machine learning-based approaches outperform competing methods in most disciplines relevant to diagnostic radiology. Interventional radiology, however, has not yet benefited substantially from the advent of deep learning, in particular because of two reasons: (1) Most images acquired during the procedure are never archived and are thus not available for learning, and (2) even if they were available, annotations would be a severe challenge due to the vast amounts of data. When considering fluoroscopy-guided procedures, an interesting alternative to true interventional fluoroscopy is in silico simulation of the procedure from 3D diagnostic CT. In this case, labeling is comparably easy and potentially readily available, yet, the appropriateness of resulting synthetic data is dependent on the forward model. In this work, we propose DeepDRR, a framework for fast and realistic simulation of fluoroscopy and digital radiography from CT scans, tightly integrated with the software platforms native to deep learning. We use machine learning for material decomposition and scatter estimation in 3D and 2D, respectively, combined with analytic forward projection and noise injection to achieve the required performance. On the example of anatomical landmark detection in X-ray images of the pelvis, we demonstrate that machine learning models trained on DeepDRRs generalize to unseen clinically acquired data without the need for re-training or domain adaptation. Our results are promising and promote the establishment of machine learning in fluoroscopy-guided procedures.

Original languageEnglish (US)
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings
EditorsAlejandro F. Frangi, Gabor Fichtinger, Julia A. Schnabel, Carlos Alberola-López, Christos Davatzikos
PublisherSpringer Verlag
Pages98-106
Number of pages9
ISBN (Print)9783030009366
DOIs
StatePublished - 2018
Event21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018 - Granada, Spain
Duration: Sep 16 2018Sep 20 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11073 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018
Country/TerritorySpain
CityGranada
Period9/16/189/20/18

Keywords

  • Beam hardening
  • Image-guided procedures
  • Monte carlo simulation
  • Volumetric segmentation

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'DeepDRR – A Catalyst for Machine Learning in Fluoroscopy-Guided Procedures'. Together they form a unique fingerprint.

Cite this