Deepdrr { a catalyst for machine learning in fluoroscopy-guided procedures

Mathias Unberath, Jan Nico Zaech, Sing Chun Lee, Bastian Bier, Javad Fotouhi, Mehran Armand, Nassir Navab

Research output: Contribution to journalArticlepeer-review


Machine learning-based approaches outperform competing methods in most disciplines relevant to diagnostic radiology. Interventional radiology, however, has not yet benefited substantially from the advent of deep learning, in particular because of two reasons: 1) Most images acquired during the procedure are never archived and are thus not available for learning, and 2) even if they were available, annotations would be a severe challenge due to the vast amounts of data. When considering uoroscopy-guided procedures, an interesting alternative to true interventional uoroscopy is in silico simulation of the procedure from 3D diagnostic CT. In this case, labeling is comparably easy and potentially readily available, yet, the appropriateness of resulting synthetic data is dependent on the forward model. In this work, we propose DeepDRR, a framework for fast and realistic simulation of uoroscopy and digital radiography from CT scans, tightly integrated with the software platforms native to deep learning. We use machine learning for material decomposition and scatter estimation in 3D and 2D, respectively, combined with analytic forward projection and noise injection to achieve the required performance. On the example of anatomical landmark detection in X-ray images of the pelvis, we demonstrate that machine learning models trained on DeepDRRs generalize to unseen clinically acquired data without the need for re-training or domain adaptation. Our results are promising and promote the establishment of machine learning in uoroscopy-guided procedures.

Original languageEnglish (US)
JournalUnknown Journal
StatePublished - Mar 22 2018


  • Beam hardening
  • Image-guided procedures
  • Monte carlo simulation
  • Volumetric segmentation

ASJC Scopus subject areas

  • General

Fingerprint Dive into the research topics of 'Deepdrr { a catalyst for machine learning in fluoroscopy-guided procedures'. Together they form a unique fingerprint.

Cite this