Mixed Reality Support for Orthopaedic Surgery

Sing Chun Lee, Keisuke Tateno, Bernhard Fuerst, Federico Tombari, Javad Fotouhi, Greg Osgood, Alex Johnson, Nassir Navab

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

This work presents a mixed reality environment for orthopaedic interventions that provides a 3D overlay of Cone-beam CT images, surgical site, and real-time tool tracking. The system uses an RGBD camera attached to the detector plane of a mobile C-arm, which is a typical device to acquire X-Ray images during surgery. Calibration of the two devices is done by acquiring simultaneous CBCT and RGBD scans of a calibration phantom and computing the rigid transformation between them. The markerless tracking of the surgical tool is computed in the RGBD view using real-time segmentation and Simultaneous Localization And Mapping. The RGBD view is then overlaid to the CBCT data with real-time point clouds of the surgical site. This visualization provides multiple desired views of the medical data, surgical site, and the tracking of surgical tools, which could be used to provide intuitive visualization for orthopedic procedures to place instrumentation and to assist surgeons with their localization and coordination. Our proposed opto-X-ray system can lead to x-ray radiation dose reduction as well as improved safety in minimally invasive orthopaedic procedures.

Original languageEnglish (US)
Title of host publicationAdjunct Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2017
EditorsWolfgang Broll, Holger Regenbrecht, Gerd Bruder, Myriam Servieres, Maki Sugimoto, J Edward Swan
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages204-205
Number of pages2
ISBN (Electronic)9780769563275
DOIs
StatePublished - Oct 27 2017
Event16th Adjunct IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2017 - Nantes, France
Duration: Oct 9 2017Oct 13 2017

Publication series

NameAdjunct Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2017

Other

Other16th Adjunct IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2017
Country/TerritoryFrance
CityNantes
Period10/9/1710/13/17

Keywords

  • Cone-Beam CT
  • Intra-operative
  • Mixed Reality
  • Multi-modal Visualization
  • Range Imaging
  • Tracking

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Media Technology
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Mixed Reality Support for Orthopaedic Surgery'. Together they form a unique fingerprint.

Cite this