Technical note: On-The-fly augmented reality for orthopaedic surgery using a multi-modal fiducial

Sebastian Andress, Alex Johson, Mathias Unberath, Alexander Winkler, Kevin Yu, Javad Fotouhi, Simon Weidert, Greg Osgood, Nassir Navab

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

C-Arm X-Ray systems are the workhorse modality for guidance of percutaneous orthopaedic surgical procedures. However, two-dimensional observations of the three-dimensional anatomy suffer from the effects of projective simplification. Consequently, many X-Ray images from various orientations need to be acquired for the surgeon to accurately assess the spatial relations between the patient's anatomy and the surgical tools. In this paper, we present an on-The-fly surgical support system that provides guidance using augmented reality and can be used in quasi-unprepared operating rooms. The proposed system builds upon a multi-modality marker and simultaneous localization and mapping technique to co-calibrate an optical see-Through head mounted display to a C-Arm fluoroscopy system. Then, annotations on the 2-D X-Ray images can be rendered as virtual objects in 3-D providing surgical guidance. In a feasibility study on a semi-Anthropomorphic phantom we found the accuracy of our system to be comparable to the traditional image-guided technique while substantially reducing the number of acquired X-Ray images as well as procedure time. Our promising results encourage further research on the interaction between virtual and real objects, that we believe will directly benefit the proposed method. Further, we would like to explore the capabilities of our on-The-fly augmented reality support system in a larger study directed towards common orthopaedic interventions.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2018
Subtitle of host publicationImage-Guided Procedures, Robotic Interventions, and Modeling
PublisherSPIE
Volume10576
ISBN (Electronic)9781510616417
DOIs
StatePublished - Jan 1 2018
Externally publishedYes
EventMedical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling - Houston, United States
Duration: Feb 12 2018Feb 15 2018

Other

OtherMedical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling
CountryUnited States
CityHouston
Period2/12/182/15/18

Fingerprint

orthopedics
Augmented reality
Orthopedics
surgery
Surgery
X-Rays
X rays
support systems
anatomy
Anatomy
Arm
x rays
Orthopedic Procedures
annotations
fluoroscopy
Operating rooms
surgeons
Fluoroscopy
Feasibility Studies
Operating Rooms

Keywords

  • Fluoroscopy
  • Interventional imaging
  • Registration
  • Surgical guidance

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Biomaterials
  • Atomic and Molecular Physics, and Optics
  • Radiology Nuclear Medicine and imaging

Cite this

Andress, S., Johson, A., Unberath, M., Winkler, A., Yu, K., Fotouhi, J., ... Navab, N. (2018). Technical note: On-The-fly augmented reality for orthopaedic surgery using a multi-modal fiducial. In Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling (Vol. 10576). [105760H] SPIE. https://doi.org/10.1117/12.2319048

Technical note : On-The-fly augmented reality for orthopaedic surgery using a multi-modal fiducial. / Andress, Sebastian; Johson, Alex; Unberath, Mathias; Winkler, Alexander; Yu, Kevin; Fotouhi, Javad; Weidert, Simon; Osgood, Greg; Navab, Nassir.

Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling. Vol. 10576 SPIE, 2018. 105760H.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Andress, S, Johson, A, Unberath, M, Winkler, A, Yu, K, Fotouhi, J, Weidert, S, Osgood, G & Navab, N 2018, Technical note: On-The-fly augmented reality for orthopaedic surgery using a multi-modal fiducial. in Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling. vol. 10576, 105760H, SPIE, Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling, Houston, United States, 2/12/18. https://doi.org/10.1117/12.2319048
Andress S, Johson A, Unberath M, Winkler A, Yu K, Fotouhi J et al. Technical note: On-The-fly augmented reality for orthopaedic surgery using a multi-modal fiducial. In Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling. Vol. 10576. SPIE. 2018. 105760H https://doi.org/10.1117/12.2319048
Andress, Sebastian ; Johson, Alex ; Unberath, Mathias ; Winkler, Alexander ; Yu, Kevin ; Fotouhi, Javad ; Weidert, Simon ; Osgood, Greg ; Navab, Nassir. / Technical note : On-The-fly augmented reality for orthopaedic surgery using a multi-modal fiducial. Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling. Vol. 10576 SPIE, 2018.
@inproceedings{7a659fd0c90d45c2b2fc34260c47c6d9,
title = "Technical note: On-The-fly augmented reality for orthopaedic surgery using a multi-modal fiducial",
abstract = "C-Arm X-Ray systems are the workhorse modality for guidance of percutaneous orthopaedic surgical procedures. However, two-dimensional observations of the three-dimensional anatomy suffer from the effects of projective simplification. Consequently, many X-Ray images from various orientations need to be acquired for the surgeon to accurately assess the spatial relations between the patient's anatomy and the surgical tools. In this paper, we present an on-The-fly surgical support system that provides guidance using augmented reality and can be used in quasi-unprepared operating rooms. The proposed system builds upon a multi-modality marker and simultaneous localization and mapping technique to co-calibrate an optical see-Through head mounted display to a C-Arm fluoroscopy system. Then, annotations on the 2-D X-Ray images can be rendered as virtual objects in 3-D providing surgical guidance. In a feasibility study on a semi-Anthropomorphic phantom we found the accuracy of our system to be comparable to the traditional image-guided technique while substantially reducing the number of acquired X-Ray images as well as procedure time. Our promising results encourage further research on the interaction between virtual and real objects, that we believe will directly benefit the proposed method. Further, we would like to explore the capabilities of our on-The-fly augmented reality support system in a larger study directed towards common orthopaedic interventions.",
keywords = "Fluoroscopy, Interventional imaging, Registration, Surgical guidance",
author = "Sebastian Andress and Alex Johson and Mathias Unberath and Alexander Winkler and Kevin Yu and Javad Fotouhi and Simon Weidert and Greg Osgood and Nassir Navab",
year = "2018",
month = "1",
day = "1",
doi = "10.1117/12.2319048",
language = "English (US)",
volume = "10576",
booktitle = "Medical Imaging 2018",
publisher = "SPIE",

}

TY - GEN

T1 - Technical note

T2 - On-The-fly augmented reality for orthopaedic surgery using a multi-modal fiducial

AU - Andress, Sebastian

AU - Johson, Alex

AU - Unberath, Mathias

AU - Winkler, Alexander

AU - Yu, Kevin

AU - Fotouhi, Javad

AU - Weidert, Simon

AU - Osgood, Greg

AU - Navab, Nassir

PY - 2018/1/1

Y1 - 2018/1/1

N2 - C-Arm X-Ray systems are the workhorse modality for guidance of percutaneous orthopaedic surgical procedures. However, two-dimensional observations of the three-dimensional anatomy suffer from the effects of projective simplification. Consequently, many X-Ray images from various orientations need to be acquired for the surgeon to accurately assess the spatial relations between the patient's anatomy and the surgical tools. In this paper, we present an on-The-fly surgical support system that provides guidance using augmented reality and can be used in quasi-unprepared operating rooms. The proposed system builds upon a multi-modality marker and simultaneous localization and mapping technique to co-calibrate an optical see-Through head mounted display to a C-Arm fluoroscopy system. Then, annotations on the 2-D X-Ray images can be rendered as virtual objects in 3-D providing surgical guidance. In a feasibility study on a semi-Anthropomorphic phantom we found the accuracy of our system to be comparable to the traditional image-guided technique while substantially reducing the number of acquired X-Ray images as well as procedure time. Our promising results encourage further research on the interaction between virtual and real objects, that we believe will directly benefit the proposed method. Further, we would like to explore the capabilities of our on-The-fly augmented reality support system in a larger study directed towards common orthopaedic interventions.

AB - C-Arm X-Ray systems are the workhorse modality for guidance of percutaneous orthopaedic surgical procedures. However, two-dimensional observations of the three-dimensional anatomy suffer from the effects of projective simplification. Consequently, many X-Ray images from various orientations need to be acquired for the surgeon to accurately assess the spatial relations between the patient's anatomy and the surgical tools. In this paper, we present an on-The-fly surgical support system that provides guidance using augmented reality and can be used in quasi-unprepared operating rooms. The proposed system builds upon a multi-modality marker and simultaneous localization and mapping technique to co-calibrate an optical see-Through head mounted display to a C-Arm fluoroscopy system. Then, annotations on the 2-D X-Ray images can be rendered as virtual objects in 3-D providing surgical guidance. In a feasibility study on a semi-Anthropomorphic phantom we found the accuracy of our system to be comparable to the traditional image-guided technique while substantially reducing the number of acquired X-Ray images as well as procedure time. Our promising results encourage further research on the interaction between virtual and real objects, that we believe will directly benefit the proposed method. Further, we would like to explore the capabilities of our on-The-fly augmented reality support system in a larger study directed towards common orthopaedic interventions.

KW - Fluoroscopy

KW - Interventional imaging

KW - Registration

KW - Surgical guidance

UR - http://www.scopus.com/inward/record.url?scp=85050657366&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85050657366&partnerID=8YFLogxK

U2 - 10.1117/12.2319048

DO - 10.1117/12.2319048

M3 - Conference contribution

AN - SCOPUS:85050657366

VL - 10576

BT - Medical Imaging 2018

PB - SPIE

ER -