Towards clinical translation of augmented orthopedic surgery

From pre-op CT to intra-op x-ray via RGBD sensing

Emerson Tucker, Javad Fotouhi, Mathias Unberath, Sing Chun Lee, Bernhard Fuerst, Alex Johnson, Mehran Armand, Greg Osgood, Nassir Navab

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Pre-operative CT data is available for several orthopedic and trauma interventions, and is mainly used to identify injuries and plan the surgical procedure. In this work we propose an intuitive augmented reality environment allowing visualization of pre-operative data during the intervention, with an overlay of the optical information from the surgical site. The pre-operative CT volume is first registered to the patient by acquiring a single C-Arm X-ray image and using 3D/2D intensity-based registration. Next, we use an RGBD sensor on the C-Arm to fuse the optical information of the surgical site with patient pre-operative medical data and provide an augmented reality environment. The 3D/2D registration of the pre-and intra-operative data allows us to maintain a correct visualization each time the C-Arm is repositioned or the patient moves. An overall mean target registration error (mTRE) and standard deviation of 5.24 ± 3.09 mm was measured averaged over 19 C-Arm poses. The proposed solution enables the surgeon to visualize pre-operative data overlaid with information from the surgical site (e.g. surgeon's hands, surgical tools, etc.) for any C-Arm pose, and negates issues of line-of-sight and long setup times, which are present in commercially available systems.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2018
Subtitle of host publicationImaging Informatics for Healthcare, Research, and Applications
PublisherSPIE
Volume10579
ISBN (Electronic)9781510616479
DOIs
StatePublished - Jan 1 2018
EventMedical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications - Houston, United States
Duration: Feb 13 2018Feb 15 2018

Other

OtherMedical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications
CountryUnited States
CityHouston
Period2/13/182/15/18

Fingerprint

orthopedics
Augmented reality
Orthopedics
surgery
Surgery
surgeons
Arm
Visualization
X-Rays
X rays
Electric fuses
x rays
fuses
line of sight
standard deviation
Sensors
Cone-Beam Computed Tomography
Intraoperative Complications
sensors
Hand

Keywords

  • 3D/2D Registration
  • Augmented reality
  • Pre-operative CT
  • RGBD camera

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Biomaterials
  • Atomic and Molecular Physics, and Optics
  • Radiology Nuclear Medicine and imaging

Cite this

Tucker, E., Fotouhi, J., Unberath, M., Lee, S. C., Fuerst, B., Johnson, A., ... Navab, N. (2018). Towards clinical translation of augmented orthopedic surgery: From pre-op CT to intra-op x-ray via RGBD sensing. In Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications (Vol. 10579). [105790J] SPIE. https://doi.org/10.1117/12.2293675

Towards clinical translation of augmented orthopedic surgery : From pre-op CT to intra-op x-ray via RGBD sensing. / Tucker, Emerson; Fotouhi, Javad; Unberath, Mathias; Lee, Sing Chun; Fuerst, Bernhard; Johnson, Alex; Armand, Mehran; Osgood, Greg; Navab, Nassir.

Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. Vol. 10579 SPIE, 2018. 105790J.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Tucker, E, Fotouhi, J, Unberath, M, Lee, SC, Fuerst, B, Johnson, A, Armand, M, Osgood, G & Navab, N 2018, Towards clinical translation of augmented orthopedic surgery: From pre-op CT to intra-op x-ray via RGBD sensing. in Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. vol. 10579, 105790J, SPIE, Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications, Houston, United States, 2/13/18. https://doi.org/10.1117/12.2293675
Tucker E, Fotouhi J, Unberath M, Lee SC, Fuerst B, Johnson A et al. Towards clinical translation of augmented orthopedic surgery: From pre-op CT to intra-op x-ray via RGBD sensing. In Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. Vol. 10579. SPIE. 2018. 105790J https://doi.org/10.1117/12.2293675
Tucker, Emerson ; Fotouhi, Javad ; Unberath, Mathias ; Lee, Sing Chun ; Fuerst, Bernhard ; Johnson, Alex ; Armand, Mehran ; Osgood, Greg ; Navab, Nassir. / Towards clinical translation of augmented orthopedic surgery : From pre-op CT to intra-op x-ray via RGBD sensing. Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. Vol. 10579 SPIE, 2018.
@inproceedings{fe3f5f16da6b4d2281f18d662a26781a,
title = "Towards clinical translation of augmented orthopedic surgery: From pre-op CT to intra-op x-ray via RGBD sensing",
abstract = "Pre-operative CT data is available for several orthopedic and trauma interventions, and is mainly used to identify injuries and plan the surgical procedure. In this work we propose an intuitive augmented reality environment allowing visualization of pre-operative data during the intervention, with an overlay of the optical information from the surgical site. The pre-operative CT volume is first registered to the patient by acquiring a single C-Arm X-ray image and using 3D/2D intensity-based registration. Next, we use an RGBD sensor on the C-Arm to fuse the optical information of the surgical site with patient pre-operative medical data and provide an augmented reality environment. The 3D/2D registration of the pre-and intra-operative data allows us to maintain a correct visualization each time the C-Arm is repositioned or the patient moves. An overall mean target registration error (mTRE) and standard deviation of 5.24 ± 3.09 mm was measured averaged over 19 C-Arm poses. The proposed solution enables the surgeon to visualize pre-operative data overlaid with information from the surgical site (e.g. surgeon's hands, surgical tools, etc.) for any C-Arm pose, and negates issues of line-of-sight and long setup times, which are present in commercially available systems.",
keywords = "3D/2D Registration, Augmented reality, Pre-operative CT, RGBD camera",
author = "Emerson Tucker and Javad Fotouhi and Mathias Unberath and Lee, {Sing Chun} and Bernhard Fuerst and Alex Johnson and Mehran Armand and Greg Osgood and Nassir Navab",
year = "2018",
month = "1",
day = "1",
doi = "10.1117/12.2293675",
language = "English (US)",
volume = "10579",
booktitle = "Medical Imaging 2018",
publisher = "SPIE",

}

TY - GEN

T1 - Towards clinical translation of augmented orthopedic surgery

T2 - From pre-op CT to intra-op x-ray via RGBD sensing

AU - Tucker, Emerson

AU - Fotouhi, Javad

AU - Unberath, Mathias

AU - Lee, Sing Chun

AU - Fuerst, Bernhard

AU - Johnson, Alex

AU - Armand, Mehran

AU - Osgood, Greg

AU - Navab, Nassir

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Pre-operative CT data is available for several orthopedic and trauma interventions, and is mainly used to identify injuries and plan the surgical procedure. In this work we propose an intuitive augmented reality environment allowing visualization of pre-operative data during the intervention, with an overlay of the optical information from the surgical site. The pre-operative CT volume is first registered to the patient by acquiring a single C-Arm X-ray image and using 3D/2D intensity-based registration. Next, we use an RGBD sensor on the C-Arm to fuse the optical information of the surgical site with patient pre-operative medical data and provide an augmented reality environment. The 3D/2D registration of the pre-and intra-operative data allows us to maintain a correct visualization each time the C-Arm is repositioned or the patient moves. An overall mean target registration error (mTRE) and standard deviation of 5.24 ± 3.09 mm was measured averaged over 19 C-Arm poses. The proposed solution enables the surgeon to visualize pre-operative data overlaid with information from the surgical site (e.g. surgeon's hands, surgical tools, etc.) for any C-Arm pose, and negates issues of line-of-sight and long setup times, which are present in commercially available systems.

AB - Pre-operative CT data is available for several orthopedic and trauma interventions, and is mainly used to identify injuries and plan the surgical procedure. In this work we propose an intuitive augmented reality environment allowing visualization of pre-operative data during the intervention, with an overlay of the optical information from the surgical site. The pre-operative CT volume is first registered to the patient by acquiring a single C-Arm X-ray image and using 3D/2D intensity-based registration. Next, we use an RGBD sensor on the C-Arm to fuse the optical information of the surgical site with patient pre-operative medical data and provide an augmented reality environment. The 3D/2D registration of the pre-and intra-operative data allows us to maintain a correct visualization each time the C-Arm is repositioned or the patient moves. An overall mean target registration error (mTRE) and standard deviation of 5.24 ± 3.09 mm was measured averaged over 19 C-Arm poses. The proposed solution enables the surgeon to visualize pre-operative data overlaid with information from the surgical site (e.g. surgeon's hands, surgical tools, etc.) for any C-Arm pose, and negates issues of line-of-sight and long setup times, which are present in commercially available systems.

KW - 3D/2D Registration

KW - Augmented reality

KW - Pre-operative CT

KW - RGBD camera

UR - http://www.scopus.com/inward/record.url?scp=85047739133&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047739133&partnerID=8YFLogxK

U2 - 10.1117/12.2293675

DO - 10.1117/12.2293675

M3 - Conference contribution

VL - 10579

BT - Medical Imaging 2018

PB - SPIE

ER -