Dynamic augmented reality for sensory substitution in robot-assisted surgical systems

Takintope Akinbiyi, Carol E. Reiley, Sunipa Saha, Darius Burschka, Christopher J. Hasser, David D. Yuh, Allison M. Okamura

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Teleoperated robot-assisted surgical systems provide surgeons with improved precision, dexterity, and visualization over traditional minimally invasive surgery. The addition of haptic (force and/or tactile) feedback has been proposed as a way to further enhance the performance of these systems. However, due to limitations in sensing and control technologies, implementing direct haptic feedback to the surgeon's hands remains impractical for clinical application. A new, intuitive augmented reality system for presentation of force information through sensory substitution has been developed and evaluated. The augmented reality system consists of force-sensing robotic instruments, a kinematic tool tracker, and a graphic display that overlays a visual representation of force levels on top of the moving instrument tips. The system is integrated with the da Vinci Surgical System (Intuitive Surgical, Inc.) and tested by several users in a phantom knot tying task. The augmented reality system decreases the number of broken sutures, decreases the number of loose knots, and results in more consistent application of forces.

Original languageEnglish (US)
Title of host publicationAnnual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings
Pages567-570
Number of pages4
DOIs
StatePublished - 2006
Event28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS'06 - New York, NY, United States
Duration: Aug 30 2006Sep 3 2006

Other

Other28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS'06
CountryUnited States
CityNew York, NY
Period8/30/069/3/06

Fingerprint

Augmented reality
Substitution reactions
Robots
Feedback
Surgery
Kinematics
Robotics
Visualization
Display devices

ASJC Scopus subject areas

  • Bioengineering

Cite this

Akinbiyi, T., Reiley, C. E., Saha, S., Burschka, D., Hasser, C. J., Yuh, D. D., & Okamura, A. M. (2006). Dynamic augmented reality for sensory substitution in robot-assisted surgical systems. In Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings (pp. 567-570). [4029719] https://doi.org/10.1109/IEMBS.2006.259707

Dynamic augmented reality for sensory substitution in robot-assisted surgical systems. / Akinbiyi, Takintope; Reiley, Carol E.; Saha, Sunipa; Burschka, Darius; Hasser, Christopher J.; Yuh, David D.; Okamura, Allison M.

Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings. 2006. p. 567-570 4029719.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Akinbiyi, T, Reiley, CE, Saha, S, Burschka, D, Hasser, CJ, Yuh, DD & Okamura, AM 2006, Dynamic augmented reality for sensory substitution in robot-assisted surgical systems. in Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings., 4029719, pp. 567-570, 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS'06, New York, NY, United States, 8/30/06. https://doi.org/10.1109/IEMBS.2006.259707
Akinbiyi T, Reiley CE, Saha S, Burschka D, Hasser CJ, Yuh DD et al. Dynamic augmented reality for sensory substitution in robot-assisted surgical systems. In Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings. 2006. p. 567-570. 4029719 https://doi.org/10.1109/IEMBS.2006.259707
Akinbiyi, Takintope ; Reiley, Carol E. ; Saha, Sunipa ; Burschka, Darius ; Hasser, Christopher J. ; Yuh, David D. ; Okamura, Allison M. / Dynamic augmented reality for sensory substitution in robot-assisted surgical systems. Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings. 2006. pp. 567-570
@inproceedings{aeb8463d681546099bf3db613ef961b9,
title = "Dynamic augmented reality for sensory substitution in robot-assisted surgical systems",
abstract = "Teleoperated robot-assisted surgical systems provide surgeons with improved precision, dexterity, and visualization over traditional minimally invasive surgery. The addition of haptic (force and/or tactile) feedback has been proposed as a way to further enhance the performance of these systems. However, due to limitations in sensing and control technologies, implementing direct haptic feedback to the surgeon's hands remains impractical for clinical application. A new, intuitive augmented reality system for presentation of force information through sensory substitution has been developed and evaluated. The augmented reality system consists of force-sensing robotic instruments, a kinematic tool tracker, and a graphic display that overlays a visual representation of force levels on top of the moving instrument tips. The system is integrated with the da Vinci Surgical System (Intuitive Surgical, Inc.) and tested by several users in a phantom knot tying task. The augmented reality system decreases the number of broken sutures, decreases the number of loose knots, and results in more consistent application of forces.",
author = "Takintope Akinbiyi and Reiley, {Carol E.} and Sunipa Saha and Darius Burschka and Hasser, {Christopher J.} and Yuh, {David D.} and Okamura, {Allison M.}",
year = "2006",
doi = "10.1109/IEMBS.2006.259707",
language = "English (US)",
isbn = "1424400325",
pages = "567--570",
booktitle = "Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings",

}

TY - GEN

T1 - Dynamic augmented reality for sensory substitution in robot-assisted surgical systems

AU - Akinbiyi, Takintope

AU - Reiley, Carol E.

AU - Saha, Sunipa

AU - Burschka, Darius

AU - Hasser, Christopher J.

AU - Yuh, David D.

AU - Okamura, Allison M.

PY - 2006

Y1 - 2006

N2 - Teleoperated robot-assisted surgical systems provide surgeons with improved precision, dexterity, and visualization over traditional minimally invasive surgery. The addition of haptic (force and/or tactile) feedback has been proposed as a way to further enhance the performance of these systems. However, due to limitations in sensing and control technologies, implementing direct haptic feedback to the surgeon's hands remains impractical for clinical application. A new, intuitive augmented reality system for presentation of force information through sensory substitution has been developed and evaluated. The augmented reality system consists of force-sensing robotic instruments, a kinematic tool tracker, and a graphic display that overlays a visual representation of force levels on top of the moving instrument tips. The system is integrated with the da Vinci Surgical System (Intuitive Surgical, Inc.) and tested by several users in a phantom knot tying task. The augmented reality system decreases the number of broken sutures, decreases the number of loose knots, and results in more consistent application of forces.

AB - Teleoperated robot-assisted surgical systems provide surgeons with improved precision, dexterity, and visualization over traditional minimally invasive surgery. The addition of haptic (force and/or tactile) feedback has been proposed as a way to further enhance the performance of these systems. However, due to limitations in sensing and control technologies, implementing direct haptic feedback to the surgeon's hands remains impractical for clinical application. A new, intuitive augmented reality system for presentation of force information through sensory substitution has been developed and evaluated. The augmented reality system consists of force-sensing robotic instruments, a kinematic tool tracker, and a graphic display that overlays a visual representation of force levels on top of the moving instrument tips. The system is integrated with the da Vinci Surgical System (Intuitive Surgical, Inc.) and tested by several users in a phantom knot tying task. The augmented reality system decreases the number of broken sutures, decreases the number of loose knots, and results in more consistent application of forces.

UR - http://www.scopus.com/inward/record.url?scp=34047138988&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34047138988&partnerID=8YFLogxK

U2 - 10.1109/IEMBS.2006.259707

DO - 10.1109/IEMBS.2006.259707

M3 - Conference contribution

C2 - 17945986

AN - SCOPUS:34047138988

SN - 1424400325

SN - 9781424400324

SP - 567

EP - 570

BT - Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings

ER -