Optical merger of direct vision with virtual images for scaled teleoperation

Samuel T. Clanton, David C. Wang, Vikram S Chib, Yoky Matsuoka, George D. Stetten

Research output: Contribution to journalArticle

Abstract

Scaled teleoperation is increasingly prevalent in medicine, as well as in other applications of robotics. Visual feedback in such systems is essential and should make maximal use of natural hand-eye coordination. This paper describes a new method of visual feedback for scaled teleoperation in which the operator manipulates the handle of a remote tool in the presence of a registered virtual image of the target in real time. The method adapts a concept already used successfully in a new medical device called the Sonic Flashlight, which permits direct in situ visualization of ultrasound during invasive procedures. The Sonic Flashlight uses a flat-panel monitor and a halt silvered mirror to merge the visual outer surface of a patient with a simultaneous ultrasound scan of the patient's interior. Adapting the concept to scaled teleoperation involves removing the imaging device and the target to a remote location and adding a master-slave control device. This permits the operator to see his hands, along with what appears to De the tool, and the target, merged in a workspace that preserves natural hand-eye coordination. Three functioning prototypes are described, one based on ultrasound and two on light microscopy. The limitations and potential of the new approach are discussed.

Original languageEnglish (US)
Pages (from-to)277-284
Number of pages8
JournalIEEE Transactions on Visualization and Computer Graphics
Volume12
Issue number2
DOIs
StatePublished - Mar 2006
Externally publishedYes

Fingerprint

Flashlights
Remote control
Ultrasonics
Feedback
Medicine
Optical microscopy
Mirrors
Robotics
Visualization
Imaging techniques

Keywords

  • Artificial
  • Augmented
  • Image display
  • Medical information systems
  • Real time
  • Virtual realities

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Software

Cite this

Optical merger of direct vision with virtual images for scaled teleoperation. / Clanton, Samuel T.; Wang, David C.; Chib, Vikram S; Matsuoka, Yoky; Stetten, George D.

In: IEEE Transactions on Visualization and Computer Graphics, Vol. 12, No. 2, 03.2006, p. 277-284.

Research output: Contribution to journalArticle

Clanton, Samuel T. ; Wang, David C. ; Chib, Vikram S ; Matsuoka, Yoky ; Stetten, George D. / Optical merger of direct vision with virtual images for scaled teleoperation. In: IEEE Transactions on Visualization and Computer Graphics. 2006 ; Vol. 12, No. 2. pp. 277-284.
@article{ba272fda8b814ba3bf8afb0e77a9003b,
title = "Optical merger of direct vision with virtual images for scaled teleoperation",
abstract = "Scaled teleoperation is increasingly prevalent in medicine, as well as in other applications of robotics. Visual feedback in such systems is essential and should make maximal use of natural hand-eye coordination. This paper describes a new method of visual feedback for scaled teleoperation in which the operator manipulates the handle of a remote tool in the presence of a registered virtual image of the target in real time. The method adapts a concept already used successfully in a new medical device called the Sonic Flashlight, which permits direct in situ visualization of ultrasound during invasive procedures. The Sonic Flashlight uses a flat-panel monitor and a halt silvered mirror to merge the visual outer surface of a patient with a simultaneous ultrasound scan of the patient's interior. Adapting the concept to scaled teleoperation involves removing the imaging device and the target to a remote location and adding a master-slave control device. This permits the operator to see his hands, along with what appears to De the tool, and the target, merged in a workspace that preserves natural hand-eye coordination. Three functioning prototypes are described, one based on ultrasound and two on light microscopy. The limitations and potential of the new approach are discussed.",
keywords = "Artificial, Augmented, Image display, Medical information systems, Real time, Virtual realities",
author = "Clanton, {Samuel T.} and Wang, {David C.} and Chib, {Vikram S} and Yoky Matsuoka and Stetten, {George D.}",
year = "2006",
month = "3",
doi = "10.1109/TVCG.2006.35",
language = "English (US)",
volume = "12",
pages = "277--284",
journal = "IEEE Transactions on Visualization and Computer Graphics",
issn = "1077-2626",
publisher = "IEEE Computer Society",
number = "2",

}

TY - JOUR

T1 - Optical merger of direct vision with virtual images for scaled teleoperation

AU - Clanton, Samuel T.

AU - Wang, David C.

AU - Chib, Vikram S

AU - Matsuoka, Yoky

AU - Stetten, George D.

PY - 2006/3

Y1 - 2006/3

N2 - Scaled teleoperation is increasingly prevalent in medicine, as well as in other applications of robotics. Visual feedback in such systems is essential and should make maximal use of natural hand-eye coordination. This paper describes a new method of visual feedback for scaled teleoperation in which the operator manipulates the handle of a remote tool in the presence of a registered virtual image of the target in real time. The method adapts a concept already used successfully in a new medical device called the Sonic Flashlight, which permits direct in situ visualization of ultrasound during invasive procedures. The Sonic Flashlight uses a flat-panel monitor and a halt silvered mirror to merge the visual outer surface of a patient with a simultaneous ultrasound scan of the patient's interior. Adapting the concept to scaled teleoperation involves removing the imaging device and the target to a remote location and adding a master-slave control device. This permits the operator to see his hands, along with what appears to De the tool, and the target, merged in a workspace that preserves natural hand-eye coordination. Three functioning prototypes are described, one based on ultrasound and two on light microscopy. The limitations and potential of the new approach are discussed.

AB - Scaled teleoperation is increasingly prevalent in medicine, as well as in other applications of robotics. Visual feedback in such systems is essential and should make maximal use of natural hand-eye coordination. This paper describes a new method of visual feedback for scaled teleoperation in which the operator manipulates the handle of a remote tool in the presence of a registered virtual image of the target in real time. The method adapts a concept already used successfully in a new medical device called the Sonic Flashlight, which permits direct in situ visualization of ultrasound during invasive procedures. The Sonic Flashlight uses a flat-panel monitor and a halt silvered mirror to merge the visual outer surface of a patient with a simultaneous ultrasound scan of the patient's interior. Adapting the concept to scaled teleoperation involves removing the imaging device and the target to a remote location and adding a master-slave control device. This permits the operator to see his hands, along with what appears to De the tool, and the target, merged in a workspace that preserves natural hand-eye coordination. Three functioning prototypes are described, one based on ultrasound and two on light microscopy. The limitations and potential of the new approach are discussed.

KW - Artificial

KW - Augmented

KW - Image display

KW - Medical information systems

KW - Real time

KW - Virtual realities

UR - http://www.scopus.com/inward/record.url?scp=31344474584&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=31344474584&partnerID=8YFLogxK

U2 - 10.1109/TVCG.2006.35

DO - 10.1109/TVCG.2006.35

M3 - Article

C2 - 16509386

AN - SCOPUS:31344474584

VL - 12

SP - 277

EP - 284

JO - IEEE Transactions on Visualization and Computer Graphics

JF - IEEE Transactions on Visualization and Computer Graphics

SN - 1077-2626

IS - 2

ER -