Optical merger of direct vision with virtual images for scaled teleoperation

Samuel T. Clanton, David C. Wang, Vikram S. Chib, Yoky Matsuoka, George D. Stetten

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


Scaled teleoperation is increasingly prevalent in medicine, as well as in other applications of robotics. Visual feedback in such systems is essential and should make maximal use of natural hand-eye coordination. This paper describes a new method of visual feedback for scaled teleoperation in which the operator manipulates the handle of a remote tool in the presence of a registered virtual image of the target in real time. The method adapts a concept already used successfully in a new medical device called the Sonic Flashlight, which permits direct in situ visualization of ultrasound during invasive procedures. The Sonic Flashlight uses a flat-panel monitor and a halt silvered mirror to merge the visual outer surface of a patient with a simultaneous ultrasound scan of the patient's interior. Adapting the concept to scaled teleoperation involves removing the imaging device and the target to a remote location and adding a master-slave control device. This permits the operator to see his hands, along with what appears to De the tool, and the target, merged in a workspace that preserves natural hand-eye coordination. Three functioning prototypes are described, one based on ultrasound and two on light microscopy. The limitations and potential of the new approach are discussed.

Original languageEnglish (US)
Pages (from-to)277-284
Number of pages8
JournalIEEE Transactions on Visualization and Computer Graphics
Issue number2
StatePublished - Mar 2006
Externally publishedYes


  • Artificial
  • Augmented
  • Image display
  • Medical information systems
  • Real time
  • Virtual realities

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Optical merger of direct vision with virtual images for scaled teleoperation'. Together they form a unique fingerprint.

Cite this