Vision-based calibration of dual RCM-based robot arms in human-robot collaborative minimally invasive surgery

Zerui Wang, Ziwei Liu, Qianli Ma, Alexis Cheng, Yun Hui Liu, Sungmin Kim, Anton Deguet, Austin Reiter, Peter Kazanzides, Russell H Taylor

Research output: Contribution to journalArticle

Abstract

This letter reports the development of a vision-based calibration method for dual remote center-of-motion (RCM) based robot arms in a human-robot collaborative minimally invasive surgery (MIS) scenario. The method does not require any external tracking sensors and directly uses images captured by the endoscopic camera and the robot encoder readings as calibration data, which leads to a minimal and practical system in the operating room. By taking advantage of the motion constraints imposed by the RCM-based kinematics of the robotic surgical tools and cameras, we can find unique relationships between the endoscope and the surgical tool using camera perspective projection geometry without the geometric information of the tool. A customized vision-based centerline detection algorithm is also proposed, which provides robust estimation of centerline positions for a variety of settings. We validate the method through simulations and an experimental study in a simulated MIS scenario, in which the first generation da Vinci surgical system controlled by the open source da Vinci research kit electronics and cisst/SAW software environment is used.

Original languageEnglish (US)
Pages (from-to)672-679
Number of pages8
JournalIEEE Robotics and Automation Letters
Volume3
Issue number2
DOIs
StatePublished - Apr 1 2018

Fingerprint

Minimally Invasive Surgery
Surgery
Calibration
Robot
Camera
Cameras
Robots
Motion
Endoscope
Operating rooms
Scenarios
Endoscopy
Robust Estimation
Encoder
Open Source
Robotics
Experimental Study
Kinematics
Electronic equipment
Electronics

Keywords

  • Calibration and identification
  • Computer vision for medical robotics
  • Laparoscopy
  • Surgical robotics

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Human-Computer Interaction
  • Biomedical Engineering
  • Mechanical Engineering
  • Control and Optimization
  • Artificial Intelligence
  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Cite this

Vision-based calibration of dual RCM-based robot arms in human-robot collaborative minimally invasive surgery. / Wang, Zerui; Liu, Ziwei; Ma, Qianli; Cheng, Alexis; Liu, Yun Hui; Kim, Sungmin; Deguet, Anton; Reiter, Austin; Kazanzides, Peter; Taylor, Russell H.

In: IEEE Robotics and Automation Letters, Vol. 3, No. 2, 01.04.2018, p. 672-679.

Research output: Contribution to journalArticle

Wang, Zerui ; Liu, Ziwei ; Ma, Qianli ; Cheng, Alexis ; Liu, Yun Hui ; Kim, Sungmin ; Deguet, Anton ; Reiter, Austin ; Kazanzides, Peter ; Taylor, Russell H. / Vision-based calibration of dual RCM-based robot arms in human-robot collaborative minimally invasive surgery. In: IEEE Robotics and Automation Letters. 2018 ; Vol. 3, No. 2. pp. 672-679.
@article{8672a0e737854b4596bb59a9259a2d33,
title = "Vision-based calibration of dual RCM-based robot arms in human-robot collaborative minimally invasive surgery",
abstract = "This letter reports the development of a vision-based calibration method for dual remote center-of-motion (RCM) based robot arms in a human-robot collaborative minimally invasive surgery (MIS) scenario. The method does not require any external tracking sensors and directly uses images captured by the endoscopic camera and the robot encoder readings as calibration data, which leads to a minimal and practical system in the operating room. By taking advantage of the motion constraints imposed by the RCM-based kinematics of the robotic surgical tools and cameras, we can find unique relationships between the endoscope and the surgical tool using camera perspective projection geometry without the geometric information of the tool. A customized vision-based centerline detection algorithm is also proposed, which provides robust estimation of centerline positions for a variety of settings. We validate the method through simulations and an experimental study in a simulated MIS scenario, in which the first generation da Vinci surgical system controlled by the open source da Vinci research kit electronics and cisst/SAW software environment is used.",
keywords = "Calibration and identification, Computer vision for medical robotics, Laparoscopy, Surgical robotics",
author = "Zerui Wang and Ziwei Liu and Qianli Ma and Alexis Cheng and Liu, {Yun Hui} and Sungmin Kim and Anton Deguet and Austin Reiter and Peter Kazanzides and Taylor, {Russell H}",
year = "2018",
month = "4",
day = "1",
doi = "10.1109/LRA.2017.2737485",
language = "English (US)",
volume = "3",
pages = "672--679",
journal = "IEEE Robotics and Automation Letters",
issn = "2377-3766",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "2",

}

TY - JOUR

T1 - Vision-based calibration of dual RCM-based robot arms in human-robot collaborative minimally invasive surgery

AU - Wang, Zerui

AU - Liu, Ziwei

AU - Ma, Qianli

AU - Cheng, Alexis

AU - Liu, Yun Hui

AU - Kim, Sungmin

AU - Deguet, Anton

AU - Reiter, Austin

AU - Kazanzides, Peter

AU - Taylor, Russell H

PY - 2018/4/1

Y1 - 2018/4/1

N2 - This letter reports the development of a vision-based calibration method for dual remote center-of-motion (RCM) based robot arms in a human-robot collaborative minimally invasive surgery (MIS) scenario. The method does not require any external tracking sensors and directly uses images captured by the endoscopic camera and the robot encoder readings as calibration data, which leads to a minimal and practical system in the operating room. By taking advantage of the motion constraints imposed by the RCM-based kinematics of the robotic surgical tools and cameras, we can find unique relationships between the endoscope and the surgical tool using camera perspective projection geometry without the geometric information of the tool. A customized vision-based centerline detection algorithm is also proposed, which provides robust estimation of centerline positions for a variety of settings. We validate the method through simulations and an experimental study in a simulated MIS scenario, in which the first generation da Vinci surgical system controlled by the open source da Vinci research kit electronics and cisst/SAW software environment is used.

AB - This letter reports the development of a vision-based calibration method for dual remote center-of-motion (RCM) based robot arms in a human-robot collaborative minimally invasive surgery (MIS) scenario. The method does not require any external tracking sensors and directly uses images captured by the endoscopic camera and the robot encoder readings as calibration data, which leads to a minimal and practical system in the operating room. By taking advantage of the motion constraints imposed by the RCM-based kinematics of the robotic surgical tools and cameras, we can find unique relationships between the endoscope and the surgical tool using camera perspective projection geometry without the geometric information of the tool. A customized vision-based centerline detection algorithm is also proposed, which provides robust estimation of centerline positions for a variety of settings. We validate the method through simulations and an experimental study in a simulated MIS scenario, in which the first generation da Vinci surgical system controlled by the open source da Vinci research kit electronics and cisst/SAW software environment is used.

KW - Calibration and identification

KW - Computer vision for medical robotics

KW - Laparoscopy

KW - Surgical robotics

UR - http://www.scopus.com/inward/record.url?scp=85063306326&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063306326&partnerID=8YFLogxK

U2 - 10.1109/LRA.2017.2737485

DO - 10.1109/LRA.2017.2737485

M3 - Article

AN - SCOPUS:85063306326

VL - 3

SP - 672

EP - 679

JO - IEEE Robotics and Automation Letters

JF - IEEE Robotics and Automation Letters

SN - 2377-3766

IS - 2

ER -