Recognizing surgical activities with recurrent neural networks

Robert Dipietro, Colin Lea, Anand Malpani, Narges Ahmidi, S. Swaroop Vedula, Gyusung I. Lee, Mija R. Lee, Gregory Hager

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We apply recurrent neural networks to the task of recognizing surgical activities from robot kinematics. Prior work in this area focuses on recognizing short,low-level activities,or gestures,and has been based on variants of hidden Markov models and conditional random fields. In contrast,we work on recognizing both gestures and longer,higher-level activites,or maneuvers,and we model the mapping from kinematics to gestures/maneuvers with recurrent neural networks. To our knowledge,we are the first to apply recurrent neural networks to this task. Using a single model and a single set of hyperparameters,we match state-of-the-art performance for gesture recognition and advance state-of-the-art performance for maneuver recognition,in terms of both accuracy and edit distance. Code is available at https://github.com/ rdipietro/miccai-2016-surgical-activity-rec.

Original languageEnglish (US)
Title of host publicationMedical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings
PublisherSpringer Verlag
Pages551-558
Number of pages8
Volume9900 LNCS
ISBN (Print)9783319467191
DOIs
StatePublished - 2016
Event1st International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2016 held in conjunction with 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2016 - Athens, Greece
Duration: Oct 21 2016Oct 21 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9900 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other1st International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2016 held in conjunction with 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2016
CountryGreece
CityAthens
Period10/21/1610/21/16

Fingerprint

Recurrent neural networks
Recurrent Neural Networks
Gesture
Kinematics
Gesture recognition
Gesture Recognition
Conditional Random Fields
Edit Distance
Hyperparameters
Hidden Markov models
Markov Model
Robot
Robots
Model

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Dipietro, R., Lea, C., Malpani, A., Ahmidi, N., Vedula, S. S., Lee, G. I., ... Hager, G. (2016). Recognizing surgical activities with recurrent neural networks. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings (Vol. 9900 LNCS, pp. 551-558). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9900 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46720-7_64

Recognizing surgical activities with recurrent neural networks. / Dipietro, Robert; Lea, Colin; Malpani, Anand; Ahmidi, Narges; Vedula, S. Swaroop; Lee, Gyusung I.; Lee, Mija R.; Hager, Gregory.

Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings. Vol. 9900 LNCS Springer Verlag, 2016. p. 551-558 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9900 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Dipietro, R, Lea, C, Malpani, A, Ahmidi, N, Vedula, SS, Lee, GI, Lee, MR & Hager, G 2016, Recognizing surgical activities with recurrent neural networks. in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings. vol. 9900 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9900 LNCS, Springer Verlag, pp. 551-558, 1st International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2016 held in conjunction with 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2016, Athens, Greece, 10/21/16. https://doi.org/10.1007/978-3-319-46720-7_64
Dipietro R, Lea C, Malpani A, Ahmidi N, Vedula SS, Lee GI et al. Recognizing surgical activities with recurrent neural networks. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings. Vol. 9900 LNCS. Springer Verlag. 2016. p. 551-558. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-46720-7_64
Dipietro, Robert ; Lea, Colin ; Malpani, Anand ; Ahmidi, Narges ; Vedula, S. Swaroop ; Lee, Gyusung I. ; Lee, Mija R. ; Hager, Gregory. / Recognizing surgical activities with recurrent neural networks. Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings. Vol. 9900 LNCS Springer Verlag, 2016. pp. 551-558 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{d242b16e031f419e8b05b5e8b52976ab,
title = "Recognizing surgical activities with recurrent neural networks",
abstract = "We apply recurrent neural networks to the task of recognizing surgical activities from robot kinematics. Prior work in this area focuses on recognizing short,low-level activities,or gestures,and has been based on variants of hidden Markov models and conditional random fields. In contrast,we work on recognizing both gestures and longer,higher-level activites,or maneuvers,and we model the mapping from kinematics to gestures/maneuvers with recurrent neural networks. To our knowledge,we are the first to apply recurrent neural networks to this task. Using a single model and a single set of hyperparameters,we match state-of-the-art performance for gesture recognition and advance state-of-the-art performance for maneuver recognition,in terms of both accuracy and edit distance. Code is available at https://github.com/ rdipietro/miccai-2016-surgical-activity-rec.",
author = "Robert Dipietro and Colin Lea and Anand Malpani and Narges Ahmidi and Vedula, {S. Swaroop} and Lee, {Gyusung I.} and Lee, {Mija R.} and Gregory Hager",
year = "2016",
doi = "10.1007/978-3-319-46720-7_64",
language = "English (US)",
isbn = "9783319467191",
volume = "9900 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "551--558",
booktitle = "Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings",

}

TY - GEN

T1 - Recognizing surgical activities with recurrent neural networks

AU - Dipietro, Robert

AU - Lea, Colin

AU - Malpani, Anand

AU - Ahmidi, Narges

AU - Vedula, S. Swaroop

AU - Lee, Gyusung I.

AU - Lee, Mija R.

AU - Hager, Gregory

PY - 2016

Y1 - 2016

N2 - We apply recurrent neural networks to the task of recognizing surgical activities from robot kinematics. Prior work in this area focuses on recognizing short,low-level activities,or gestures,and has been based on variants of hidden Markov models and conditional random fields. In contrast,we work on recognizing both gestures and longer,higher-level activites,or maneuvers,and we model the mapping from kinematics to gestures/maneuvers with recurrent neural networks. To our knowledge,we are the first to apply recurrent neural networks to this task. Using a single model and a single set of hyperparameters,we match state-of-the-art performance for gesture recognition and advance state-of-the-art performance for maneuver recognition,in terms of both accuracy and edit distance. Code is available at https://github.com/ rdipietro/miccai-2016-surgical-activity-rec.

AB - We apply recurrent neural networks to the task of recognizing surgical activities from robot kinematics. Prior work in this area focuses on recognizing short,low-level activities,or gestures,and has been based on variants of hidden Markov models and conditional random fields. In contrast,we work on recognizing both gestures and longer,higher-level activites,or maneuvers,and we model the mapping from kinematics to gestures/maneuvers with recurrent neural networks. To our knowledge,we are the first to apply recurrent neural networks to this task. Using a single model and a single set of hyperparameters,we match state-of-the-art performance for gesture recognition and advance state-of-the-art performance for maneuver recognition,in terms of both accuracy and edit distance. Code is available at https://github.com/ rdipietro/miccai-2016-surgical-activity-rec.

UR - http://www.scopus.com/inward/record.url?scp=84996486504&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84996486504&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46720-7_64

DO - 10.1007/978-3-319-46720-7_64

M3 - Conference contribution

AN - SCOPUS:84996486504

SN - 9783319467191

VL - 9900 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 551

EP - 558

BT - Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings

PB - Springer Verlag

ER -