Recognizing surgical activities with recurrent neural networks

Robert Dipietro, Colin Lea, Anand Malpani, Narges Ahmidi, S. Swaroop Vedula, Gyusung I. Lee, Mija R. Lee, Gregory Hager

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We apply recurrent neural networks to the task of recognizing surgical activities from robot kinematics. Prior work in this area focuses on recognizing short,low-level activities,or gestures,and has been based on variants of hidden Markov models and conditional random fields. In contrast,we work on recognizing both gestures and longer,higher-level activites,or maneuvers,and we model the mapping from kinematics to gestures/maneuvers with recurrent neural networks. To our knowledge,we are the first to apply recurrent neural networks to this task. Using a single model and a single set of hyperparameters,we match state-of-the-art performance for gesture recognition and advance state-of-the-art performance for maneuver recognition,in terms of both accuracy and edit distance. Code is available at https://github.com/ rdipietro/miccai-2016-surgical-activity-rec.

Original languageEnglish (US)
Title of host publicationMedical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings
PublisherSpringer Verlag
Pages551-558
Number of pages8
Volume9900 LNCS
ISBN (Print)9783319467191
DOIs
StatePublished - 2016
Event1st International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2016 held in conjunction with 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2016 - Athens, Greece
Duration: Oct 21 2016Oct 21 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9900 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other1st International Workshop on Simulation and Synthesis in Medical Imaging, SASHIMI 2016 held in conjunction with 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2016
CountryGreece
CityAthens
Period10/21/1610/21/16

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Recognizing surgical activities with recurrent neural networks'. Together they form a unique fingerprint.

  • Cite this

    Dipietro, R., Lea, C., Malpani, A., Ahmidi, N., Vedula, S. S., Lee, G. I., Lee, M. R., & Hager, G. (2016). Recognizing surgical activities with recurrent neural networks. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Proceedings (Vol. 9900 LNCS, pp. 551-558). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9900 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46720-7_64