An improved model for segmentation and recognition of fine-grained activities with application to surgical training tasks

Colin Lea, Gregory D. Hager, René Vidal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Automated segmentation and recognition of fine-grained activities is important for enabling new applications in industrial automation, human-robot collaboration, and surgical training. Many existing approaches to activity recognition assume that a video has already been segmented and perform classification using an abstract representation based on spatio-temporal features. While some approaches perform joint activity segmentation and recognition, they typically suffer from a poor modeling of the transitions between actions and a representation that does not incorporate contextual information about the scene. In this paper, we propose a model for action segmentation and recognition that improves upon existing work in two directions. First, we develop a variation of the Skip-Chain Conditional Random Field that captures long-range state transitions between actions by using higher-order temporal relationships. Second, we argue that in constrained environments, where the relevant set of objects is known, it is better to develop features using high-level object relationships that have semantic meaning instead of relying on abstract features. We apply our approach to a set of tasks common for training in robotic surgery: suturing, knot tying, and needle passing, and show that our method increases micro and macro accuracy by 18.46% and 44.13% relative to the state of the art on a widely used robotic surgery dataset.

Original languageEnglish (US)
Title of host publicationProceedings - 2015 IEEE Winter Conference on Applications of Computer Vision, WACV 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1123-1129
Number of pages7
ISBN (Electronic)9781479966820
DOIs
StatePublished - Feb 19 2015
Event2015 15th IEEE Winter Conference on Applications of Computer Vision, WACV 2015 - Waikoloa, United States
Duration: Jan 5 2015Jan 9 2015

Publication series

NameProceedings - 2015 IEEE Winter Conference on Applications of Computer Vision, WACV 2015

Other

Other2015 15th IEEE Winter Conference on Applications of Computer Vision, WACV 2015
CountryUnited States
CityWaikoloa
Period1/5/151/9/15

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'An improved model for segmentation and recognition of fine-grained activities with application to surgical training tasks'. Together they form a unique fingerprint.

Cite this