Learning convolutional action primitives for fine-grained action recognition

Colin Lea, René Vidal, Gregory Hager

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Fine-grained action recognition is important for many applications of human-robot interaction, automated skill assessment, and surveillance. The goal is to segment and classify all actions occurring in a time series sequence. While recent recognition methods have shown strong performance in robotics applications, they often require hand-crafted features, use large amounts of domain knowledge, or employ overly simplistic representations of how objects change throughout an action. In this paper we present the Latent Convolutional Skip Chain Conditional Random Field (LC-SC-CRF). This time series model learns a set of interpretable and composable action primitives from sensor data. We apply our model to cooking tasks using accelerometer data from the University of Dundee 50 Salads dataset and to robotic surgery training tasks using robot kinematic data from the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS). Our performance on 50 Salads and JIGSAWS are 18.0% and 5.3% higher than the state of the art, respectively. This model performs well without requiring hand-crafted features or intricate domain knowledge. The code and features have been made public.

Original languageEnglish (US)
Title of host publication2016 IEEE International Conference on Robotics and Automation, ICRA 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1642-1649
Number of pages8
Volume2016-June
ISBN (Electronic)9781467380263
DOIs
StatePublished - Jun 8 2016
Event2016 IEEE International Conference on Robotics and Automation, ICRA 2016 - Stockholm, Sweden
Duration: May 16 2016May 21 2016

Other

Other2016 IEEE International Conference on Robotics and Automation, ICRA 2016
CountrySweden
CityStockholm
Period5/16/165/21/16

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Learning convolutional action primitives for fine-grained action recognition'. Together they form a unique fingerprint.

  • Cite this

    Lea, C., Vidal, R., & Hager, G. (2016). Learning convolutional action primitives for fine-grained action recognition. In 2016 IEEE International Conference on Robotics and Automation, ICRA 2016 (Vol. 2016-June, pp. 1642-1649). [7487305] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICRA.2016.7487305