Analysis of composite gestures with a coherent probabilistic graphical model

Jason J. Corso, Guangqi Ye, Gregory D. Hager

Research output: Contribution to journalArticle


Traditionally, gesture-based interaction in virtual environments is composed of either static, posture-based gesture primitives or temporally analyzed dynamic primitives. However, it would be ideal to incorporate both static and dynamic gestures to fully utilize the potential of gesture-based interaction. To that end, we propose a probabilistic framework that incorporates both static and dynamic gesture primitives. We call these primitives Gesture Words (GWords). Using a probabilistic graphical model (PGM), we integrate these heterogeneous GWords and a high-level language model in a coherent fashion. Composite gestures are represented as stochastic paths through the PGM. A gesture is analyzed by finding the path that maximizes the likelihood on the PGM with respect to the video sequence. To facilitate online computation, we propose a greedy algorithm for performing inference on the PGM. The parameters of the PGM can be learned via three different methods: supervised, unsupervised, and hybrid. We have implemented the PGM model for a gesture set of ten GWords with six composite gestures. The experimental results show that the PGM can accurately recognize composite gestures.

Original languageEnglish (US)
Pages (from-to)242-252
Number of pages11
JournalVirtual Reality
Issue number4
StatePublished - Sep 2005


  • Gesture recognition
  • Hand postures
  • Human computer interaction
  • Probabilistic graphical model
  • Vision-based interaction

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design

Fingerprint Dive into the research topics of 'Analysis of composite gestures with a coherent probabilistic graphical model'. Together they form a unique fingerprint.

  • Cite this