Visual modeling of dynamic gestures using 3D appearance and motion features

Guangqi Ye, Jason J. Corso, Gregory Hager

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

We present a novel 3D gesture recognition scheme that combines the 3D appearance of the hand and the motion dynamics of the gesture to classify manipulative and controlling gestures. Our method does not directly track the hand. Instead, we take an object-centered approach that efficiently computes 3D appearance using a region-based coarse stereo matching algorithm. Motion cues are captured by differentiating the appearance feature with respect to time. An unsupervised learning scheme is carried out to capture the cluster structure of these features. Then, the image sequence of a gesture is converted to a series of symbols that indicate the cluster identities of each image pair. Two schemes, i.e., forward HMMs and neural networks, are used to model the dynamics of the gestures. We implemented a real-time system and performed gesture recognition experiments to analyze the performance with different combinations of the appearance and motion features. The system achieves recognition accuracy of over 96% using both the appearance and motion cues.

Original languageEnglish (US)
Title of host publicationReal-Time Vision for Human-Computer Interaction
PublisherSpringer US
Pages103-120
Number of pages18
ISBN (Print)0387276971, 9780387276977
DOIs
StatePublished - 2005

Fingerprint

Gesture recognition
Unsupervised learning
Real time systems
Neural networks
Experiments

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Ye, G., Corso, J. J., & Hager, G. (2005). Visual modeling of dynamic gestures using 3D appearance and motion features. In Real-Time Vision for Human-Computer Interaction (pp. 103-120). Springer US. https://doi.org/10.1007/0-387-27890-7_7

Visual modeling of dynamic gestures using 3D appearance and motion features. / Ye, Guangqi; Corso, Jason J.; Hager, Gregory.

Real-Time Vision for Human-Computer Interaction. Springer US, 2005. p. 103-120.

Research output: Chapter in Book/Report/Conference proceedingChapter

Ye, G, Corso, JJ & Hager, G 2005, Visual modeling of dynamic gestures using 3D appearance and motion features. in Real-Time Vision for Human-Computer Interaction. Springer US, pp. 103-120. https://doi.org/10.1007/0-387-27890-7_7
Ye G, Corso JJ, Hager G. Visual modeling of dynamic gestures using 3D appearance and motion features. In Real-Time Vision for Human-Computer Interaction. Springer US. 2005. p. 103-120 https://doi.org/10.1007/0-387-27890-7_7
Ye, Guangqi ; Corso, Jason J. ; Hager, Gregory. / Visual modeling of dynamic gestures using 3D appearance and motion features. Real-Time Vision for Human-Computer Interaction. Springer US, 2005. pp. 103-120
@inbook{c4474934b2664aabbf035ab6856afb5c,
title = "Visual modeling of dynamic gestures using 3D appearance and motion features",
abstract = "We present a novel 3D gesture recognition scheme that combines the 3D appearance of the hand and the motion dynamics of the gesture to classify manipulative and controlling gestures. Our method does not directly track the hand. Instead, we take an object-centered approach that efficiently computes 3D appearance using a region-based coarse stereo matching algorithm. Motion cues are captured by differentiating the appearance feature with respect to time. An unsupervised learning scheme is carried out to capture the cluster structure of these features. Then, the image sequence of a gesture is converted to a series of symbols that indicate the cluster identities of each image pair. Two schemes, i.e., forward HMMs and neural networks, are used to model the dynamics of the gestures. We implemented a real-time system and performed gesture recognition experiments to analyze the performance with different combinations of the appearance and motion features. The system achieves recognition accuracy of over 96{\%} using both the appearance and motion cues.",
author = "Guangqi Ye and Corso, {Jason J.} and Gregory Hager",
year = "2005",
doi = "10.1007/0-387-27890-7_7",
language = "English (US)",
isbn = "0387276971",
pages = "103--120",
booktitle = "Real-Time Vision for Human-Computer Interaction",
publisher = "Springer US",

}

TY - CHAP

T1 - Visual modeling of dynamic gestures using 3D appearance and motion features

AU - Ye, Guangqi

AU - Corso, Jason J.

AU - Hager, Gregory

PY - 2005

Y1 - 2005

N2 - We present a novel 3D gesture recognition scheme that combines the 3D appearance of the hand and the motion dynamics of the gesture to classify manipulative and controlling gestures. Our method does not directly track the hand. Instead, we take an object-centered approach that efficiently computes 3D appearance using a region-based coarse stereo matching algorithm. Motion cues are captured by differentiating the appearance feature with respect to time. An unsupervised learning scheme is carried out to capture the cluster structure of these features. Then, the image sequence of a gesture is converted to a series of symbols that indicate the cluster identities of each image pair. Two schemes, i.e., forward HMMs and neural networks, are used to model the dynamics of the gestures. We implemented a real-time system and performed gesture recognition experiments to analyze the performance with different combinations of the appearance and motion features. The system achieves recognition accuracy of over 96% using both the appearance and motion cues.

AB - We present a novel 3D gesture recognition scheme that combines the 3D appearance of the hand and the motion dynamics of the gesture to classify manipulative and controlling gestures. Our method does not directly track the hand. Instead, we take an object-centered approach that efficiently computes 3D appearance using a region-based coarse stereo matching algorithm. Motion cues are captured by differentiating the appearance feature with respect to time. An unsupervised learning scheme is carried out to capture the cluster structure of these features. Then, the image sequence of a gesture is converted to a series of symbols that indicate the cluster identities of each image pair. Two schemes, i.e., forward HMMs and neural networks, are used to model the dynamics of the gestures. We implemented a real-time system and performed gesture recognition experiments to analyze the performance with different combinations of the appearance and motion features. The system achieves recognition accuracy of over 96% using both the appearance and motion cues.

UR - http://www.scopus.com/inward/record.url?scp=34548271039&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34548271039&partnerID=8YFLogxK

U2 - 10.1007/0-387-27890-7_7

DO - 10.1007/0-387-27890-7_7

M3 - Chapter

AN - SCOPUS:34548271039

SN - 0387276971

SN - 9780387276977

SP - 103

EP - 120

BT - Real-Time Vision for Human-Computer Interaction

PB - Springer US

ER -