TY - GEN
T1 - A three tiered approach for articulated object action modeling and recognition
AU - Lu, Le
AU - Hager, Gregory D.
AU - Younes, Laurent
PY - 2005
Y1 - 2005
N2 - Visual action recognition is an important problem in computer vision. In this paper, we propose a new method to probabilistically model and recognize actions of articulated objects, such as hand or body gestures, in image sequences. Our method consists of three levels of representation. At the low level, we first extract a feature vector invariant to scale and in-plane rotation by using the Fourier transform of a circular spatial histogram. Then, spectral partitioning [20] is utilized to obtain an initial clustering; this clustering is then refined using a temporal smoothness constraint. Gaussian mixturemodel (GMM) based clustering and density estimation in the subspace of linear discriminant analysis (LDA) are then applied to thousands of image feature vectors to obtain an intermediate level representation. Finally, at the high level we build a temporal multiresolution histogram model for each action by aggregating the clustering weights of sampled images belonging to that action. We discuss how this high level representation can be extended to achieve temporal scaling invariance and to include Bi-gram or Multi-gram transition information. Both image clustering and action recognition/segmentation results are given to show the validity of our three tiered representation.
AB - Visual action recognition is an important problem in computer vision. In this paper, we propose a new method to probabilistically model and recognize actions of articulated objects, such as hand or body gestures, in image sequences. Our method consists of three levels of representation. At the low level, we first extract a feature vector invariant to scale and in-plane rotation by using the Fourier transform of a circular spatial histogram. Then, spectral partitioning [20] is utilized to obtain an initial clustering; this clustering is then refined using a temporal smoothness constraint. Gaussian mixturemodel (GMM) based clustering and density estimation in the subspace of linear discriminant analysis (LDA) are then applied to thousands of image feature vectors to obtain an intermediate level representation. Finally, at the high level we build a temporal multiresolution histogram model for each action by aggregating the clustering weights of sampled images belonging to that action. We discuss how this high level representation can be extended to achieve temporal scaling invariance and to include Bi-gram or Multi-gram transition information. Both image clustering and action recognition/segmentation results are given to show the validity of our three tiered representation.
UR - http://www.scopus.com/inward/record.url?scp=70350389487&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=70350389487&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:70350389487
SN - 0262195348
SN - 9780262195348
T3 - Advances in Neural Information Processing Systems
BT - Advances in Neural Information Processing Systems 17 - Proceedings of the 2004 Conference, NIPS 2004
PB - Neural information processing systems foundation
T2 - 18th Annual Conference on Neural Information Processing Systems, NIPS 2004
Y2 - 13 December 2004 through 16 December 2004
ER -