VICs: A modular vision-based HCI framework

Guangqi Ye, Jason Corso, Darius Burschka, Gregory Hager

Research output: Contribution to journalArticle

Abstract

Many Vision-Based Human-Computer Interaction (VB-HCI) systems are based on the tracking of user actions. Examples include gaze-tracking, head-tracking, finger-tracking, and so forth. In this paper, we present a framework that employs no user-tracking; instead, all interface components continuously observe and react to changes within a local image neighborhood. More specifically, components expect a predefined sequence of visual events called Visual Interface Cues (VICs). VICs include color, texture, motion and geometric elements, arranged to maximize the veridicality of the resulting interface element. A component is executed when this stream of cues has been satisfied. We present a general architecture for an interface system operating under the VIC-Based HCI paradigm, and then focus specifically on an appearance-based system in which a Hidden Markov Model (HMM) is employed to learn the gesture dynamics. Our implementation of the system successfully recognizes a button-push with a 96% success rate. The system operates at frame-rate on standard PCs.

Original languageEnglish (US)
Pages (from-to)257-267
Number of pages11
JournalLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume2626
StatePublished - 2003

Fingerprint

Human computer interaction
Interfaces (computer)
Computer operating systems
Hidden Markov models
Textures
Color
Interface Element
Gesture
Markov Model
Texture
Maximise
Paradigm
Framework
Vision
Motion
Interaction

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

VICs : A modular vision-based HCI framework. / Ye, Guangqi; Corso, Jason; Burschka, Darius; Hager, Gregory.

In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 2626, 2003, p. 257-267.

Research output: Contribution to journalArticle

@article{04dffe39ae8b433fa56ffd4ed49c8b90,
title = "VICs: A modular vision-based HCI framework",
abstract = "Many Vision-Based Human-Computer Interaction (VB-HCI) systems are based on the tracking of user actions. Examples include gaze-tracking, head-tracking, finger-tracking, and so forth. In this paper, we present a framework that employs no user-tracking; instead, all interface components continuously observe and react to changes within a local image neighborhood. More specifically, components expect a predefined sequence of visual events called Visual Interface Cues (VICs). VICs include color, texture, motion and geometric elements, arranged to maximize the veridicality of the resulting interface element. A component is executed when this stream of cues has been satisfied. We present a general architecture for an interface system operating under the VIC-Based HCI paradigm, and then focus specifically on an appearance-based system in which a Hidden Markov Model (HMM) is employed to learn the gesture dynamics. Our implementation of the system successfully recognizes a button-push with a 96{\%} success rate. The system operates at frame-rate on standard PCs.",
author = "Guangqi Ye and Jason Corso and Darius Burschka and Gregory Hager",
year = "2003",
language = "English (US)",
volume = "2626",
pages = "257--267",
journal = "Lecture Notes in Computer Science",
issn = "0302-9743",
publisher = "Springer Verlag",

}

TY - JOUR

T1 - VICs

T2 - A modular vision-based HCI framework

AU - Ye, Guangqi

AU - Corso, Jason

AU - Burschka, Darius

AU - Hager, Gregory

PY - 2003

Y1 - 2003

N2 - Many Vision-Based Human-Computer Interaction (VB-HCI) systems are based on the tracking of user actions. Examples include gaze-tracking, head-tracking, finger-tracking, and so forth. In this paper, we present a framework that employs no user-tracking; instead, all interface components continuously observe and react to changes within a local image neighborhood. More specifically, components expect a predefined sequence of visual events called Visual Interface Cues (VICs). VICs include color, texture, motion and geometric elements, arranged to maximize the veridicality of the resulting interface element. A component is executed when this stream of cues has been satisfied. We present a general architecture for an interface system operating under the VIC-Based HCI paradigm, and then focus specifically on an appearance-based system in which a Hidden Markov Model (HMM) is employed to learn the gesture dynamics. Our implementation of the system successfully recognizes a button-push with a 96% success rate. The system operates at frame-rate on standard PCs.

AB - Many Vision-Based Human-Computer Interaction (VB-HCI) systems are based on the tracking of user actions. Examples include gaze-tracking, head-tracking, finger-tracking, and so forth. In this paper, we present a framework that employs no user-tracking; instead, all interface components continuously observe and react to changes within a local image neighborhood. More specifically, components expect a predefined sequence of visual events called Visual Interface Cues (VICs). VICs include color, texture, motion and geometric elements, arranged to maximize the veridicality of the resulting interface element. A component is executed when this stream of cues has been satisfied. We present a general architecture for an interface system operating under the VIC-Based HCI paradigm, and then focus specifically on an appearance-based system in which a Hidden Markov Model (HMM) is employed to learn the gesture dynamics. Our implementation of the system successfully recognizes a button-push with a 96% success rate. The system operates at frame-rate on standard PCs.

UR - http://www.scopus.com/inward/record.url?scp=84886606642&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84886606642&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:84886606642

VL - 2626

SP - 257

EP - 267

JO - Lecture Notes in Computer Science

JF - Lecture Notes in Computer Science

SN - 0302-9743

ER -