RadSense: Enabling one hand and no hands interaction for sterile manipulation of medical images using Doppler radar

Elishiah Miller, Zheng Li, Helena Mentis, Adrian Park, Ting Zhu, Nilanjan Banerjee

Research output: Contribution to journalArticle

Abstract

In this paper, we show how surgeons can interact with medical images using finger and hand gestures in two situations: one hand-free and no hands-free interaction. We explain how interaction with only one hand or a couple of fingers is beneficial and can help surgeons have continuous interaction, without the need to release their tools and leave the operating table, saving valuable patient time. To this end, we present RadSense, an end-to-end and unobtrusive system that uses Doppler radar-sensing to recognize hand and finger gestures when either one or both hands are busy. Our system permits the following important capabilities: (1) touch-less input for sterile interaction with connected health applications, (2) hand and finger gesture recognition when either one or both hands are busy holding tools, extending multitasking capabilities for health professionals, and (3) mobile and networked, allowing for custom wearable and non-wearable configurations. We evaluated our system in a simulated operating room to manipulate preoperative images using four gestures: circle, double tap, swipe, and finger click. We collected data from five subjects and trained a K-Nearest-Neighbor multi-class classifier using 15-fold cross validation, achieving a 94.5% precision for gesture classification. We conclude that our system performs with high accuracy and is useful in cases where only one hand or a few fingers are free to interact when the hands are busy.

Original languageEnglish (US)
Article number100089
JournalSmart Health
Volume15
DOIs
Publication statusPublished - Mar 2020

    Fingerprint

Keywords

  • Busy hand interaction
  • Gesture recognition
  • Healthcare
  • Human centered computing
  • Wearable devices

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Information Systems
  • Health Informatics
  • Computer Science Applications
  • Health Information Management

Cite this