Orientation estimation and grasp type detection of household objects for upper limb prostheses with dynamic vision sensor

Siyi Tang, Rohan Ghosh, Nitish V Thakor, Sunil L. Kukreja

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Although the past decade has seen important advances in prosthetic technologies, grasping household objects with an artificial hand still requires significant skill and effort for an amputee to regulating hand behaviour. A solution to this problem is to automate the process by using vision sensors that determine the object's orientation and optimal grasp procedure. In this paper, we use a neuromorphic dynamic vision sensor (DVS) to assist amputees with object grasping. Event-driven sensors such as the DVS have gained popularity in recent years as an alternative to conventional frame-based sensors due to their low-power consumption and low-latency. Here, we use event data from a DVS to find a grasp-appropriate orientation for the object and subsequently its optimal grasp type. Our estimation technique exploits general assumptions such as object symmetry and grasps preference to be along the smallest major dimension of an object. The grasp type is determined through a combination of multiple convolutional neural network (CNN) classifiers. We evaluated our grasp estimation methodology on a set of 20 household objects. The results of this study show that 96.25% of the estimated orientations were within ±10° of the actual orientations. In addition, our grasp detection method yielded a 99.47% accuracy on unseen object classes.

Original languageEnglish (US)
Title of host publicationProceedings - 2016 IEEE Biomedical Circuits and Systems Conference, BioCAS 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages99-102
Number of pages4
ISBN (Electronic)9781509029594
DOIs
StatePublished - Jan 25 2017
Externally publishedYes
Event12th IEEE Biomedical Circuits and Systems Conference, BioCAS 2016 - Shanghai, China
Duration: Oct 17 2016Oct 19 2016

Other

Other12th IEEE Biomedical Circuits and Systems Conference, BioCAS 2016
CountryChina
CityShanghai
Period10/17/1610/19/16

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Instrumentation
  • Biomedical Engineering

Fingerprint Dive into the research topics of 'Orientation estimation and grasp type detection of household objects for upper limb prostheses with dynamic vision sensor'. Together they form a unique fingerprint.

Cite this