Stable haptic feedback generation for mid-air gesture interactions: a hidden Markov model-based motion synthesis approach

Dennis Babu, Masashi Konyo, Hikaru Nagano, Ryunosuke Hamada, Satoshi Tadokoro

Research output: Contribution to journalArticle

Abstract

Generation of stable and realistic haptic feedback during mid-air gesture interactions have recently garnered significant research interest. However, the limitations of the sensing technologies such as unstable tracking, range limitations, nonuniform sampling duration, self occlusions, and motion recognition faults significantly distort motion based haptic feedback to a large extent. In this paper, we propose and implement a hidden Markov model (HMM)-based motion synthesis method to generate stable concurrent and terminal vibrotactile feedback. The system tracks human gestures during interaction and recreates smooth, synchronized motion data from detected HMM states. Four gestures—tapping, three-fingered zooming, vertical dragging, and horizontal dragging—were used in the study to evaluate the performance of the motion synthesis methodology. The reference motion curves and corresponding primitive motion elements to be synthesized for each gesture were obtained from multiple subjects at different interaction speeds by using a stable motion tracking sensor. Both objective and subjective evaluations were conducted to evaluate the performance of the motion synthesis model in controlling both concurrent and terminal vibrotactile feedback. Objective evaluation shows that synthesized motion data had a high correlation for shape and end-timings with the reference motion data compared to measured and moving average filtered data. The mean R 2 values for synthesized motion data was always greater than 0.7 even under unstable tracking conditions. The experimental results of subjective evaluation from nine subjects showed significant improvement in perceived synchronization of vibrotactile feedback based on synthesized motion.

Original languageEnglish
Article number2
JournalROBOMECH Journal
Volume6
Issue number1
DOIs
Publication statusPublished - 2019 Dec 1

Keywords

  • Hidden Markov model
  • Mid-air interaction
  • Motion synthesis
  • Occlusion
  • Stable haptic feedback
  • Unstable tracking
  • Vibrotactile feedback

ASJC Scopus subject areas

  • Modelling and Simulation
  • Instrumentation
  • Mechanical Engineering
  • Control and Optimization
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Stable haptic feedback generation for mid-air gesture interactions: a hidden Markov model-based motion synthesis approach'. Together they form a unique fingerprint.

  • Cite this