TY - JOUR
T1 - Stable haptic feedback generation for mid-air gesture interactions
T2 - a hidden Markov model-based motion synthesis approach
AU - Babu, Dennis
AU - Konyo, Masashi
AU - Nagano, Hikaru
AU - Hamada, Ryunosuke
AU - Tadokoro, Satoshi
N1 - Funding Information:
DB and MK developed the original concept and analytical formulation. HN guided DB in experimental design and analysis of the experimental design. RH guided DB in the hidden markov model implementation. ST encouraged DB to investigate [a specific aspect] and supervised the findings of this work. DB wrote the manuscript with inputs from MK. All authors discussed the results and contributed to the final manuscript. All authors read and approved the final manuscript. The authors would like to thank all the members of Human Robot Informatics Lab, Tohoku University for helping conduct the experiments and for their fruitfull feedbacks. The authors declare that they have no competing interests. This work was supported in part by ImPACT (Tough Robotics Challenge). Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Publisher Copyright:
© 2019, The Author(s).
PY - 2019/12/1
Y1 - 2019/12/1
N2 - Generation of stable and realistic haptic feedback during mid-air gesture interactions have recently garnered significant research interest. However, the limitations of the sensing technologies such as unstable tracking, range limitations, nonuniform sampling duration, self occlusions, and motion recognition faults significantly distort motion based haptic feedback to a large extent. In this paper, we propose and implement a hidden Markov model (HMM)-based motion synthesis method to generate stable concurrent and terminal vibrotactile feedback. The system tracks human gestures during interaction and recreates smooth, synchronized motion data from detected HMM states. Four gestures—tapping, three-fingered zooming, vertical dragging, and horizontal dragging—were used in the study to evaluate the performance of the motion synthesis methodology. The reference motion curves and corresponding primitive motion elements to be synthesized for each gesture were obtained from multiple subjects at different interaction speeds by using a stable motion tracking sensor. Both objective and subjective evaluations were conducted to evaluate the performance of the motion synthesis model in controlling both concurrent and terminal vibrotactile feedback. Objective evaluation shows that synthesized motion data had a high correlation for shape and end-timings with the reference motion data compared to measured and moving average filtered data. The mean R2 values for synthesized motion data was always greater than 0.7 even under unstable tracking conditions. The experimental results of subjective evaluation from nine subjects showed significant improvement in perceived synchronization of vibrotactile feedback based on synthesized motion.
AB - Generation of stable and realistic haptic feedback during mid-air gesture interactions have recently garnered significant research interest. However, the limitations of the sensing technologies such as unstable tracking, range limitations, nonuniform sampling duration, self occlusions, and motion recognition faults significantly distort motion based haptic feedback to a large extent. In this paper, we propose and implement a hidden Markov model (HMM)-based motion synthesis method to generate stable concurrent and terminal vibrotactile feedback. The system tracks human gestures during interaction and recreates smooth, synchronized motion data from detected HMM states. Four gestures—tapping, three-fingered zooming, vertical dragging, and horizontal dragging—were used in the study to evaluate the performance of the motion synthesis methodology. The reference motion curves and corresponding primitive motion elements to be synthesized for each gesture were obtained from multiple subjects at different interaction speeds by using a stable motion tracking sensor. Both objective and subjective evaluations were conducted to evaluate the performance of the motion synthesis model in controlling both concurrent and terminal vibrotactile feedback. Objective evaluation shows that synthesized motion data had a high correlation for shape and end-timings with the reference motion data compared to measured and moving average filtered data. The mean R2 values for synthesized motion data was always greater than 0.7 even under unstable tracking conditions. The experimental results of subjective evaluation from nine subjects showed significant improvement in perceived synchronization of vibrotactile feedback based on synthesized motion.
KW - Hidden Markov model
KW - Mid-air interaction
KW - Motion synthesis
KW - Occlusion
KW - Stable haptic feedback
KW - Unstable tracking
KW - Vibrotactile feedback
UR - http://www.scopus.com/inward/record.url?scp=85061483732&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85061483732&partnerID=8YFLogxK
U2 - 10.1186/s40648-019-0130-5
DO - 10.1186/s40648-019-0130-5
M3 - Article
AN - SCOPUS:85061483732
VL - 6
JO - ROBOMECH Journal
JF - ROBOMECH Journal
SN - 2197-4225
IS - 1
M1 - 2
ER -