Vision-based finger tapping detection without fingertip observation

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)


A machine learning approach is investigated in this study to detect a finger tapping on a handheld sur-face, where the movement of the surface is observed visually; however, the tapping finger is not directly visible. A feature vector extracted from consecutive frames captured by a high-speed camera that observes a surface patch is input to a convolutional neural network to provide a prediction label indicating whether the surface is tapped within the sequence of consecutive frames (“tap”), the surface is still (“still”), or the surface is moved by hand (“move”). Receiver operat-ing characteristics analysis on a binary discrimination of “tap” from the other two labels shows that true positive rates exceeding 97% are achieved when the false positive rate is fixed at 3%, although the generaliza-tion performance against different tapped objects or different ways of tapping is not satisfactory. An in-formal test where a heuristic post-processing filter is introduced suggests that the use of temporal history information should be considered for further improve-ments. biho.

Original languageEnglish
Pages (from-to)484-493
Number of pages10
JournalJournal of Robotics and Mechatronics
Issue number3
Publication statusPublished - 2021


  • Convolutional neural network
  • Finger tapping
  • High-speed vision
  • Touch interface

ASJC Scopus subject areas

  • Computer Science(all)
  • Electrical and Electronic Engineering


Dive into the research topics of 'Vision-based finger tapping detection without fingertip observation'. Together they form a unique fingerprint.

Cite this