Mixed-reality human-machine-interface for motor learning of physical activities

Regular physical activity reduces the risk of suffering obesity and high blood pressure, and slows down age-related loss of mobility and cognitive capabilities. However, 31% of the world population does not perform even the minimum recommend levels of physical activity to have a healthy life. On top of that, due to the COVID-19 Pandemic prevention measures involving isolation, lockdown, and working-from-home policies, adults have drastically reduced their physical activity by 30%, which further aggravates existing health conditions. In order to encourage exercising at home while still receiving proper instruction, this paper proposes a human-machine interface capable of supporting the motor learning of physical activities by providing training with constant practice of exercises and multimodal feedback. It consists of an interactive mixed-reality environment that does not require a human instructor or specialized facilities. As an application of the system, dance coaching was implemented. The information to be conveyed to the users are feet velocity and position trajectories, as well as the tempo of the desired motion. This is done by providing directional haptic feedback with wearable vibroactuators on the ankles of the user, visual feedback with a floor projection, and aural feedback with a metronome. In order to validate the proposed methodology, an experiment where ballroom dance is taught to 10 novice subjects was performed. Results show that when using the developed multimodal system, position and velocity trajectory errors are reduced by 60% and 37%, respectively, which demonstrates that users can understand and follow the multimodal feedback. After finishing the training and removing the system, users are still able to keep the position and velocity error at 61% and 42% lower than their initial performance, respectively. This fact suggests that subjects are able to retain the motor skills obtained during training.