Effects of vibration information on the senses of presence and verisimilitude of audio-visual scenes

To enhance people's immersive experience from high-definition multimedia communications systems, one of the important sensory modalities is vibration. We investigated the effects of full-body vibration information on perceived reality from multimodal contents. Specifically, the relation between full-body vibration amplitude and the degree of perceived reality from audio-visual contents was measured, together with the changes of perceived reality when vibrational information was presented asynchronously against audio-visual information. To create multimodal content including vibration, we recorded sound, moving pictures, and ground vibration when a train passed just beside the measurement point. Observers' ratings of the senses of presence and verisimilitude were used as indicators of perceived reality. Results revealed that the sense of presence increased monotonically according to the full-body vibration amplitude, while the sense of verisimilitude exhibited a saturating curvilinear tendency. Moreover, the perceived subjective reality in terms of both senses decreased when full-body vibration was presented asynchronously against audio-visual information. Especially, the observed degradation was steeper when full-body vibration was presented after audio-visual information than when it was presented before audio-visual information. These results indicated clearly that perceived reality from multimodal contents can be increased by presenting full-body vibration of an appropriate amount and timing.