We perceive our environment by integrating multi-modal information. Although researchers have attempted to interpret sound perception solely through hearing, few attempts have been made using multi-modal information to understand our perception of environmental sounds. In this study, we conducted three experiments in which the sound stimuli were presented alone, with corresponding verbal information, or with corresponding visual information. In order to investigate the difference in the perception attributable to the verbal and visual information, analysis of variance was applied to the factor scores of each stimulus. Our analyses revealed that the effects of additional information on factor scores were different for verbal and visual information, except in the case of brightness. We suggest that the image intrinsic to a sound sources, evoked by verbal and visual information, affects the evaluation of sounds. Furthermore, moving pictures (visual information) contain a variety of information, whereas verbal information gives only determinative information. When this discrepancy produces appreciable differences between verbal and visual information, the changes in the factor scores can be rationally explained. Hence, we consider both the visibility and movement of the sound source to be important factors in our perception of environmental sounds.
|ジャーナル||Acta Acustica united with Acustica|
|出版ステータス||Published - 2006 1月 1|
ASJC Scopus subject areas