We can perceive a sound position from binaural signals using mainly head-related transfer functions (HRTFs). Using the theorem presented herein, we can display a sound image to a specific position in virtual auditory space by HRTFs. However, HRTF is defined commonly in a free-field, and a virtual sound image is perceived as a dry source without reflection, reverberation, or ambient noise. Therefore, the virtual sound space might be unnatural. The authors developed a software-based virtual auditory display (VAD) that outputs audio signals for a set of headphones with a three-dimensional position sensor. The VAD software can display a dynamic virtual auditory space that is responsive to a listener's head movement. Subjective evaluations were conducted to clarify the relation between the perceived reality of virtual sound space and ambient sound. Evaluation results of the reality of the virtual sound space displayed by the VAD software are introduced.