Three-dimensional sound auralization systems, virtual auditory displays (VADs), have been developed actively in the last few decades. In conventional VADs based on headrelated transfer functions (HRTFs), a sound source position alone is rendered. However, because various sounds surround us in daily life, we usually hear not only a target direct sound but also ambient sounds in an actual sound space. A lack of ambient sound often engenders an unnatural perception of virtual auditory space presented by VAD based on HRTFs. Therefore we investigated an effective rendering method of ambient sound. Furthermore, using subjective evaluations, we discussed the relation between the realism of sound space with ambient sounds and a listener's head movement. This paper presents a review of the results.