We propose a novel background subtraction method for robust region extraction of moving objects in the dynamic background. In our method, a set of recently observed frame (reference) images is used as a background model. To withstand the constant fluctuations of background appearance, a current frame (input) image is compared with every reference image, and pixels in the input image deviated from those in the reference images are extracted as moving object regions. Such a simply-structured background model enables our method to adaptively vary the size of a spatial block for comparing the input image with the reference images. Setting an appropriate size block for each subject pixel in the input image, our method increases the stability of image comparison, and then improves the robustness of region extraction. Experimental results indicate that our method outperforms the existing methods in the region extraction accuracy of moving objects in the dynamic background.