This paper proposes an accurate ultrasound probe localization method for 3D US image reconstruction. The proposed method consists of (i) feature tracking of a video sequence and (ii) camera pose estimation using structure from motion (SfM). SfM is used to reconstruct 3D point clouds from multiple-view images and simultaneously estimate each camera position. To apply SfM to a video sequence, the accurate method is required to track features between adjacent frames. We employ a sub-pixel image matching technique using Phase-Only Correlation (POC) for feature tracking. POC is a technique of image matching using phase components in the Fourier transforms of images. Through a set of experiments, we demonstrate that the proposed method can estimate the location of the US probe with about 2mm error for the probe travel distance of 150-200mm.