In this paper, we propose a method to estimate the geometrical relationships between servo-brake-controlled caster units arbitrarily located on a moving base. The method utilizes only the velocity information of each caster unit and recursively estimates the pose of the caster units by the extended Kalman filter. In the state estimation problem, the estimation performance depends on the quality of the input signals. The input signals must contain sufficiently rich information to appropriately estimate the states. Since the system controlled by servo brakes is driven by a user-applied force, the user can arbitrarily select the input signals of velocity information. Therefore, we also suggest a simple pattern that enables users to provide sufficiently rich information for relationship estimates. Finally, the proposed method is validated in a computer simulation and an experiment using a mobile robot platform equipped with double-wheel caster units.