This paper presents the analysis of a prototype scanning electrostatic force microscope (SEFM) system developed for noncontact surface profile measurement. In the SEFM system, with a dual height method, the distance between the probe tip and the sample surface can be accurately obtained through removing the influence of the electric field distribution on the sample surface. Since the electrostatic force is greatly influenced by the capacitance between the probe tip and the sample surface, a new approach for modeling and analysis of the distribution of capacitance between the probe tip with an arbitrary shape and the sample surface with a random topography by using the finite difference method (FDM) is proposed. The electrostatic forces calculated by the FDM method and the conventional sphere-plane model are compared to verify the validity of the FDM method. The frequency shift values measured by experiment are also compared with the simulation results computed by the FDM method. It has been demonstrated that the electrostatic force between arbitrary shapes of the probe tip and the sample surface can be well calculated by the finite difference method.