In this paper, an inverse approach based on the inherent strain method is proposed for constructing the residual stress field induced by welding. First, some smooth basis functions in form of polynomial have been employed to approximate the inherent strains. To select the basis functions properly, the previous valuable knowledge about the distributions of residual stress for some typical welding structures should be considered roundly. Furthermore, the singular modes in the assumed inherent strains, which do not cause the residual stresses, are excluded. Then, a stable profile of the inherent strain field can be assumed. Second, by employing the finite element method (FEM) and the least-square technique, the inherent strain field can be identified from limited experimental data at some key points. Finally, the distribution of residual stress can be constructed efficiently using the obtained inherent strain field. Compared with the traditional inherent strain method, in the present work, the sensitivity matrix for predicting the inherent strains can be evaluated more effectively and the experimental data needed in the identification procedure can be reduced significantly. A practical example is used to demonstrate the effectiveness of the present method.