Strength of tempered martensite is greatly affected by precipitation of fine carbides. Refinement of iron carbides becomes more important for dispersion hardening from view points of easy recyclability, reduction of the environmental load and saving metal resources. In this paper, the effects of addition of alloying elements to Fe-C binary alloy and rapid heating tempering on the precipitation of iron carbide in tempering of martensite at various temperatures are discussed Softening by tempering is retarded by addition of substitutional alloying elements because coarsening of cementite is suppressed due to alloy partitioning between ferrite and cementite. Retardation of softening is significant at higher tempering temperature in Mn and Cr added alloys whereas Si and Al addition is effective in retardation of softening in a wide range of tempering temperature. Microanalysis using energy dispersive x-ray spectroscopy and 3D atom probe microscopy has revealed that Si is rejected from cementite into ferrite quickly during tempering. Cementite is formed without partitioning of Mn at an early stage of precipitation but is gradually enriched with Mn in prolonged tempering. Application of rapid-heating tempering leads to finer and more uniform distribution of cementite. Especially, precipitation of coarse cementite at high-angle boundaries in lath martensite structure can be suppressed, resulting in improving the balance of strength and ductility. It is suggested that significant refinement can be achieved in combination of rapid-heating tempering and Si addition.