A long periodic stacking "ordered" (LPSO) structures in a Mg-Y-Zn system is characterized by the periodical arrangement of a stacking fault (SF), which is locally fcc structure, introduced on the hcp lattice. In this study, we address the formation of LPSO structures in a Mg-Y-Zn system, especially the segregation of substituted atoms into SF site. We present thermodynamic properties of the Mg-Y-Zn ternary hcp and fcc phases at finite temperature have been investigated by means of first-principles calculations combined with the cluster variation method (CVM). The calculated results show that the Y and Zn atoms will segregate to the SFs, mainly because of the difference in chemical potentials for each element between fcc and hcp. We propose that the dominant factors in the formation of a novel long period stacking ordered structure include so-called Suzuki effect , as well as structure transformation from 2H to other structures having periodic stacking faults.