We study the effects of double quadrupole and octupole phonon excitations of 144Sm nucleus on heavy-ion fusion reaction and large angle quasi-elastic scattering for 16O+144Sm reaction using the coupled-channels approach. We explicitly taken into account the anharmonicites of nuclear vibrations using the sdf-interacting boson model. It is shown that the anhamronicities play an essential role in reproducing the experimental data of the fusion cross section as well as the fusion barrier distribution for this system. Also the quasi-elastic cross section is well reproduced in this way. However, the quasi-elastic barrier distribution has a high distinct peak which is smeared out in the experimental data. Our study indicates that the fusion and quasi-elastic barrier distribution for 16O+144Sm system cannot be accounted for simultaneously with the standard coupled-channels formalism.