Novel geometrical frustration effects in the two-dimensional triangular-lattice antiferromagnet NiGa₂S₄ and related compounds

Geometrical frustration may suppress conventional magnetic long-range order and possibly promote a novel type of ordering associated with a higher order degree of freedom than spin dipolar-moment such as vector spin chirality and spin quadrupole moment. After a brief overview of such phenomena due to vector spin chirality, we focus on the two-dimensional (2D) frustrated magnetism in the layered chalcogenide Mott insulator NiGa₂S₄ and related compounds. NiGa₂S₄ provides the unique example of a S = 1 2D antiferromagnet on a regular exact triangular lattice. Extensive studies using high-purity samples of NiGa₂S₄ have revealed that Ni²⁺ S = 1 Heisenberg spins exhibit resonant critical slowing down at T^*= 8:5K without forming a magnetic long-range order, signaling a viscous spin-liquid state. The critical spin-fluctuation regime extends over almost an order of magnitude in temperature both above and below T^* . Even well below T^*, the spin-spin correlation remains short-ranged at an incommensurate wave vector close to (1/6, 1/6, 0), corresponding to a 120° correlation with 2a period. Interestingly, however, a 2D linearly dispersive magnetic mode and a quasi-static spin component exist in the low temperature limit. Possibly relevant scenarios including topological phase transition associated with Z₂ vortex due to vector spin chirality, spin quadrupolar nematic correlation, and C ₃ bond-ordering are reviewed.