Introduction

The phase transition on cooling from a paramagnetic to a magnetically ordered state necessarily breaks certain symmetries of the system. The static nature of magnetic order implies breaking of the time-reversal symmetry, and most often some of the rotational and/or translational symmetries are also lost. Such phase transitions take place in the vast majority of magnetic materials with non-negligible interatomic magnetic interactions. Defects and disorder-free systems of interacting spins that do not exhibit long-range order down to zero temperature are called spin liquids, which seem to be exceptionally rare in nature1,2,3. Spin liquids are recognized to demonstrate unconventional behavior and exotic quasi-particle excitations4,5,6 stemming from their extensive degeneracy of the ground state. The magnetic order in spin liquids is precluded by strong frustration. Instead, they feature a strongly-correlated state that still preserves the spin rotational symmetries of the paramagnetic system down to zero temperature. The spin-liquid state is often realized on a phase boundary between two ordered states, described in terms of propagation vectors anchored to the high-symmetry points of the Brillouin zone (BZ) boundary. For instance, the theoretical studies of the quantum J1–J2 Heisenberg model on the triangular lattice7,8,9 revealed that the quantum spin liquid appears for a narrow parameter range between the 120° and stripe orders. However, recent works also showed that a highly-fluctuating state, akin to the quantum spin liquid, can also be formed in the classic regime at an arbitrary non-zero wavenumber. The latter was termed the spiral spin liquid (SSL) as the spins in such a state maintain a well-defined spiral pitch10,11,12,13,14,15,16.

Hence, the SSL is a classical spin liquid state, which lacks a long-range order but exhibits strong short-range correlations that retain the periodicity of the spin spiral. Unlike an ordinary spin spiral state, the propagation vector of the SSL spans a contour or a surface in reciprocal space, meaning that the magnetic structure is not characterized by a singled-out propagation vector but instead has a manifold of propagation vectors that define the continuous degeneracy of the ground-state. Spins in the SSL state exhibit collective fluctuations, so the SSL-hosting materials were proposed as a promising platform for the experimental realization of emergent excitations such as subdimentional fractons, which have restricted mobility and are associated with gauge fields14,15,17,18,19,20. Materials that were found to exhibit the SSL signatures so far are the honeycomb compound FeCl321, the pyrochlore ZnCr2Se222,23,24, the breathing-kagome lattice crystal Ca10Cr7O2825, and the diamond-lattice material MnSc2S426,27. In addition, the compound LiYbO2 with an elongated diamond lattice was proposed as an SSL based on the powder-averaged data28. The triangular lattice is, in turn, the prototype of a geometrically frustrated lattice, which was also predicted to host the SSL state29,30 yet, no experimental realizations of the SSL on a triangular lattice were reported so far.

In Fig. 1b we show the magnetic phase diagram of the triangular lattice with the J1 − J2 − J3 Heisenberg exchange interactions [Fig. 1(a)] with a ferromagnetic (FM) J1 (negative) and antiferromagnetic (AFM) J2 and J3 (both positive). To improve the visual clarity, the phase diagram is decorated by the color-coded magnitude of the propagation vector q, which depends on the relative ratio of exchange interactions, J2/J1 and J3/J1 (see Section S2 in Supplemental Materials31). In addition to the trivial FM order and commensurate \((\frac{1}{3}\,\frac{1}{3}\,0)\) AFM states, two spiral ground states can be realized: the phase II, where the propagation vector is aligned in reciprocal space as q = (ξ 0 0) [Fig. 1(c3)], and the phase III, for which the spiral propagates along an alternative direction q = (ξ ξ 0) [Fig. 1(c1)]. In other words, the phases II and III describe the same spin spiral but oriented along each of the two principal crystallographic directions. The most interesting part is the boundary between the two spiral phases defined by J2/J3 = 2, for which the two orientations of propagation wavevector become degenerate. At the critical line, the spiral propagation vector is no longer bound to the underlying lattice, and thus can arbitrarily rotate (but keeping its magnitude) on the two-dimensional plane [see the cartoon in Fig. 1(c2)]. This leads to the characteristic singularities in its structure factor directly probed by neutron scattering, namely, to the emergence of a continuous ring of scattering intensity in reciprocal space with the radius set by the same ratio of the exchange parameters J2/J3.

Fig. 1: The spiral spin liquid state in AgCrSe2.
figure 1

a Scheme of intralayer exchange interactions up to third-nearest neighbor. b Classical zero-temperature phase diagram in coordinates J2/∣J1∣ and J3/∣J1∣, the color represents the magnitude of the propagation vector30. (c1–c3) Schematic illustration of the cycloidal magnetic order in phase III (c1), phase II (c3) and the SSL state at the boundary between II and III (c2). Crystal structure (d) and Brillouin zone (e) of AgCrSe2. f Temperature dependency of heat capacity of AgCrSe235. (g1,g2) The SANS-I data measured in AgCrSe2 at two temperatures. Note that the measured data include q-points with h > − 0.04, which were symmetrized to visually illustrate the angular intensity distribution. (k1,k2) The classical energy calculated for exchange parameters corresponding to qm = (0.2 0.2 0) r.l.u. and J2/J3 = 1.7 as a function of the propagation vector of the spin cycloid. The filled part of the surface shows the level of thermal energy kBT. The colormap at the bottom shows the emergence probability of the spiral state with associated propagation vector in a form of \(\exp (-\Delta E/{k}_{{{\rm{B}}}}T)\).

Full size image

Here we demonstrate that AgCrSe2, a layered compound with the crystal structure similar to the delafossite family32,33,34,35,36,37, is a perfect realization of the triangular lattice with the nearest-neighbor FM exchange and the next-nearest-neighbor AFM exchange interactions. The material has a trigonal crystal structure, where the magnetic Cr3+ ions (S = 3/2) form triangular layers stacked in the ABC-fashion [Fig. 1(d)]. Unlike the delafossites, the Ag ions in AgCrSe2 occupy only one of two triangular sublattices, leading to a non-centrosymmetric R3m structure. Although the layers are weakly coupled by an AFM interaction along the trigonal c axis, the physics of each triangular layer governed by the phase diagram of Fig. 1(b) remains intact.

The bulk properties of AgCrSe2 are comprehensively discussed in a previous study35. While a broad maximum was observed in the magnetic susceptibility measurements, no signatures of any phase transition could be seen in the specific heat. Instead, the extracted magnetic contribution to the total specific heat is a very broad hump with a maximum centered at TX = 43 K, suggesting a build-up of strong magnetic fluctuations without long-range order35. On the other hand, powder neutron diffraction at 1.5 K showed clear magnetic Bragg peaks that are sharp in the 2θ angle, implying existence of magnetic correlations with a well-defined period. The Bragg peaks were indexed by an incommensurate propagation vector Q = (0.037 0.037 3/2)35 corresponding to the spiral spin correlations akin to the phase III of Fig. 1(b).

In this work, we resolved the existing controversies by employing single-crystal elastic neutron scattering, in which the magnetic intensities can by fully mapped out in 3D momentum space. We show that the anomalies at TX occur at a cross-over from a weakly-correlated paramagnetic state to the SSL. We further corroborate our experimental observations by spin-dynamics simulations that reproduce the SSL state and its neutron scattering intensity distribution in reciprocal space, including its behavior upon temperature variation and applied magnetic field.

Results

Observation of the SSL state

We begin presenting our single-crystal neutron diffraction results with a brief overview of the geometry in reciprocal space. Interacting Cr spins in AgCrSe2 reside on the vertices of the triangular lattice in the ab plane. The spins in individual triangular layers align strictly antiparallel due to an additional AFM exchange along c, leading to an out-of-plane component to the propagation vector, i.e. Q = qm + (00l), where l = 3/2. This shifts the magnetic reflections to the (H K 3/2) reciprocal plane of the first Brillouin zone (BZ), as depicted schematically in Fig. 1e.

The diffraction map collected at T = 40 K, just below the maximum of a broad hump in Cmag [Fig. 1(f)], displays a continuous ring of magnetic intensity at ∣qm∣, demonstrating that the ground state of AgCrSe2 at high temperature is indeed highly degenerate [Fig. 1(g2)]. It is important to note that the uniform ring retains its radial sharpness, indicating that the periodicity of the spiral correlations is well-defined (the radial broadening is limited by the instrumental resolution).

Having confirmed the formation of the SSL state at elevated temperatures, we now turn to the detailed analysis of the scattering intensity upon temperature variation. A series of diffraction patterns in the same reciprocal plane were taken at several temperatures between 1.5 and 37.5 K are shown in Fig. 2(a1–a5). One can notice that the scattering intensity undergoes a smooth transformation upon lowering the sample temperature. Namely, the initially isotropic intensity redistributes into a set of six broad maxima and minima, such that the 〈110〉 directions become favorable for the propagation of the spiral correlations. Because the anisotropy of scattering intensity occurs gradually upon cooling, it is in full agreement with the crossover behavior (as opposed to a phase transition) seen in the specific heat measurements [Fig. 1(f)]35. As thermal fluctuations diminish, the full SSL degeneracy is lifted, yet the system effectively lacks the long-range order even at T ≪ Tx as the spiral propagation is never locked into a singled-out crystallographic direction. We note, that a long-range order, if existed in AgCrSe2, would manifest itself as resolution-limited Bragg peaks.

Fig. 2: Neutron diffraction in AgCrSe2.
figure 2

a1–a5 The diffraction maps measured using DMC instrument at different temperatures, as indicated in each panel. b1–b5 The calculated structure factor. c1–c5 The real space spin configurations corresponding to the structure factors in (b1–b5) panels. The color represents the in-plane angle of the spins.

Full size image

The magnetic correlations preserve the spiral periodicity in the full temperature range below Tx, as can be seen by the resolution-limited radial width of the scattering intensity. To further quantity the spiral-orientation disorder in AgCrSe2 at low temperatures, we analyze its azimuthal profiles of intensity in detail. We described the azimuthal profiles using Lorentzian functions, whose full width at half maximum (FWHM) is proportional to the degree of disorder38.

Figure 3 (a) summarizes the angular profiles of intensity within the ring, which was plotted by rebinning the diffraction maps into polar coordinates. The maxima spaced by 60° were fitted with a Lorentzian function convoluted with the instrumental resolution (see Section S1 Supplemental Materials31 for the details on instrumental resolution). The FWHM of the Lorentzians exhibits no change between 2 and 30 K but increases significantly at higher temperatures [Fig. 3(b1)]. As was noted above, the intensity broadening takes place gradually and diverges in proximity to the crossover temperature, where separate Lorentzian profiles are no longer applicable. As can be seen, there is still a significant remnant azimuthal width of  ~10° (far surpassing the instrumental resolution) at 1.5 K, associated with a moderate directional disorder of the spirals even at T ≪ Tx. The reliability of the extracted FWHM can be confirmed by comparing the data collected on two different instruments with distinct resolution functions (see Methods and Supplemental Materials31). The extracted FWHMs from the two datasets are plotted along in Fig. 3(b1), confirming that the remnant broadening is an intrinsic property of AgCrSe2.

Fig. 3: Details of temperature evolution.
figure 3

a The angular dependency of the intensity (DMC), the color filled peaks are the fitted Lorenzian peaks convoluted with instrumental resolution. The bottom of each color filled peak shows applied offset. b1, b2 The obtained fit parameters for peaks in panel (a): The circles are the Lorentzian FWHM and the intensity extracted from DMC experiment data. The square points show the Lorentzian FWHM in SANS-I experiment, extracted similarly. c The susceptibility data measured with in-plane field of 1 T on cooling35. d1, d2 The calculated FWHM and the intensity. e The magnetization calculated in field of 0.02∣J1∣ applied along [110] direction.

Full size image

It is representative to compare the crossover behavior seen in azimuthal width with the integral intensity of the whole ring. As shown in Fig. 3(b2), the observed intensity displays a gradual decrease upon warming without any sharp anomalies and does not resemble a typical critical \(\sim \sqrt{T-{T}_{{{\rm{C}}}}}\) behavior that is characteristic of a second-order phase transition. The absence of sharp anomalies was also evident from the magnetic susceptibility measurements in Fig. 3c that show a broad maximum centered at Tx35.

It is worth mentioning that the observed SSL state may explain the recent observation of the anomalous Hall effect in AgCrSe237. The anomalous part of the transverse resistivity was maximized at the low temperatures but was also observed at temperatures well above TX without any sharp onset akin to a crossover. This suggest that the inhomogeneous spin texture of the SSL may have a nontrivial impact on the conduction electrons.

Modeling of the SSL state

To simulate the SSL state in AgCrSe2 we considered the J1 − J2 − J3 Heisenberg model with an FM J1 and an AFM J2 and J3. We also included a weak easy-plane anisotropy K that enforces coplanar spin texture with 〈S2z〉 = 0, which was deduced in the previous study35. To verify the adequacy of this approach, we first successfully reproduced the magnetic phase diagram of Fig. 2(b1) that was originally obtained by analytic equations30.

To reproduce the magnetic behavior of AgCrSe2 we chose the following model parameters: J2 = 0.33∣J1∣, J3 = 0.19∣J1∣ and K = 0.03∣J1∣. The ground state propagation vector for these parameters equals to (0.2 0.2 0) r.l.u., which is larger than the experimental value. Nevertheless, this parameter set allows us to capture all the general features of the SSL model at reasonable computation time. The larger propagation vector only leads to minor quantitative changes with respect to the experimental values.

The results of our simulations are presented in Fig. 2(b1–b5), where the calculated structure factors are compared to the experimental patterns. For a clear comparison, we convoluted the calculated patterns with the experimental resolution. The simulations were performed at temperatures given in the units of the first exchange J1, as it serves as the overall scaling of the total energy of the system. As one can see, the simulations correctly predict the anisotropic intensity distribution at low temperatures, where the azimuthal broadening shows minor variation from T = 0.01∣J1∣ to T = 0.45∣J1∣. The broadening is enhanced at higher T, and a fully isotropic ring of intensity is emerging at T = 0.65∣J1∣ in a good agreement with the experimental pattern at 37.5 K.

We further analyzed the simulated intensities in terms of Lorentzian FWHM and plotted it in Fig. 3(d1) along with the ring integral ring intensity in Fig. 3(d2). We can conclude that the crossover behavior expected for the SSL state is fully realized in the simulations, in excellent agreement with the experimental data. Moreover, the simulated temperature dependence of magnetization, M(T), closely resembles the experimental data, yet with a somewhat larger drop of the susceptibility at the lowest temperatures. The maximum of the simulated M(T) is found at T = 0.7∣J1∣. The simulations above 0.7∣J1∣ predict that the SSL state loses its radial correlations and smoothly transforms to what was previously termed a “pancake liquid” (see Section S3, S4 in Supplemental Materials31)15,16,39. Because the intensity of the pancake liquid state is very low, it becomes indistinguishable from instrumental background in our measurements.

It is worth noting that a minor feature identified in the magnetic susceptibility in the previous work35, namely a maximum in its temperature derivative for the in-plane fields at  ~ 32 K corresponds to an accelerated increase of the peak broadening accompanying a small drop of the intensity seen in both the experimental data in Fig. 3(b1, b2) and our simulations in Fig. 3(d1–d2) at T ≈ 0.55K∣J1∣, which further supports the chosen exchange parameters used for the modeling.

Having achieved a good agreement between the simulated and experimental intensities in reciprocal space, we can closely examine the real-space configurations of the spins that correspond to each structure factor. The magnetic texture in real space can be presented through the rotation of the classical spin in the ab-plane [Fig. 2(c1–c5)], as the Sz component is mostly zero across the sample plane due to the easy-plane anisotropy. At the lowest temperature [Fig. 2(c1)], the plane consists of two major areas of the cycloidal modulations, where the propagation vector retains its orientation over the distances exceeding 100 Å. The modulations in the two areas are continuously merged such that the spins of the same phase form hexagonal-shaped contours across the sample, as opposed to the distinct domains with domains walls characteristic of the ordered spiral magnets. As the temperature is elevated [Fig. 2(c2, c3)], the hexagonal-shaped spin stripes become more and more concentrically bent, such that the propagation vector is found in one position over the regions less than 100 Å. The cycloid orientation starts fluctuating already on the scale of 10 Å at T = 0.55∣J1∣ [Fig. 2(c4)]. Finally, the propagation direction is defined only on the scale under 10 Å at T = 0.65∣J1∣ [Fig. 2(c5)]. This state corresponds to the fully degenerate SSL as opposed to the states with partially lifted degeneracies at low temperatures.

Effect of external field

The SSL state features isotropic angular distribution of the intensity, preserving the rotational symmetry in ab-plane. By applying external in-plane magnetic field, one might break the rotation symmetry and potentially tune the SSL properties. However, the spins in AgCrSe2 are confined to the ab plane35, which makes the SSL propagation vectors insensitive to the external field direction (as soon as it is applied in plane). Indeed, the effective Zeeman field averages out to zero due to full spin winding in the ab plane. Contrary to this naive expectations, the SSL state in AgCrSe2 is found to respond by a significant redistribution of the intensity. A field applied along (110) at 40 K gradually suppresses the spiral correlations whose propagation vector is aligned with the field qm∥H, and favors the ones with the propagation vector being orthogonal to the field qm⊥H [Fig. 4(a1–a7)]. An apparent explanation for such behavior is that the out-of-plane spin component, which was previously precluded by the easy-plane anisotropy, becomes favorable in the applied field. Consequently, the spin correlations are no longer confined in the ab plane, and hence can become sensitive to the orientation of the in-plane field. This scenario is further supported by the previous magnetization measurements, where a spin-flop transition was observed at  ~5 T35.

Fig. 4: Effect of external field.
figure 4

a1–a7 The diffraction maps measured with SANS-I in different fields along [110] at 40 K. b1, b2 Comparison of simulated structure factors in absence and presence of external field applied along [11] direction (horizontal to the figure) for the model with easy plane anisotropy only. c1, c2 Simulations in the same conditions but with introduction of the DMI interaction into the model. d1, d2 The real space representation of simulated above structure factors. Each column corresponds to the (b1–c2) panel above. The top row shows the in-plane orientation of the magnetic moment as \(\arctan ({S}_{{{\rm{y}}}}/{S}_{{{\rm{x}}}})\), the color circle in-between panels indicates the angle-color correspondence. The bottom row shows the out-of-plane component Sz.

Full size image

In our simulations, the frustrated J1–J2–J3 model with an easy-plane anisotropy indeed does not show any rotation symmetry breaking in the presence of an external field [Fig. 4(b1, b2)]. The spiral correlations maintain their orientational disorder and the pitch, while the net magnetization is build because of a slight anharmonisity of the spin spiral modulation: the magnetic moments along the spiral propagation hold on for slightly longer at the “preferable angles”, as reflected in Fig. 4(d1) by the increased amount of red regions under the applied field. Out-of-plane spin components, as the result of an interplay between the spiral correlations and an external field, can be realized via the Dzyaloshinskii-Moriya interaction (DMI), which is allowed by the R3m symmetry of AgCrSe2. The in-plane DMI vector, which couples the nearest-neighbor spins (the J1 bond), is oriented perpendicular to the bond and was assumed to have the same magnitude as the easy-plane anisotropy, namely 0.03∣J1∣. In this configuration, the DMI vector favors spirals with the spin rotation plane within the ac-plane and the equilibrium orientation of the rotation plane is defined by the balance between the anisotropy and the DMI.

In the absence of a field, a model augmented with the DMI is characterized by the structure factor [Fig. 4(c1)] that is very similar to the one obtained in simulations with only the easy-plane anisotropy and no DMI. Because the DMI interaction is weak compared to the primary isotropic interactions, it does not affect the SSL behavior, nor breaks the rotational symmetry.

However, clear alteration in the Sz component of spins is evident in the real space images [Fig. 4(d2)]. The DMI causes the Sz component to closely follow the spin spiral propagation. This directly influences the system’s response to an external in-plane field: the spiral correlations propagating along the field direction are now disfavored as compared to the rest. The structure factor in applied field [Fig. 4(c2)] then well matches the experimental data [Fig. 4(a)], providing a clear evidence to the presence of DMI in the system.

It is worth noting that the neutron powder diffraction measurements35 indicated that the cycloidal modulations occur within a plane that is oriented at 89(7)° with respect to the c axis35, possibly slightly off the ab plane. This sets constraints on the possible strength of the DMI. The DMI is expected to be sufficiently weak not to overcome the easy-plane anisotropy at low temperatures in zero field, while still being strong enough to affect spiral propagation vector in the applied fields, as has been experimentally observed. However, an exact quantitative analysis of the DMI remains beyond the present study.

Discussion

It is important to note that the zero-temperature phase diagram [in Fig. 1(b)] predicts the SSL ground state exclusively at J2/J3 = 2, where the maximum level of frustration is realized. However, at finite temperatures, the SSL properties are not limited to the phase boundary of the two incommensurate states and effectively appear due to thermal fluctuations. Since the effect is mainly based on the frustration of exchange interaction, it virtually exists in the entire space of parameters compatible with an incommensurate spiral order. The temperature region where it is realistically observable heavily depends on the level of frustration, namely the proximity to the II/III phase boundary. Quantitative characteristics, such as the SSL temperature range extent and how quickly this regime develops, can change with the J2/J3 ratio and the magnitude of the propagation vector (for details see the Section S2 of Supplementary Materials31). However, the general feature of the SSL state appearance through a crossover is preserved. To further illustrate this, we calculated the Heisenberg classical energy in this model as a function of q, see Fig. 1(k1, k2). Clearly, the energy has a deep minimum at ∣qm∣, but exhibits only a small difference between q∥(110) and q∥(100), which provides an intuitive understanding for our neutron diffraction data and the spin-dynamics calculations at finite temperatures. Essentially, thermal fluctuations first destroy the long-range order by making any orientation of the spin spiral energetically equal, and only at much higher temperatures do they destroy the spiral correlations. A rough estimation of temperature smearing is \(\exp (-\Delta E/{k}_{{{\rm{B}}}}T)\), which gives a probability to find a spiral state with \(E={E}_{\min }+\Delta E\) in a system thermalized at temperature T. The corresponding colormap is shown in the lower part of Fig. 1(k1, k2).

The sizable easy-plane anisotropy in AgCrSe2 effectively restricts its spin dimensionality at low temperatures, which makes the XY spin model applicable for the low-energy physics of this material. Its ground state has the U(1)  × U(1) symmetry, where one U(1) describes the spin rotation, whereas another U(1) stems from the momentum rotation on the spiral ring. This contrasts with the O(3)  × U(1) symmetry of the Heisenberg model seemingly applicable to the SSL hosts FeCl321 and MnSc2S426,27. Because the SSL formation within the XY model has already been quite extensively discussed in the theoretical works12,13,14,15,16,17,18,19,20, AgCrSe2 becomes an attractive playground for testing those predictions. For example, Yan and Reuther15 presented a topological classification of the momentum defects formed in the SSL state. After a close look into the spin configurations stabilized in our simulations, we were able to conclude that the nontopological momentum vortices are indeed realized for the model parameters of AgCrSe2 (see Section 5 of Supplementary Materials31). However, the momentum vortices with a non-zero topological charge (as defined in15) did not appear in our simulations, which might be related to either their higher energy costs or the fact that the system is simulated on cooling from a high temperature where the XY model is no longer applicable, which may preclude the nucleation of the topological defects. Therefore, application of the emergent higher-rank gauge theory15 to AgCrSe2 remains an open question worth addressing in future studies.

To summarize, we report the observation of the SSL state by neutron scattering in a material with the perfect triangular lattice, and qualitatively reproduce our observations by the Landau-Lifshitz spin-dynamics simulations. The origin of the SSL, relying on thermal fluctuations and degeneracy, could be unraveled in terms of a specific energy landscape favoring orientational disorder at finite temperatures. The SSL behavior in external magnetic field suggests the presence of non-negligible DMI in AgCrSe2 providing a way of tuning the spiral correlations with an external in-plane field. Our study confirms the validity of the preceding theoretical proposals, and broadens the class of potential SSL-hosting compounds to include the triangular lattice materials, these being one of the most important model systems in the frustrated magnetism.

Methods

Sample preparation

A high-quality single crystal of AgCrSe2 with lateral dimensions of a few mm and mass  ~ 8 mg was grown by chemical vapor transport using chlorine as transport agent, as described in details in35. The sample composition and crystalline quality was characterized by energy- dispersive x-ray spectroscopy, wavelength-dispersive x-ray spectroscopy, x-ray powder diffraction, Laue x-ray diffraction, and differential scanning calorimetry. The bulk magnetic properties were characterized by DC and AC magnetic measurements and specific heat measurements reported in35.

Neutron scattering experiments

The neutron diffraction data were collected using the cold neutron diffractometer DMC and the small angle neutron scattering instrument SANS-I, both located in Paul Scherrer Institute (PSI), Switzerland.

In the DMC experiment, we used a PG(002) monochromator to select wavelength λ = 2.45 Å. The in-plane Q resolution of DMC is considerable better than the out-of-plane, which effectively elongates the Bragg peaks in the [\(\overline{1}10\)] direction (perpendicular to the scattering plane). The sample was placed in a standard orange cryostat and rotated around its vertical axis with a small angle step of 0.1°, such that a wide reciprocal-space volume was mapped out by the detector array covering 2θ of 127° in plane and  ± 7° out of plane. The obtained 3D dataset was then sliced in high-symmetry planes for further analysis.

The SANS-I experiment was conducted on the same sample and the same sample orientation with respect to the scattering plane. The magnetic field was applied a using a horizontal 6.8 T cryomagnet (opening angles 45°), with the magnetic field being applied along the incident neutron beam [close to the (\(\overline{1}10\)) reciprocal direction of the sample]. The sample and magnet were rotated around the vertical axis over 25° range in 0.5° steps to continuously span the momentum range of interest [the magnetic satellites in the (H K 3/2) plane]. The individual diffraction patterns collected at different sample angles by 2D detector of 0.96 × 0.96 m2 were combined into a 3D dataset. The 3D dataset was then sliced in the (H K 3/2) plane for further analysis. Due to the geometry of the SANS-I setup, the resolution ellipsoid is elongated along the (00L) reciprocal direction, whereas the resolution in the (HK0) plane (the plane of the SSL intensity) is significantly narrower and is approximately twice better than in the DMC measurements. For all the measurements, we subtracted the pattern collected at 50 K as a background, see Section S4 in the Supplementary Information31.

Numerical simulations

The classical spin-dynamics simulations were performed with the Landau-Lifshitz dynamics approach as implemented in SU(N)NY program package40. A single triangular lattice layer formed with up to 300 × 300 dipolar spins and periodic boundary conditions was considered. For a detailed description, see Section S2 in the Supplementary Information31.