Introduction

In the past decade, the field of emergent many-body phenomena in solids has undergone a paradigm shift driven by the convergence of two major streams of quantum materials: band topology and electron-electron interaction1. In heavy transition metal compounds with comparable strength of spin-orbit coupling (SOC) and on-site Coulomb repulsion, the electron band topology is intertwined with magnetism, leading to novel correlated topological phases with spontaneous time-reversal symmetry breaking2,3,4. Pyrochlore iridates R2Ir2O7 (R = Y and rare-earth elements) are a representative model system that has received considerable attention for exploring the delicate balance between SOC, correlations, and the itinerancy of 5d electrons embedded on the geometrically frustrated pyrochlore lattice. From theoretical viewpoint, extremely rich phase diagrams of R2Ir2O7 have been constructed with a plethora of interesting correlated topological phenomena5,6,7,8,9,10,11,12,13,14,15,16. Recently, the link between topological properties and dimensionality has further inspired the predictions and search for emergent hidden quantum states that are only accessible in (111)-oriented thin films of pyrochlore iridates17,18,19,20,21,22,23,24,25,26,27.

In bulk, the high temperature “parent” phase of R2Ir2O7 is the Luttinger-Abrikosov-Beneslavskii (LAB) non-Fermi-liquid semimetal, with a quadratic band touching point located at the zone center28,29,30. At low temperatures, formation of magnetic long-range ordering (LRO) on the Ir sublattice can turn the LAB semimetal into various topological Weyl states28. The magnetic interactions between Ir4+ ions on the corner-sharing tetrahedral network are governed by two leading terms: antiferromagnetic (AFM) Heisenberg exchange and Dzyaloshinskii-Moriya (DM) interaction. The AFM interaction can induce strong frustration and lead to a highly degenerate spin-liquid ground state. In contrast, the DM interaction tends to lift the degeneracy and stabilize LRO with the peculiar all-in-all-out (AIAO) spin configuration31,32,33,34,35,36,37 [Fig. 1b]. Furthermore, the inclusion of pseudo-dipolar interaction, single-ion anisotropy, or the variation of rare-earth R ion rapidly expands the phase space, stabilizing twelve symmetry-allowed q = 0 spin configurations and their linear combinations as the plausible magnetic ground state10,11,16,38.

Fig. 1: Schematics of exotic magnetism on (111) Y2Ir2O7 lattice.
figure 1

a 3D pyrochlore framework constructed by Ir atoms. b Long-range ordering with the AIAO antiferromagnetic spin configuration on Ir sublattice. c (111) quasi-2D slab of pyrochlore lattice composed of alternating kagome and triangle atomic layers. J1 (J2) represents the Ir nearest-neighbor (next-nearest-neighbor) interaction within the kagome layers, while \({J}^{{\prime} }\) the nearest-neighbor interaction between kagome and triangle layers. d A possible snapshot of the chiral spin liquid configuration on (111) quasi-2D pyrochlore slab. The Ir spins pair up forming dimers covering the entire lattice. The two spin directions in one dimer can be either the same (red dimer: both towards or away from the center of tetrahedra) or opposite (blue dimer: one towards and one away from the center of tetrahedra).

Full size image

In the quasi-two-dimensional (quasi-2D) limit, however, the tetrahedral network can be transformed into a thin (111)-oriented slab composed of several kagome and triangle atomic layers. In this case, one can envision how the competition among multiple AFM exchanges may restore strong frustration and stabilize an exotic short-range order with finite scalar spin chirality39. Conceptually, this novel magnetic state is akin to a chiral spin liquid (CSL), where fluctuating spin dimers cover the entire pyrochlore slab and break the global time-reversal symmetry40,41. As such, it is interesting to explore the challenging question of the true magnetic ground state of pyrochlore iridates on the thin (111)-oriented slab.

For this purpose, we have developed a set of high-quality (111) Y2Ir2O7 films using the in-situ solid phase epitaxy method42. By means of resonant x-ray magnetic scattering techniques and magneto-transport measurements, we have discovered the formation of q = 0 AFM LRO in 100 nm (111) Y2Ir2O7 films at a Néel temperature near 140 K. Surprisingly, the LRO is absent in thin films (≤30 nm). In this quasi-2D limit, a novel CSL state with anomalous Hall effect (AHE) appears below a characteristic temperature around 135 K, stemming from the fluctuating non-coplanar spin textures with non-zero chirality. In sharp contrast to the dispersive magnon mode observed in bulk crystals and predicted from the linear spin-wave theory, the CSL state exhibits spin-gapped but dispersionless spectrum of magnetic excitations. Our findings highlight the previously unexplored role of dimensional confinement to stabilize emergent quantum states in thin slabs of pyrochlore iridates for the first time.

Results

It is generally recognized that upon lowering the temperature, all R2Ir2O7 except for R = Pr undergo a metal-to-insulator/semimetal transition concurrently with the AFM phase transition43,44,45,46,47. In particular, since bulk Y2Ir2O7 shows the highest Néel temperature ∼150 K and has no f-electron moments on the Y sublattice, it serves as an ideal playground to explore the intrinsic magnetism of the Ir4+ sublattice. Despite active experimental efforts, inconsistent results on the nature of the ground state have been reported. For instance, well-defined oscillations of single frequency were observed in muon spin resonance/relaxation spectra, indicating the formation of LRO48. But meanwhile, neutron powder diffraction data yielded no sign of magnetic peaks at low temperatures, attributed to high neutron absorption of Ir49. With the availability of high-quality single-crystalline films, we use the resonant X-ray diffraction (RXD) at the Ir L3 edge to address this fundamental question.

The AIAO spin structure is a q = 0 LRO, which triggers charge-forbidden reflections such as (0 0 4n+2) visible in the resonant magnetic scattering (RMS). However, due to the local trigonal distortion of the IrO6 octahedra, the anisotropic tensor susceptibility (ATS) scattering is also attributed to the (0 0 4n+2) reflections50. As seen in Fig. 2a, in 100 nm Y2Ir2O7, the magnetic Bragg signal is clearly observed at 5 K in the energy spectra of (0 0 10) reflection at peak A (E = 11.215 keV); the signal gradually diminishes to zero at 160 K. The hump feature at peak B (E = 11.22 keV) is a leakage of the ATS scattering into the σ−π channel, and its intensity barely varies with temperature (Supplementary Fig. 2). The temperature dependence of the (0 0 10) integrated intensity indicates the onset of magnetic phase transition near 140 K in 100 nm Y2Ir2O7 [Fig. 2c]. These observations are in good agreement with the reported bulk value, demonstrating the establishment of LRO on the Ir sublattice48. A power-law fit of the scattered intensities in the vicinity of transition transition, \(I\propto {M}^{2}\propto {(1-\frac{T}{{T}_{N}})}^{2\beta }\) yields the critical exponent β ≈ 0.30, falling into the 3D Ising universality class consistent with the AIAO spin configuration13.

Fig. 2: Magnetic phase transition in 100 nm and 30 nm (111) Y2Ir2O7 films.
figure 2

Resonance profiles of the magnetic (0 0 10) reflection near Ir L3 edge in the σ−π channel for (a) 100 nm and (b) 30 nm Y2Ir2O7 films. A special azimuthal alignment which includes setting the [110] axis of the Y2Ir2O7 films perpendicular to the scattering plane allows for effectively suppressing the ATS contribution and leaving the RMS signal dominant in the σ−π channel32. Peak A at 11.215 keV represents the RMS signal, which is significantly enhanced at 5 K in the long-range antiferromagnetic state. Peak B at 11.22 keV is due to the leakage of ATS that is dominant in the σ−σ channel. c Temperature dependence of the integrated intensity of magnetic (0 0 10) reflection for 100 nm (red) and 30 nm (blue) Y2Ir2O7 films. The transition temperature TN ≈ 140 K and the critical exponent β ≈ 0.30 of 100 nm sample are extracted from the data following the relationship \(I\propto {M}^{2}\propto {(1-\frac{T}{{T}_{N}})}^{2\beta }\). The error bars refer to the standard deviation of the integrated intensity calculated with a Gaussian function. d Temperature dependence of the resistivity of 100 nm (red) and 30 nm (blue) samples, which exhibit a “kink” behavior at around 145 K and 135 K, respectively. The black dashed lines are power-law fits (ρ ∝ T−α) of the data between 100 and 20 K, with the power index α ∼ 1.3 (1.4) for 100 nm (30 nm) sample.

Full size image

In stark contrast, direct examination of Fig. 2b immediately reveals that, other than the small leakage of ATS in the σ−π scattering channel, the RMS signal is no longer detectable in 30 nm Y2Ir2O7 in a wide range of temperatures down to 5 K! Moreover, the absence of the expected magnetic scattering signal is verified on the set of (0 0 4n+2) reflections. A potential sample quality issue is ruled out, as the charge scattering peaks allowed for the pyrochlore lattice and the presence of characteristic ATS contributions are all detected [see Supplementary Figs. 1 and 2 for more details]. On the other hand, the X-ray magnetic circular dichroism (XMCD) spectra at 4 K have shown that the averaged net magnetic moment per Ir4+ ion is ∼0.023(4) μB/Ir [Supplementary Fig. 3]. This is remarkably lower than the expected local moment of about 0.33 μB/Ir for the Jeff = 1/2 configuration49, indicating an unusual paramagnetic (PM) behavior in 30 nm Y2Ir2O7 even under 6 T magnetic field, where the local moments of Ir persistently fluctuate at low temperatures.

Next, we focus on the magneto-transport response of our (111)-oriented films. In bulk Y2Ir2O7, the PM to AFM transition manifests itself as a “kink” on the temperature-dependent resistivity curve due to the inclusion of additional scattering channels from the long-range ordered spins. As seen in Fig. 2d, both films exhibit a semiconducting behavior with the kink feature on the ρ(T) curve. The kink of 100 nm Y2Ir2O7 indeed appears near 145 K, coinciding with the Néel temperature TN obtained from our magnetic scattering data [Fig. 2d, upper panel]. Surprisingly, the kink is also present in 30 nm Y2Ir2O7 near T* ≈ 135 K [Fig. 2d, bottom panel], which signals the existence of a hidden order not captured by magnetic scattering.

The AHE is a direct and macroscopic probe of time-reversal symmetry in quantum antiferromagnets4. Our Hall transport data reveal the presence of distinct AHE at zero magnetic fields for both Y2Ir2O7 films, signifying that the time-reversal symmetry is spontaneously broken at low temperatures [Fig. 3a, b]. Specifically, the transverse resistivity ρxy of 100 nm Y2Ir2O7 displays a clear hysteresis loop right below the Néel temperature. For 30 nm Y2Ir2O7, despite the absence of LRO, non-zero spontaneous Hall resistivity starts to emerge at 125 K, slightly below the characteristic temperature T* obtained from the longitudinal ρxx(T) curve. It is also worth noting that the coercive fields of both samples increase rapidly as temperature decreases, making only a small portion of the loops visible [see “Methods” and Supplementary Figs. 5 and 6].

Fig. 3: Anomalous Hall effect.
figure 3

Evolution of the transverse magneto-resistivity ρxy measured at a set of temperatures across (a) TN of 100 nm and (b) T* of 30 nm sample. Each branch of the curve (in light or heavy color) is obtained after field cooling, with an black arrow showing the corresponding scan direction. For clarity, the center of curve at each temperature is offset by 5 μΩ ⋅ cm. c Temperature dependence of the spontaneous Hall conductivity σAHE of 100 nm and 30 nm Y2Ir2O7 films. d The scaling relationship between σAHE and σxx of Y2Ir2O7 films.

Full size image

In addition, both films exhibit a non-monotonic temperature dependence of spontaneous Hall conductivity σAHE [Fig. 3c]. For 100 nm Y2Ir2O7, the emergence of non-zero σAHE concurrently with the onset of the AIAO AFM order agrees with the theoretical prediction for the formation of Weyl semimetallic state in (111) pyrochlore iridate films17,18,20. Here, the incomplete cancellation of the Chern vectors over all Weyl pairs triggers an intrinsic AHE, as demonstrated recently in Eu2Ir2O7 films27. Notably, symmetry analysis based on magnetic group theory strictly enforces zero scalar spin chirality for the AIAO configuration51. The non-monotonic σAHE(T) behavior (i.e., σAHE first increases to the maximum and gradually decreases on cooling) in 100 nm Y2Ir2O7 thus reflects the evolution of the Berry curvatures in momentum-space, due to the motion of the Weyl nodes with decreasing temperatures15. Nevertheless, it is noteworthy that while in Eu2Ir2O7 film, σAHE tends to level off to a finite value, σAHE of Y2Ir2O7 is significantly suppressed as approaching the base temperature. This might indicate the tendency of annihilation of the Weyl pairs when reaching the edge of the Brillouin zone on approaching its ground state [Supplementary Fig. 7].

In contrast, because of the absence of LRO in 30 nm Y2Ir2O7, it precludes the formation of Weyl nodes in this film. Meanwhile, the minute averaged magnetization obtained from XMCD at the Ir L-edge under 6 T excludes the origin of AHE from aligned moments in a conventional ferromagnet or a spin glass [Supplementary Fig. 3]. These findings strongly imply the presence of a chiral spin-liquid-like state below 125 K, with σAHE directly measuring the scalar spin chirality averaged over all short-range non-coplanar configurations40.

Next, we explore the scaling relation between the longitudinal conductivity σxx and the σAHE. Conventionally, this relationship can be divided into three empirical regimes based on the value of σxx52,53. As depicted in Fig. 3d, both Y2Ir2O7 films belong to the bad-metal-hopping regime, where σAHE decreases as σxx decreases, following the universal scaling of \({\sigma }_{xy} \sim {\sigma }_{xx}^{1.6}\) at temperatures below 100 K. Although the fundamental mechanism for the hopping regime is beyond resent theoretical framework and remains an open issue, this scaling behavior is indicative of complicated interplay between the intrinsic (i.e., the Berry-phase mechanism) and extrinsic (e.g., the side-jump scattering, inelastic scattering events, and higher-order perturbative terms) contributions53.

In the presence of the CSL state in 30 nm Y2Ir2O7, one can also anticipate the breakdown of classical spin-wave theory, as the spin excitations may exhibit marked deviations from the magnon modes of the bulk. In single crystals of pyrochlore iridates with the AIAO magnetic ground state, dispersive magnon modes with a spin gap of 25 meV and 28 meV have been probed by resonant inelastic X-ray scattering (RIXS) in Sm2Ir2O733 and Eu2Ir2O754. Figure 4a demonstrates a set of RIXS spectra taken on 30 nm Y2Ir2O7 at 10 K, with the energy loss range below 300 meV taken along the Γ-K direction of the Brillouin zone. The overall spectrum is composed of several distinct contributions. These include a peak at zero energy due to elastic scattering (solid curve) and two other peaks next to the elastic line (shadowed curve and dashed curve). The mode associated with the shadowed peak has an energy close to the spin gap observed in other pyrochlore iridates and is interpreted as a magnetic excitation. The dashed peak represents a particle-hole continuum that is incoherent.

Fig. 4: Magnetic excitation in 30 nm Y2Ir2O7 film.
figure 4

a The stacking RIXS spectra along the Γ-K direction recorded at 10 K. The data (solid dots) are normalized with respect to the elastic scattering peak at 0 meV. Fitting of each spectrum yields the elastic background (solid curve), inelastic peak from the magnetic excitation (shadowed curve), and inelastic background continuum (dashed curve). b The extracted dispersion of magnetic excitation in 30 nm Y2Ir2O7 film along the Γ-K direction. The inset illustrates the Brillouin zone with notations for high-symmetry positions. For comparison, the dispersive magnon reported in Eu2Ir2O7 bulk crystal54 with the AIAO LRO is plotted together on the figure.

Full size image

The overall dispersion of the magnetic excitation of 30 nm Y2Ir2O7 is shown in Fig. 4b. As immediately seen, it exhibits a surprisingly dispersionless feature over a momentum range larger than half Brillouin zone along the Γ-K direction. This unusual behavior is in sharp contrast to the previously reported RIXS data on pyrochlore iridates with the AIAO LRO, where gapped and dispersive magnon modes have been observed in the energy range from 20 meV to 40 meV along this direction33,54. Such a dispersionless magnetic excitation observed in 30 nm Y2Ir2O7 necessitates a fundamentally different magnetic Hamiltonian in the quasi-2D case. While the exact form is unknown, the CSL state is plausibly stabilized due to the competition among multiple exchange interactions, melting the LRO and generating chiral spin textures imposed on the triangle and kagome structural units.

Discussion

The CSL state breaks the time-reversal symmetry, while remains magnetically disordered and entangled, fluctuating among the massively degenerate states40. For bulk pyrochlore iridates, the Ir twelve types of q = 0 magnetic basis vectors ψi are categorized into four irreducible representations Γ3,5,7,9 (Supplementary Fig. 8). Notably, to break the macroscopic time-reversal symmetry, only those six bases under Γ9 are allowed to generate nonvanishing scalar spin chirality based on the multipole cluster theory51. In this sense, the overall wave function of the CSL in 30 nm Y2Ir2O7 must include spin configurations containing these chiral bases, such as “two-in-two-out”, “three-in-one-out”, and/or other linearly combined ones, which is analogous to the Pr spin configurations in the metallic CSL state of Pr2Ir2O744.

On the other hand, theoretical calculations reveal that the spin configurations of pyrochlore iridates are located very close in energy, within several meV6,10. This, in turn, implies the possibility of interconversion among the magnetic states by tuning the magnitudes of direct and indirect hopping between the Ir atoms11,16,38,55. Based on this observation, we speculate that in approaching the quasi-2D limit, truncation of the pyrochlore lattice along the (111) orientation and enhanced fluctuations rescale the relative strength of hopping and magnetic interactions, leading to a degenerate ground-state manifold. Figure 1d provides one of the plausible snapshots of the CSL state in 30 nm Y2Ir2O7, where red and blue dimers denote the distribution of fluctuating short-range spin textures.

In summary, we unravel the abnormally “fragile” nature of the AIAO order and unlock a fresh region that has not been recognized on the global phase diagram of pyrochlore iridates. Moreover, the emergence of the CSL state enables new opportunities to explore other exotic quantum phases proposed for pyrochlore iridates, particularly quantum nematic55, axion insulator9, and unconventional superconductivity22. Future experiments at milli-Kelvin temperature and ultra-high magnetic field are essential to verify if the dynamic CSL state can persist or if the quantum-order-by-disorder mechanism eventually prevails and selects a static configuration.

Methods

Sample fabrication

The (111) oriented Y2Ir2O7 thin films were fabricated on 5 × 5 mm2 (111) yttria-doped ZrO2 (YSZ) substrates by pulsed laser deposition. Ir-rich phase-mixed ceramic targets (Y:Ir = 1:3) were ablated using a KrF excimer laser (λ = 248 nm, energy density ∼5 J/cm2) with a repetition rate of 10 Hz. The deposition was first carried out at a substrate temperature of 450 °C, under 100 mTorr atmosphere composed of Ar and O2 gases (partial pressure ratio, Ar:O2 = 10:1). Amorphous film with the proper stoichiometry was obtained from this stage. Then the film was post-annealed inside the chamber at 950 °C under 500 Torr atmosphere of pure O2 for 15–20 min, followed by cooling to room temperature. High-quality single crystalline film was eventually achieved.

Transport measurements

The electrical transport measurements were performed in a 9 T physical property measurement system (PPMS, Quantum Design) with the 4-point contact method. The magnetic field was applied along the [111] direction, and the current was driven along the [1\(\bar{1}\)0] direction. In particular, due to the extremely large coercivity, the magneto-transport at each temperature was measured in the following procedures. First, +9 T was applied at 150 K and the sample was field cooled to the desired temperature. The resistance was measured by sweeping field from +9 to −9 T. Then, sample was heated to 150 K (to quench any magnetic ordering), and cooled back to the same temperature, after which the resistance was measured again by sweeping field from −9 to +9 T. Symmetrization (anti-symmetrization) was further applied using these two branches on the longitudinal (transverse) resistance to achieve pure signal from the magnetoresistance (the Hall effect).

Resonant x-ray diffraction

The RXD experiments were performed at beamline 6-ID-B of the Advanced Photon Source in Argonne National Laboratory using a 6-circle diffractometer. The (111) Y2Ir2O7 films were mounted in a close-cycle displex such that the in-plane [\(\bar{1}\)10] direction was along the x-rays, and the [110] direction was aligned perpendicular to the scattering plane. The incident beam was linearly polarized perpendicular to the scattering plane (σ), with energy tuned around the Ir L3 absorption edge. A PG (008) analyzer was used to filter the polarization of the scattered x-rays, leading to two detection channels, σ−σ and σ−π. The signal of resonant magnetic scattering can be effectively probed in the σ−π channel.

X-ray magnetic circular dichroism spectra

The resonant XMCD experiments were performed at beamline 4-ID-D of the Advanced Photon Source at Argonne National Laboratory. X-ray absorption spectra around Ir L3 and L2 edges were collected with left- and right-circularly polarized beams at normal incidence (k // [111]) to obtain the XMCD spectra. To exclude any artifact, XMCD spectra were measured in both positive and negative external field. In case of large magnetic coercivity, the measurements were conducted in two ways for comparison. (1) XMCD was recorded after sample was +6 T field cooled to 4 K; then sample was heated to 150 K (to quench any magnetic ordering) and cooled back to 4 K with −6 T field, after which XMCD was recorded. (2) XMCD was recorded after sample was +6 T field cooled to 4 K. Then the magnetic field was directly swept to −6 T at 4 K, after which XMCD was recorded. The spectra obtained in both ways are almost identical.

Resonant inelastic x-ray scattering

RIXS measurements were taken at beamline 27-ID-B of Advanced Photon Source. The incoming photon energy was tuned to the Ir L3 pre-edge (11.215 keV) with 28 meV resolution (FWHM) to enhance the magnetic scattering cross-section. Measurements on thin films of several tens’ nm at such a hard x-ray facility imposed several constraints on the sample geometry: (a) grazing-incident geometry to guarantee the maximal footprint of x-ray beams on the film, enhancing the overall scattering signal; (b) 2θ ≈ 90° to minimize the elastic scattering signal and to purify the magnetic signal. Furthermore, the magnetic signal was enhanced by selecting a peak which was structurally forbidden but magnetically allowed. As a result, measurements were taken around the (5 12 −1) peak. To obtain the exact information of these peaks, the elastic line was fitted to a Voigt function with a resolution determined at Γ point. The inelastic peaks were fitted to anti-symmetrized Lorentzian line shapes of the form: \(S(Q,\omega )=\frac{A}{2\pi (1-exp(\frac{\omega }{{k}_{B}T}))}\gamma (\frac{1}{{(\omega -{\omega }_{0})}^{2}+{(\gamma /2)}^{2}}+\frac{1}{{(\omega+{\omega }_{0})}^{2}+{(\gamma /2)}^{2}})\), where A is the peak amplitutde, \(\frac{1}{1-exp(\frac{\omega }{{k}_{B}T})}\) is the Bose factor, γ is the peak width, and ω0 is the peak position.