Introduction

Broken time reversal symmetry (\({{{\mathcal{T}}}}\)) and its interplay with the spin-orbit coupling (SOC) result in the transverse flow of electrons generating an anomalous Hall voltage, a signature that distinguishes a ferromagnet (FM) from conventional antiferromagnets (AFM)1,2. However, it has been predicted that a certain class of AFMs with noncollinear spin textures can lead to an unusual type of anomalous Hall effect (AHE) that can exist even in the absence of a net magnetization due to spin-lattice coupling3,4. Whilst in noncollinear antiferromagnets (NC-AFM) it has been shown that their weak magnetic moment is insufficient to account for the large AHE5,6,7,8,9, further work to experimentally substantiate the fundamental mechanism responsible for the observed AHE signals is missing. This is because most studies of the AHE have been performed with the electrical measurements in plane while driving a magnetic field out of the plane, entangling the magnetization signal to any novel contribution to the AHE. In order to understand the origin of the AHE signal, one must go beyond the perpendicular field geometry and apply the field in different directions in space to identify the symmetry, and from this the mechanism leading to the AHE signal. Here we show the full dependence of the AHE in the NC-AFM Mn3Ni0.35Cu0.65N (Mn3NiCuN) when the magnetic field is swept not only out-of-plane but also in the plane. The in-plane field experimental configuration, by construction, does not allow for the conventional dipole (magnetization) component of the AHE signal to contribute. We can therefore show, using a variety of theoretical techniques, that at high fields the in-plane AHE comes purely from the octupole moment and, surprisingly, an additional topological Hall-like (THE) signal occurs at low fields, consistent with prior evidence of scalar chirality-driven transport in the same compound10. Our results demonstrate that in NC-AFMs one must expand beyond the dipole contribution to the AHE to explain their rich, coexisting orders. Such coexisting orders go beyond the magnetization dynamics achievable in conventional FMs and AFMs. Harnessing these coexisting orders may enable future exploration of antiferromagnetic transport phenomena. Details of the spin-orbit torque behavior in Mn3Ni0.35Cu0.65N are discussed in a recent complementary study10.

The antisymmetric AHE in the NC-AFM arises from the octupole moment produced by the noncollinear frustrated spin structure in the kagome plane3,11 which is in the (111) plane in the case of Mn3NiCuN, the material we are investigating in this work (Fig. 1a). This can be visualized as an emergent octupole moment11,12,13,14,15,16,17, \({\vec{Q}}_{{T}_{1g}}\) (the first term in Eq. (1)) that purely originates from the coplanar orientation of the spin texture lying in the kagome plane (Fig. 1a). In Mn3NiCuN, the direction of \({\vec{Q}}_{{T}_{1g}}\) points out of the kagome plane, along [111] direction. Upon expanding the AHE signal in the local spins, beyond the conventional ferromagnetic case, the octupole and topological Hall effects18,19,20,21 emerge as higher order contributions to the Hall effect in noncollinear compounds. Hence, the total contributions to the AHE are,

$${\sigma }_{xy}={\gamma }_{Oct}\,{\left[{\vec{Q}}_{{T}_{1g}}\right]}_{z}+{\gamma }_{dip}\,{\left[{\vec{M}}_{{T}_{1g}}\right]}_{z}+{\gamma }_{SSC}\,\left[{\sum}_{ijk}{\vec{S}}_{i}\cdot ({\vec{S}}_{j}\times {\vec{S}}_{k})\right],$$
(1)

where Eq. (1) is obtained by using representation theory to derive the irreducible representations of Mn3NiCuN as described in ‘2 Representation theory’ in the Supplemental Material. The second term of Eq. (1) represents the conventional AHE, proportional to the net magnetic dipole moment, \({\vec{M}}_{{T}_{1g}}\), as observed in a regular ferromagnet such as Fe1, and that can arise in our system due to the canting of the spins upon the application of an external magnetic field. Generally, in this type of NC-AFM22, including our system (see Supplementary Fig. 4a in ‘1.1 Sample characterization’ in the Supplemental Material), the induced moment is very small22 and therefore has a very small effect on the AHE.

Fig. 1: Octupole structure and field-induced spin configurations in a noncollinear antiferromagnet.
Fig. 1: Octupole structure and field-induced spin configurations in a noncollinear antiferromagnet.
Full size image

a Crystal structure showing Mn atoms (orange), Ni/Cu atoms (blue), and N atoms (off-white). Blue arrows denote the coplanar 120° spin arrangement in the (111) kagome plane. b Schematic representation of the eight symmetry-allowed octupole components, including the two out-of-plane projections labeled ϒ0 and \({\overline{\Upsilon }}_{0}\), aligned with the [111] and \([\overline{1}\,\overline{1}\,\overline{1}]\) axes, respectively. c Spin configurations and corresponding octupole vectors when magnetic fields are applied along selected crystallographic directions within the (111) plane. Each labeled direction matches an in-plane projection of an octupole component. The central configuration illustrates the case of an out-of-plane field probing the dominant octupole component. d Calculated evolution of the octupole vector \({\vec{Q}}_{{T}_{1g}}\) in spherical coordinates as the in-plane magnetic field is rotated. The out-of-plane projection displays a 120° periodicity, which determines the expected angular dependence of the AHE.

Equation (1) predicts that \({\vec{Q}}_{{T}_{1g}}\) produces the maximum AHE when the magnetic field is swept out of the plane while the electrical measurements are performed in the kagome plane. This arrangement is similar to the measurement of AHE in a regular ferromagnet (FM) where the direction of the applied electric current (Jxx), voltage measurement (Vxy) and magnetic moment (Mz) are orthogonal to each other (Vxy ∝ (JxxMz)). Alternatively, in NC-AFMs with a vanishing magnetic moment, \({\vec{Q}}_{{T}_{1g}}\) plays the role of a fictitious magnetization, akin to Mz in ferromagnets. Nonetheless our symmetry analysis reveals a fundamental difference between the octupole driven AHE and the conventional dipole-driven AHE: the magnetic octupole supports a specific finite AHE response when the external field is rotated in the plane of the electrical measurements, whereas the dipole-driven AHE remains unaffected. Thus, in plane fields have to be used to reveal this contribution.

The octupole vector \({\vec{Q}}_{{T}_{1g}}\) hosts a total of eight poles in Mn3NiCuN as shown with the gray vectors in Fig. 1b, two of which point out of the plane, along the [111] direction and six others have projections both in the plane and out of the plane with 120∘ in-plane rotational symmetry enforced by the crystal structure (space group \(Pm\overline{3}m\) (No. 221), point group \(m\overline{3}m\) (Oh)). It is evident that when an external field is applied out-of-plane, along the [111] axis, one expects to obtain an AHE from the combination of octupole (\({\vec{Q}}_{{T}_{1g}}\)) and dipole (\({\vec{M}}_{{T}_{1g}}\)) contributions. However, our symmetry analysis additionally shows (see ‘2.6 Approximate ground state’ in the Supplemental Material) that when the in-plane applied magnetic field is collinear to the in-plane projections of any one of the six components of these \({\vec{Q}}_{{T}_{1g}}\) vectors, it immediately couples to the octupoles by reorienting the spin-configuration into one of the equivalent (111) planes as shown in Fig. 1c. This coupling provides direct access to the out-of-plane projection of \({\vec{Q}}_{{T}_{1g}}\) which can lead to an AHE as per Eq. (1) when electrical measurements are performed in the plane. Due to this, one expects a 120∘ angular dependence of the measured AHE as theoretically calculated in Fig. 1d. We point out here that we plot all components of the \({\vec{Q}}_{{T}_{1g}}\) vector in Fig. 1d and find that its dependence on the magnetic field resembles that of the magnetization, \({\vec{M}}_{{T}_{1g}}\), which illustrates its role as a fictitious magnetization; however, the key difference is that the octupole produces finite contributions to the in-plane AHE, unlike \({\vec{M}}_{{T}_{1g}}\). This defines a distinct difference from the conventional (dipole-driven) AHE in a FM as the AHE is not allowed to occur when electrical measurements and applied magnetic field are coplanar.

In addition, our theoretical calculations suggest a third term in Eq. (1) that predicts a nontrivial Hall effect originating from the scalar chirality of the spin textures. Such a quantity was first discussed in high temperature superconductors23,24 when considering contributions to the Hall conductivity, whose underlying physics can similarly be attributed to the THE in a skyrmion18,19,20,21,25. This can be observed in the low field regime when the spins reorient, maximizing the scalar spin chirality.

In this work, we reveal the microscopic origin of the AHE in a NC-AFM by systematically probing how the response evolves when magnetic fields are applied both out of the plane and along crystallographically defined in-plane directions. This approach allows us to separate any residual dipole contribution from the intrinsic octupole-driven signal and to identify the characteristic three-fold rotational symmetry associated with the octupole order. We further uncover an additional low-field Hall-like contribution that emerges when the spins form noncoplanar textures with finite scalar chirality. By combining symmetry analysis, first-principles calculations and magnetotransport measurements, we map how these different orders coexist and dominate in distinct field regimes. Our results establish a clear experimental and theoretical framework for interpreting Hall transport in noncollinear antiferromagnets and highlight the multi-component magnetic orders that can be harnessed for future spintronic functionality.

Results

To test these theoretical predictions, Mn3NiCuN (111) thin film of 20 nm thickness are grown on an MgO (111) substrate, and then capped with a 3 nm thin Pt layer to prevent oxidation. Crystallographically aligned Mn3NiCuN growth on MgO was confirmed by X-ray diffraction measurements with Cu Kα radiation (see Supplementary Figs. 1–3 in the Supplemental Material). After the thin film growth, Hall bar devices were patterned on the (111) kagome plane as shown in Fig. 2g. The details of the sample preparation can be found in the method section and in reference22. First, we verify the presence of the \({\vec{Q}}_{{T}_{1g}}\) component’s contribution to the AHE signal by measuring the standard Hall effect in the kagome plane while sweeping the magnetic field out-of-plane field along the [111] direction (z-axis) at 100 K, well below the Néel temperature (TN ~ 200 K)22. The longitudinal resistivity of the film is ~150 μΩcm−1 at a temperature 100 K. Figure 2a shows the measured AHE data for the z-field sweep that can be fitted by using \({R}_{xy}={R}_{xy0}\tanh (a(B-{B}_{c}))\), where Rxy0 is the saturation amplitude, a is a scaling factor and Bc the coercive magnetic field strength. We also verify that this AHE signal disappears above the Néel temperature (see Supplementary Fig. 4b in ‘1.1 Sample characterization’ in the Supplemental Material). This result is consistent within the framework of the octupole moment (\({\vec{Q}}_{{T}_{1g}}\)) induced AHE (Eq. (1), first term), but additionally a small magnetic moment is also present in our system (see Supplementary Fig. 4a in ‘1.1 Sample characterization’ in the Supplemental Material), meaning that the magnetic dipole component (\({\vec{M}}_{{T}_{1g}}\)) also contributes to the AHE. The measured AHE is comparable to previous reports in Mn3NiCuN and Mn3NiN but lower in magnitude than other NC-AFMs such as Mn3Ge and Mn3Sn8,9,26,27.

Fig. 2: Angular evolution of octupole-driven and chirality-driven Hall responses.
Fig. 2: Angular evolution of octupole-driven and chirality-driven Hall responses.
Full size image

Transverse Hall resistance measured as a function of magnetic field applied (a) out of the kagome plane and (b, c) within the plane at two representative in-plane field angles. Black symbols show experimental data; green curves are fits to the combined model in Eq. (2), consisting of an octupole-driven term (red curves) and a scalar-chirality-driven term (blue curves). Small arrows indicate the direction of hysteresis loop shift. d–f Schematics of the magnetic-field directions corresponding to the measurements in (a–c). g Device geometry used for magnetotransport measurements, indicating crystallographic axes and the orientation of current and voltage lines. h Extracted amplitudes of the octupole-driven contribution \({R}_{xy}^{Oct}\) (red) and the scalar-chirality contribution \({R}_{xy}^{SSC}\) (black) as functions of in-plane angle. Both show an approximate 120∘ periodicity. Error bars represent the standard deviation of five repeated measurements. i Symmetry-based theoretical calculation of scalar spin chirality, displaying the same 120∘ periodicity but with an opposite sign compared to the octupole contribution.

Now we perform the measurements of transverse resistance on the (111) plane while sweeping the magnetic field in the plane along different in-plane crystallographic directions (Fig. 2b, c). We observe two important features: (1) AHE-like signals, representing a step in the measured resistance, and (2) topological Hall-like (THE) signals, which are additional features only appearing in the low field regime which we will discuss later. Our data can be fitted well to Eq. (2),

$${R}_{xy}(B)={R}_{xy}^{Oct}\tanh (a(B-{B}_{c}))+{R}_{xy}^{SSC}\exp (-c{(B-{B}_{0})}^{2}),$$
(2)

where \({R}_{xy}^{Oct}\) represents the maximum contribution coming from the octupole moment, and \({R}_{xy}^{SSC}\) the maximum contribution of the scalar spin chirality, B is the magnitude of the magnetic field, Bc is the coercivity, a is related to the slope of \({R}_{xy}^{Oct}\) switching, and c is related to the width of the scalar spin chirality signal.

The observation of AHE-like signal in such coplanar geometry is a nontrivial finding as all but the highest coplanar anisotropies will not manifest a dipole contribution of this form. As such coplanar anisotropies do not exist in this material, the observed signal implies that applied magnetic fields can access one of the components of \({\vec{Q}}_{{T}_{1g}}\) depending on the direction of the magnetic field-sweep with respect to the crystal axis (Fig. 1c, d). This is further evident following the red curve in Fig. 2h that exhibits a roughly 120∘ angular dependence, consistent with our theoretical predictions (Fig. 1). The error bars in the data points represent the standard deviation for five different measurements.

In these experiments surprising additional features appear at low field values which can be fitted by a Gaussian function that changes sign for positive and negative applied field cycling direction. Features like these are often attributed to THE as they occur in the presence of skyrmions whose nontrivial topology can induce an emergent effective magnetic field19,25. The spin textures in our system exhibit no such topology25, however it is possible that in the low field regime the spins rotate out of their coplanar orientation and maximize the scalar spin chirality (\({\vec{S}}_{i}\cdot ({\vec{S}}_{j}\times {\vec{S}}_{k})\))18 thereby producing THE-like signals as predicted by the third term in Eq. (1). A consequence of this is that for certain ϕ where the THE-like component is larger than the \({\vec{Q}}_{{T}_{1g}}\) component, Rxy switches before B changes polarity as shown in Fig. 2b for ϕ = −75∘. This would be thermodynamically prohibited if the AHE has contributions only from \({\vec{Q}}_{{T}_{1g}}\) or \({\vec{M}}_{{T}_{1g}}\). To verify the source of the additional contribution, we compute the components of the scalar spin chirality by parameterizing a free energy expression of the irreducible representations with first principles calculations and solve a Landau–Lifshitz–Gilbert equation to provide the dependence of the scalar spin chirality (\({\vec{S}}_{i}\cdot ({\vec{S}}_{j}\times {\vec{S}}_{k})\)) and octupole (\({\vec{Q}}_{{T}_{1g}}\)) moment as a function of in-plane magnetic field angle ϕ. The simulation suggests that the noncoplanar spin-textures are possible and predicts that both the octupole signal (arising from \({\vec{Q}}_{{T}_{1g}}\)) and the THE-like signal (arising from \({\vec{S}}_{i}\cdot ({\vec{S}}_{j}\times {\vec{S}}_{k})\)) possess an angular dependence with 120∘ rotational symmetry (Fig. 2i). This result is qualitatively consistent with the angular dependence of the both the signals in our experiment (Fig. 2h) complete with their opposing signs.

Discussion

Whilst attempts have been made to quantify the behavior of the complex spin structures present in NC-AFMs using simple pictures to explain the sign change, for example, in the AHE5,6,7,8,9,28,29,30,31,32,33,34,35,36, our work provides a full understanding of the signal of the AHE as a function of the magnetic field. Notable exceptions to this rule include Song et al.37, however they probed the symmetric planar Hall effect and not the antisymmetric AHE. Additionally work by Ghosh et al.38 provided an argument for an alternative origin for the THE-like signal in Nd2Ir2O7, however this origin suggests the presence of Weyl nodes arising from broken inversion symmetry which is not present in our system.

To rule out other sources of the measured signals, one needs to confirm the crystalline structure and orientation of the thin film. In particular, the orientation of the film’s crystal structure needs to be checked to rule out the effects of other phases or orientations. We find that our thin films exhibit alignment with both the OOP and in-plane directions of the substrate (see ‘1.1 Sample Characterization’ in the Supplemental Material). However, the non-negligible lattice mismatch between the substrate and Mn3NiCuN presents a situation where one does not expect a perfect epitaxial thin film alignment. While it is thus clear that qualitatively, the observed symmetries of the transport signal mirror the symmetry of the crystal structure with no other phases and orientations present, the small local variations of the crystallinity can result in the attenuation of the Rxy in comparison to a fully crystalline bulk sample, for instance. This is compounded by the current shunting effect due to the Pt layer on top, due to which a 20% underestimation of Rxy can be expected. However, this is merely a suppression of the overall signal and does not lead to any modification in the interpretation of the underlying effect. The Pt layer could potentially induce magnetoresistance effects at the interface due to spin accumulation and adds a degree of uncertainty into the interpretation of the data. However, the consistent 120∘ symmetry of both the AHE and SSC signal at two different devices and across a range of temperatures (See Supplementary Fig. 7 in ‘1.2 Data processing’ in the Supplemental Material) provides compelling evidence in support of our interpretation. To check the influence of Pt, additional measurements on a Mn3NiCuN thin film without a capping Pt layer were performed (see Supplementary Fig. 8 in ‘1.2 Data processing’ in the Supplemental Material). The \({R}_{xy}^{Oct}\) and \({R}_{xy}^{SSC}\) show analogous ϕ-dependence as for the stack shown in Fig. 2h showing that any potential effects of the Pt capping layer are negligible for the purpose of our work.

Our analysis of the underlying spin dynamics elucidates a clear interplay of coexisting orders within a NC-AFM, which sets this class of magnetic order apart from conventional ferromagnets and antiferromagnets. These collinear spin structures host only one order parameter apiece, the magnetization and the Néel vector, whereas NC-AFM exhibit further complexities that have direct consequences on the AHE that can only be understood through expansion in local spins beyond the ferromagnetic case. For Mn3NiCuN, and subsequently the whole class of Mn3XY compounds, one can uncover the coexistence of three distinct orders: the magnetization, a quantity in part responsible for the conventional AHE in this NC-AFM, the octupole contribution, that provides both conventional and also in-plane AHE contributions and finally the THE-like component generated when the spins are noncoplanar. Although this particular material is merely a prototypical example of the vast array of other NC-AFM orders, we present a very clear paradigm shift in the way these materials are to be viewed: that they have multiple unconventional order parameters which behave like a magnetization, yet they affect the Hall transport in unique ways. This procedure can also be applied to other classes of materials, such as Mn3X compounds, which have already shown similar in-plane AHE signatures5.

In summary, we systematically study the symmetry of the transverse Hall resistance in Mn3NiCuN (111) by applying magnetic fields in the kagome plane along selected crystallographic directions. We find that whenever the external applied magnetic field is collinear to the in-plane projections of any one of the octupoles, an AHE can be detected even in the limit of vanishing magnetization. The six octupoles that have in-plane components lead to a 120∘ symmetric AHE signal even when magnetic field and electrical measurements are coplanar, a remarkable contrast from the conventional AHE in FMs. We observe THE-like features and decompose the signal into octupole and THE-like components, finding the THE-like components to behave with a similar 120∘ angular dependence as the octupole component while sweeping the magnetic field in the kagome plane. We attribute this effect to a scalar-chirality emerging during the spin-reorientation at low magnetic fields. Our experimental results are well supported by theoretical predictions. Our work represents a key step in uncovering the complex, nontrivial phenomena in NC-AFMs, greatly enhancing the understanding of the octupole moments, and distilling the complex spin textures into clear, coexisting orders. This understanding opens the possibility to explore multiple means to harness NC-AFMs for novel spintronic technological applications.

Methods

Material and device fabrication

On MgO (111) substrate, 20 nm Mn3NiCuN thin film was deposited using reactive magnetron sputtering at 375 °C substrate temperature under 2.0 Pa with 4% N2 + 96% Ar gas mixtures. After growth, the sample was annealed in-situ at 500 °C under the same atmosphere as film growth. Please refer to ref. 22 for further details on thin film deposition. A 3 nm Pt layer was deposited in-situ to prevent oxidation. After the thin film growth, Hall bar devices with current line of width 10 μm and voltage line of width 3 μm were patterned using electron beam lithography, and the surrounding area removed using Ar+ ion etching.

Transport measurements

The sample with patterned Hall bars was mounted on a standard PCB with Au contacts contacted to the Hall bars by wire bonding using TPT Hybond 572-40. The PCB with contacted sample was then mounted on attocube ANRv51/RES piezo-controlled rotatable sample holder which was inserted in a variable temperature cryostat from Oxford Instruments to perform magneto-transport measurements. Keithley 2400 was used as a current source, and the Hall voltage was measured using Keithley 2182A nano-voltmeter.

Data processing

The acquired Hall signal was anti-symmetrized (Rxy(B) = (Vxy(B) − Vxy(− B)/2Ixx) to extract the AHE, and to remove any contributions from the longitudinal signal. The anomalous Hall signal was centered and the contribution linear with B corresponding to the ordinary Hall effect subtracted. The planar Hall effect was also analyzed and removed as in Supplementary Fig. 5 and is plotted in Supplementary Fig. 6 (see Supplemental Material).

Computational details

We performed Density Functional Theory calculations for bulk Mn3NiCuN using the experimental lattice constant 3.9012 Å from Zhao et al.22. We performed calculations using the FLEUR code (for the program description, see https://www.flapw.de) which implements the full potential linear augmented plane wave method (FP-LAPW)39, employing the generalized gradient approximation (GGA)40. For the symmetry analysis, full self-consistencies were obtained at multiple noncoplanar canting angles from the ϒ0 spin configuration in Fig. 1a. In order to quantify the AHE from first principles, we employed Wannier interpolation41,42 to efficiently compute the Berry curvature using the Kubo formalism1. Full details are provided in ‘3 Numerical details’ in the Supplemental Material.

LLM disclosure

Large language model assistance (ChatGPT) was used for language editing and formatting suggestions only. All scientific content, data interpretation, and conclusions are solely the authors’ work.