Introduction

Quantum anomalous Hall (QAH) states, as the typical two-dimensional topological insulating state, exhibit vanishing longitudinal resistance, and quantized Hall resistance in the absence of an external magnetic field1, in which dissipationless chiral edge states encircling QAH insulators (QAHIs) are anticipated to be advantageous in advancing low-power electronics. There have been several experimental observations of QAHIs with C = 1 at low temperatures, such as magnetically doped topological insulator (TI) films and twist moiré systems2,3,4,5,6,7,8,9. Although chiral edge current exists in QAHIs with C = 1, the presence of contact resistance still constrains these proof-of-concept devices to have limited breakdown currents. High-Chern-number QAHIs provide more nondissipative chiral current channels and decrease the effective Hall resistance of h/Ce2, facilitating the practical applications of the QAH devices. Despite there have been a series of theoretical predictions on high-Chern-number QAHIs in magnetically doped TIs under strong exchange field10 and electric field11, heterostructures spaced by additional buffer layers12,13, quantized Hall conductance has predominantly been observed in magnetically doped topological insulator films14, moiré systems15,16, and spin-orbit proximitized rhombohedral graphene17. Given that these studies are still restricted to a narrow range of material systems, the exploration and engineering of additional high-Chern-number QAHIs, particularly in realistic materials, are essential for expanding the scope of this field18.

Meanwhile, extensive theoretical predictions19,20,21 and experimental fabrication22,23,24,25 have identified MnBi2Te4 (MBT) as an excellent candidate for exploring the fascinating interplay among spatial dimension, topology and magnetism7,19,21,26,27. This material has demonstrated a wide range of extraordinary physical phenomena, including mysterious Dirac cones at the top surface28,29, QAH insulators7, axion insulating states27,30,and high-Chern-number Chern insulators under external magnetic field31. Notably, as a typical van der Waals material, interlayer coupling strength in MBT multi-layer thin films undergoes significant variations under distinct stacking orders, even giving rise to stacking-order dependent topology32,33,34,35,36,37. Given the unique properties of MBT and the pressing demand for high-Chern-number QAH insulators, further exploration of MBT and its derivatives holds great promise for overcoming current limitations and advancing the field of quantum topological materials.

The realization of QAHIs with C > 1 can be achieved through two distinct strategies: the design of novel functional two-dimensional (2D) materials and the construction of heterostructures by stacking van der Waals (vdW) layers from bulk counterparts. Among these approaches, the most convenient way is to cut slabs from three-dimensional (3D) Chern insulators or Weyl semimetals (WSMs) with the vdW nature. Notably, Chern numbers can be defined on all closed 2D manifolds, such as 2D Brillouin zone and the sphere surrounding Weyl points. As shown in Fig. 1a, the simplest 3D Chern insulator can be regarded as the periodic pattern of 2D Chern insulators in the stacking direction with negligible interlayer coupling due to the existence of buffer layer. This 3D Chern insulator possesses consistent C = 1 on these kz plane (Fig. 1b), but Chern number in WSMs abruptly jumps when kz planes cross the Weyl point. Chern numbers of slabs cut from 3D Chern insulators can exhibit a linear increase as the number of layers is incremented, and band gaps of thin films almost keep constant (Fig. 1c). Conversely, the Chern number of thin films derived from WSMs shows a step-like increase, and exhibits an oscillating gap with an overall decreasing trend as the number of layers increases. Nevertheless, the materialization of 3D vdW Chern insulators, which inherently possess the vdW characteristics, remains elusive. This significant gap impedes both the material realization and experimental observation of QAH insulators with high Chern numbers.

Fig. 1: Schematic illustration of electronic and topological properties in the bulk and slab from typical magnetic Weyl semimetals (WSMs) and 3D Chern insulators(CIs).
Fig. 1: Schematic illustration of electronic and topological properties in the bulk and slab from typical magnetic Weyl semimetals (WSMs) and 3D Chern insulators(CIs).
Full size image

a Chern number defined on a series of kz planes. b Chern numbers and (c) band gaps of slabs cut from bulk WSM and 3D CI as a function of layer number (N). d Typical 3D CIs built from periodic pattern of 2D Chern insulators with buffer layers along the stacking direction.

In the work, we demonstrate that MBT thin films with mixed stacking orders can achieve arbitrarily large Chern numbers without introducing additional buffer layer (Fig. 1(d)), which originates from varied interlayer coupling present in the different stacking configurations. Within these QAHIs, local anomalous Hall conductivity (AHC) is found to be concentrated in the consecutive correct stacking order, while the contribution from the consecutive incorrect stacking order is negligible. Furthermore, the bulk gaps in these QAHIs can be tuned to be over 20 meV by increasing layer numbers of both correct and incorrect stacking orders, through utilizing first-principles calculations and tight-binding effective Hamiltonians.

Results

First-principles calculations

Bulk MBT crystalline in the \(R\bar{3}m\) structure, which is characterized by the ABC-stacked building blocks in the Te-Bi-Te-Mn-Te-Bi-Te sequence19,22,23,24,25. Interlayer sliding can be realized in MBT thin films due to weak vdW interactions between neighboring SLs, allowing for different stacking orders, such as AB and AC high-symmetry stacking order. The most stable stacking order is AB-stacking, which exactly corresponds to the normal bulk form, while AC-stacking is energetically higher by 4 meV per atom, compared to AB-stacking. Dynamical stability is validated by phonon dispersion without imaginary phonon modes38. Therefore, thick films with mixed stacking orders are expected to be dynamically stable, and can be realized in the experiments.

In the AB-stacked bulk, the interlayer coupling is more pronounced due to the reduced interlayer spacing of 2.8 Å, in contrast to 3.2 Å observed in the AC-stacking. Stronger interlayer coupling, combined with significant spin-orbit coupling from the Bi and Te atoms, leads to the emergence of a topological band inversion at Γ in the AB-stacked configuration. Consequently, AB-stacked AFM bulk belongs to antiferromagnetic (AFM) TIs with Z2 = 119,20,21, and the AC-stacked AFM bulk falls in a topological trivial insulating case with Z2 = 0, elucidating the strong coupling in stacking order and topology in MBT materials36. When MBT bulks is exfoliated into films, the consecutive ABC-stacking order exhibit odd-even oscillation between QAH insulators and axion insulators19,21, while their counterparts with the consecutive CBA-stacking order persist as unequivocal trivial magnetic insulators without any topological feature. Below, both stacking orders are simultaneously employed to fabricate QAHIs endowed with large Chern numbers (C > 1), where the consecutive ABC stacking order is utilized to form QAH insulating layers with C = 1, and the CBA stacking order is employed as the buffer layer with C = 0.

As a representative example, let us consider 15-SL films with the ABCABACBACBCABC stacking sequence, shown in the inset of Fig. 2a. According to varied interlayer coupling strength, the entire thin film can be divided into consecutive correct stacking layers with the ...ABC... stacking order and consecutive incorrect stacking layers with the ...CBA... stacking order. Thus, the above 15-SL film can be divided into the ABCAB-ACBAC-BCABC stacking pattern, represented as (5, −5, 5). This representation indicates that this 15-SL film begins with five consecutive correct stacking layers (ABC), followed by five consecutive incorrect stacking layers (CBA), and ends with another set of five consecutive correct stacking layers (ABC). This alternating sequence of consecutive correct and incorrect stacking layers is essential for inducing large Chern number QAH insulators.

Fig. 2: Electronic and topological properties of 15-SL MBT thin films with stacking order in the ABCABACBACBCABC sequence.
Fig. 2: Electronic and topological properties of 15-SL MBT thin films with stacking order in the ABCABACBACBCABC sequence.
Full size image

a Electronic band structures along high symmetry line M − Γ − K. b Distribution of Berry curvatures in the Brillouin zone. c Local density of states on the [110] edge.

Electronic band structures of the (5, − 5, 5) stacking configuration along these high-symmetry lines M − Γ − K are shown in Fig. 2a, where a direct band gap of 18 meV is located at Γ. Furthermore, the gapped Dirac cone observed in this band structure resembles the gapped Dirac cones on the top surface of ABC-stacked MBT bulk, implying that films inherit crucial electronic and topological properties of AFM TIs, despite their reduced dimensionality. The k-resolved Berry curvature in Fig. 2b shows that Berry curvatures are mainly concentrated around Γ, and exhibit sixfold rotational symmetry, guaranteed by the presence of spatial inversion symmetry \({\mathcal{I}}\) and three-fold rotational symmetry C3z. The sum of Berry curvature in the whole Brillouin zone confirms that (5, -5, 5) film possesses C = 2. Furthermore, the calculated local density of states on the semi-infinite ribbon in Fig. 2c clearly reveals the presence of two chiral edge modes that transverse the bulk band gap.

To further illustrate the layer-resolved AHC in 15-SL MBT films, a numerical approach based on local basis sets has been implemented to calculate the real-space Chern-Simons contribution to AHC39,40,41,

$$C(\alpha )=-4\pi {\rm{Im}}{\int}_{{\rm{BZ}}}d{\bf{k}}\sum _{v{v}^{{\prime} }c}{\psi }_{v{\bf{k}}}^{\alpha }{{\bf{X}}}_{vc{\bf{k}}}{{\bf{Y}}}_{{v}^{{\prime} }c{\bf{k}}}^{\dagger }{\psi }_{{v}^{{\prime} }{\bf{k}}}^{\alpha * },$$
(1)

where

$${{\bf{X}}}_{vc{\bf{k}}}=\left\langle {\psi }_{v{\bf{k}}}\right\vert \hat{x}\left\vert {\psi }_{c{\bf{k}}}\right\rangle =\frac{\left\langle {\psi }_{v{\bf{k}}}\right\vert i\hslash {\hat{v}}_{x}\left\vert {\psi }_{c{\bf{k}}}\right\rangle }{{E}_{c{\bf{k}}}-{E}_{v{\bf{k}}}},$$
(2)

and Yvck take the similar form. Here \(\left\vert {\psi }_{v{\bf{k}}}\right\rangle\) (Evk) and \(\left\vert {\psi }_{c{\bf{k}}}\right\rangle\) (Eck) represent eigenstates (eigenvalues) in the valence and conduction bands with k = (kx, ky) traversing the entire Brillouin zone, respectively. \({\hat{v}}_{x}\) and \({\hat{v}}_{y}\) are the velocity operator defined as \(\frac{1}{\hslash }\frac{\partial \hat{H}}{\partial {\bf{k}}}\), and the symbol α denotes local basis set. And layer-dependent Chern number C(l) can be defined as the sum of real-space AHC terms of basis sets located with this layer (l),

$$C(l)=\sum _{\alpha }C(\alpha ).$$
(3)

Comparison between effective Hamiltonians and tight-binding models

Based on the above equation, detailed calculations have been performed to determine the layer-resolved AHC in (5, -5, 5) thin films, as shown in Fig. 3a. The oscillation of local AHC between neighboring SLs in Fig. 3b is attributed to an effective Zeeman field induced by opposite magnetic moments located on Mn atoms, originating from the A-type antiferromagnetic configuration. The top and bottom SLs of the consecutive correct stacking layers exhibit the largest and nearly half-quantized AHC (Fig. 3b), which is indeed the signature of half-quantized AHC on the top surface in the ABC-stacking bulk phase. The cumulative sum of layer-resolved AHC is presented in Fig. 3c, and its increment reveals noticeable discrepancy across different stacking regions. Notably, the local AHC is predominantly concentrated in the top and bottom regions consisting of consecutive correct stacking order. In contrast, the middle region, which consists of incorrect stacking layers, contributes little to the overall AHC. When summing the contributions of AHC from these three consecutive stacking regions, we observe that both top and bottom consecutive regions exhibit a quasi-quantized AHC behavior (Fig. 3d). The layer-resolved contribution to total AHC not only provides valuable insights into the underlying physics of topological electronic states, but also is essential for unraveling the mechanisms that govern AHC and for optimizing the performance of QAH insulators.

Fig. 3: Local anomalous Hall conductivity (AHC) in the 15-SL MBT thin films with ABCABACBACBCABC (5, -5, 5) stacking order.
Fig. 3: Local anomalous Hall conductivity (AHC) in the 15-SL MBT thin films with ABCABACBACBCABC (5, -5, 5) stacking order.
Full size image

a Schematic illustration of its crystal structure, with each consecutive stacking order marked by colored shades. b Calculated local AHC of each layer C(N) and c the cumulative sum of local AHC \({C}_{{\rm{int}}}(N)=\mathop{\sum }\nolimits_{i = 1}^{N}C(i)\). d The sum of local AHC in the consecutive correct (incorrect) stacking order as a function of chemical potential shift (EF), where the top, middle and bottom 5-SL are denoted by bot, mid and top, respectively.

To provide a quantitatively good description of electronic and topological properties in multi-SL MBT films of mixed AB- and AC-stacking orders, a low-energy effective model Hamiltonian is constructed by adding the onsite Zeeman term into the widely adopted Bernevig-Hughes-Zhang (BHZ) model42,43,44. Therefore, the effective model Hamiltonians, that describe varied interlayer coupling in distinct stacking configurations, can be written as

$$\begin{array}{lll}H({{\bf{k}}}_{\parallel })=\sum _{{\bf{k}},ij}\left[({H}_{0}({\bf{k}})+{H}_{Z}){\delta }_{i,j}\right.\\ \\\qquad\quad+\left.{H}_{+}({\bf{k}}){\delta }_{i,j-1}+{H}_{-}({\bf{k}}){\delta }_{i,j+1}\right],\end{array}$$
(4)

where i denotes the ith SL in the films, H0, H+ (H−), and HZ represent the intralayer Hamiltonian, interlayer coupling and onsite Zeeman term, respectively. τ and σ operate on the orbital and spin degrees of freedom. The latter two terms are assumed to be dependent only on stacking order. Following the effective \(\overrightarrow{{\bf{k}}}\cdot \overrightarrow{{\bf{p}}}\) Hamiltonians based on four low-lying state \(\left\vert {{\rm{Bi}}}_{z}^{+},\uparrow (\downarrow )\right\rangle\) and \(\left\vert {{\rm{Te}}}_{z}^{-},\uparrow (\downarrow )\right\rangle\), the intralayer and interlayer Hamiltonians can be written as

$$\begin{array}{lll}{H}_{0}=C{\tau }_{0}{\sigma }_{0}+M{\tau }_{z}{\sigma }_{0}+A({k}_{x}{\tau }_{x}{\sigma }_{x}+{k}_{y}{\tau }_{x}{\sigma }_{y}),\\ {H}_{+}=-{C}_{1}{\tau }_{0}{\sigma }_{0}-{M}_{1}{\tau }_{z}{\sigma }_{0}+i{B}_{0}{\tau }_{x}{\sigma }_{z},\\ {H}_{Z}={Z}_{1}{\tau }_{0}{\sigma }_{z}+{Z}_{2}{\tau }_{z}{\sigma }_{z},\end{array}$$
(5)

where \(C={C}_{0}+{C}_{2}({k}_{x}^{2}+{k}_{y}^{2})\), \(M={M}_{0}+{M}_{2}({k}_{x}^{2}+{k}_{y}^{2})\), and all the higher-order hopping parameters are ignored. \({H}_{-}={H}_{+}^{\dagger }\), which correspond to interlayer hopping terms to the lower and upper neighboring SL, respectively. By incorporating an onsite Zeeman term into the BHZ model, the essential low-energy physics can be effectively captured, allowing for a quantitative depiction of both electronic and topological properties.

By conducting first-principles calculations in distinct mixed stacking patterns and employing the aforementioned effective model Hamiltonian, we illustrate the evolution of band gaps across various stacking order scenarios characterized by the sequence (X, − Y, X) in Fig. 4a, b. Our effective model Hamiltonians are in good agreement with first-principles calculations, demonstrating the reliability and accuracy of the effective model in the multi-SL MBT films with mixed stacking configurations. All these stacking configurations yield large Chern number QAHIs with C = 2. Similar to the former calculation of local AHC in the (5, -5, 5) film, the consecutive incorrect stacking layers with odd SLs (Y ≥ 3) can be regarded as buffer layers that separate periodic patterned QAHIs in the stacking direction and strongly suppress the hybridization of QAH insulating layers. Contrarily, the consecutive correct stacking layers with odd SLs (X ≥ 3) can be treated as an effective Chern layer with local AHC of approximately e2/h. To construct high-Chern-number QAH insulators using MBT SLs, consecutive correct and incorrect regions, each with a minimum thickness of 3 SLs, should be alternated to form complete stacking configurations.

Fig. 4: Electronic and topological properties of mixed-stacking MnBi2Te4 films with distinct stacking patterns and artificial spin-orbit coupling, through effective model Hamiltonians and first-principle calculations.
Fig. 4: Electronic and topological properties of mixed-stacking MnBi2Te4 films with distinct stacking patterns and artificial spin-orbit coupling, through effective model Hamiltonians and first-principle calculations.
Full size image

Band gaps of distinct stacking patterns (X, −Y, X) calculated by a four-band effective model and b first-principles calculations. c The cumulative sum of Chern number C(N) as a function of layer number, where the consecutive correct (incorrect) stacking part are shaded by white (gray). d Chern number (C) and band gap as a function of artificial spin-orbit coupling strength λBi in the (7, − 5, 7, − 5, 7) stacking configuration.

As the layer number of consecutive correct and incorrect stacking configurations increases, band gaps demonstrate a clear increasing trend from several meV to tens of meV, highlighting a clear tendency towards larger band gaps in thicker films, with the maximum observed band gap reaching approximately 20 meV. Thus, large Chern number with C = N can be realized in ([X, − Y]. . . N−1, X) multi-SL MBT films without the need for additional materials, greatly enhancing the feasibility of experimentally observing large-gap QAHIs with C > 1.

Doping effects

Considering the typical electron-doped nature of MBT28,29,45,46, experimental synthesis of Mn(BixSb1−x)2Te4 (MBST) has been successfully accomplished47, revealing a discernibly lower concentration of electron doping. Moreover, the electronic and topological characteristics of MBST can be adroitly manipulated through the variable composition parameter x. In order to make theoretical predictions regarding doped MBST systems, first-principles calculations that incorporate diverse atomic spin-orbit coupling (λBi) specifically on Bi element are performed to circumvent the complexity of doped systems, where λBi = 1.0 (0.0) refers to signifies the presence (absence) of normal spin-orbit coupling (SOC) on Bi element in the system. All the possible interlayer and intralayer local stacking orders are iterated, and then corresponding tight-binding Hamiltonians based on localized Wannier functions are taken as building blocks to construct arbitrary stacking configurations with varied spin-orbit coupling in thick films.

Utilizing the aforementioned methodology, we construct 31-SL MBST film with (7, -5, 7, -5, 7) stacking consequence of varied spin-orbit coupling strength, and confirm MBST with λBi = 1.0 to be QAHIs with C = 3 and band gaps of 20 meV. The cumulative sum of layer-resolved AHC in Fig. 4c shows behavior similar to the (5, -5, 5) case. When λBi is tuned from 100% to 80%, the MBST film undergoes a topological phase transition from C = 3 to C = 0 at λBi = 0.9, as shown in Fig. 4d. Thus, this method serves as a computationally efficient tool for investigating the electronic and topological properties of MBST films with arbitrary stacking configurations, without requiring extensive and time-consuming first-principles calculations.

Discussion

Our work predicts a series of high-Chern-number quantum anomalous Hall insulators (QAHIs) with C > 1 in MnBi2Te4 (MBT) multilayer thin films, highlighting the significant role of mixed stacking orders in achieving large Chern numbers. We demonstrate the feasibility of constructing such QAHIs in typical van der Waals magnetic topological materials, obviating the necessity for extraneous interstitial buffer layers. Our further calculations on real-space-resolved anomalous Hall conductivity (AHC) within these QAHIs reveal that consecutive correct stacking orders are primarily responsible for the total AHC, while consecutive incorrect stacking orders contribute negligibly. Moreover, our combined approach of utilizing first-principles calculations and tight-binding effective Hamiltonians proves to be a highly powerful and accurate tool for quantitatively predicting electronic and topological properties of MBT heterostructures with mixed stacking orders. Our results offer strategic ramifications for the advancement of QAHIs with enhanced performance in next-generation topological electronic devices.

Methods

Computational details

First-principles calculations are performed using the Vienna ab initio Simulation Package (VASP)48 within the framework of density functional theory (DFT). Ionic core environment is simulated by the projector augmented wave method and the Perdew-Burke-Ernzerhof (PBE) functional49. The localized d-orbitals on Mn atoms are treated by the DFT + U method with U = 4 eV, which is consistent with the previous papers19,26. Lattice parameters and atomic positions are performed with a force criterion of 0.01 eV/Å, using the plane-wave basis with energy cutoff of 350 eV and Monkhorst-pack k-point mesh grid of 9 × 9 × 1. Topological invariants and surface state calculations are calculated by the WannierTools package50, using tight-binding Hamiltonians constructed from maximally localized Wannier functions51. Phonon dispersions are calculated by the Phonopy package52 using the finite displacement method.

Wannier-based tight-binding Hamiltonians are built using first-principles calculations for various MBT arrangements, specifically for 5- and 6-SL structures. This approach enables us to explore all possible stacking patterns, including both intralayer and interlayer stacking configurations, when only the nearest neighboring SL is taken into consideration. Consequently, tight-binding Hamiltonians for MBT thick films with various stacking configurations can be readily constructed, akin to assembling LEGO building blocks. Utilizing these tight-binding Hamiltonians, the electronic and topological properties of the thick films can be calculated efficiently and quantitatively, especially when compared to first-principles methods. Electronic band structures and topological properties of multilayer films with varied λBi are obtained by interpolating the tight-binding Hamiltonians constructed by Wannier functions under λBi = 1.0 and λBi = 0.8.