Introduction

With the pursuit of understanding and researching certain fundamental physical topics, such as quantum anomalous Hall effect1, axion electrodynamics2, topological conductivity3, and Majorana fermions2,4, tremendous efforts have been devoted to exploring suitable materials platforms theoretically and experimentally1,3,5,6,7,8,9. Recently, a new kind of van der Waals layered MB2T4 family materials has acquired considerable attention due to their exotic properties containing topological electronic states and (anti)ferromagnetism simultaneously2,7,10,11,12,13. Specifically, layer-dependent magnetic behaviors have been predicted and observed in MnBi2Te4 single crystals, where odd-layer samples indicate uncompensated ferromagnetic properties while even-layer crystals display antiferromagnetic features below 25 K14. Besides, both theoretical simulation and angle-resolved photoemission spectroscopy (ARPES) results show that the electronic structure of specific members of MB2T4 family materials, such as MnSb2Te4 and MnBi2Te4, possesses the surface Dirac cone structure, showing their promising potential for topological research and applications1,5,15. For instance, it is reported that a large magnetic gap of up to 90 meV at 1 K has been observed in MnBi4Te7, which is significantly enhanced by a self-organized alternating sequence of MnBi2Te4 septuple layers (SL) and Bi2Te3 quintuple layers (QL)1, inspiring the design of devices to achieve dissipation-less transport in the future16,17.

Despite the superior and attractive properties of the MB2T4 family, the direct preparation of their ultrathin single-crystal flakes is still restricted due to their complex septuple-atom structures, varieties of phases, and strong interlayer van der Waals forces18,19,20. Recent attempts to obtain several-layer MB2T4 family samples only rely on Au-assisted exfoliation, Al2O3-assisted exfoliation, and molecular beam epitaxy (MBE)10,19,21,22,23,24,25,26,27,28. Other methods, such as evaporation-rate-controlled chemical vapor deposition (CVD), have been adapted to grow multilayered MnBi2Te4 and MnBi4Te7. However, due to the difficulty in controlling precursor distribution, thermodynamics, and kinetics, the effective phase-controlled growth routes and patterns of these MB2T4 family materials are still not fully understood, which restricts the widespread exploration and application29.

Herein, we developed the flux-assisted phase-controlled growth method to obtain MnSb2Te4 nanosheets directly. MnSb2Te4 powder is directly utilized as precursors to be dissolved into the NaCl and KCl mixture flux, ensuring the stoichiometry ratio and uniform distribution. Then reaction thermodynamics and kinetics are regulated by temperature, together with the ratio of flux. An in-situ heating experiment and energy dispersive X-ray spectroscopy (EDS) mapping analysis prove the mechanism of crystallization. The thickness of MnSb2Te4 nanosheets can be adjusted to as thin as 2.4 nm, and the lateral size is tens of microns. Then aberration-corrected transmission electron microscope (AC-TEM) images show the crystal structure with SL-atom arrangement, indicating the capability of the phase control in 2D MnSb2Te4. Other 2D MB2T4 family crystals, including MnSb4Te7, MnBi2Te4, MnBi4Te7, MnBi2Se4, and MnBi4Se7, can be obtained via a similar strategy, which provides promising platforms to investigate their magnetic properties and applications.

Results and discussion

Flux-assisted growth strategy and process

The growth process and phase control are shown in Fig. 1a. First, MnSb2Te4 powder is synthesized and then tested by EDS in Supplementary Fig. 1. Then, a mixture of 1:1 molar ratio of KCl and NaCl as cosolvent with MnSb2Te4 powder is placed between two pieces of mica30. Next, these mica are tightly pressed between the self-designed setup, which consists of two 304 stainless steel plates (10 cm in length, 2 cm in width, and 0.3 cm in thickness). Screw holes are machined into the plates to allow for mechanical fastening. When all screws are fully tightened, the setup can generate a pressure of about 1.7 × 105 Pa, as measured by the thin-film sensor in Supplementary Fig. 3. However, due to the size of the precursor and cosolvent particles, the stacked mica cannot achieve perfect nanoscale contact, leaving the confined space partially open. After loading it into a tube furnace, the setup was heated under a continuous Ar flow. When the temperature goes up to 650 °C, the mixed salts gradually melt into a liquid state to dissolve the precursor powder stoichiometrically. It can almost maintain a stable and evenly distributed atomic ratio compared with the traditional methods, which provide the primary conditions for phase control. Molten flux may be squeezed out, and any volatile species formed inside can escape with the carrier Ar gas through the mica edges. Then the acquisition of the target phase can be achieved through careful temperature control. Notably, at high temperatures over 760 °C, MnSb2Te4 powder is quite easy to decompose and separate into Sb2Te3 and MnTe phases. In contrast, when the synthesis temperature is below the melting temperature of the flux mixture, no obvious reaction occurs. The melting point of the mixed salt of equimolar KCl and NaCl is ~650 °C according to the differential scanning calorimeter (DSC) result in Fig. 1b. Additionally, it is found that the endothermic peaks around the red region for the mixture of flux and precursors (mass ratio of 2:1) are similar to that for the NaCl–KCl flux, indicating the fact that the melting of the flux and the dissolution of the precursor are nearly simultaneous. Besides, nine kinds of fluxes with different melting points have been applied and listed in Fig. 1c, where the cyan region indicates that the temperature is too high to prevent the phase separation. As for the flux candidates located in the blue region, such as NaI, NaI–KCl, and LiCl, together with KI, are prone to inducing deliquescence reactions easily in the air, leading to the growth of antimony oxyhalides31. MnCl2 and Te in the red part may provide an excess of Mn or Te sources, which goes against the composition control. Consequently, the cosolvent of NaCl–KCl with a proper melting point and good chemical inertness is the best choice for synthesizing MnSb2Te4 nanocrystals.

Fig. 1: Synthesis scheme and characterizations of MnSb2Te4 nanosheets.
Fig. 1: Synthesis scheme and characterizations of MnSb2Te4 nanosheets.
Full size image

a Phase-controlled growth schematic of MnSb2Te4 via the flux-assisted method. b Differential scanning calorimeter (DSC) results of the flux and the mixed reactants, showing that the melting points of both are close to 650 °C. c Comparison of nine typical fluxes with different melting points based on FTsalt–FACT salt phase diagrams30. The blue part indicates deliquescent, moisture-absorbing cosolvents; the cyan part contains salts with melting points above 650 °C that may cause phase separation; and the red region marks salts that tend to alter the MB2T4 stoichiometry. d Histogram of 156 samples with various Mn: Sb molar ratios at different temperatures to determine proper growth conditions, where 650, 670, 700, 730, and 760 °C are set as the growth temperatures. e Diagram of reaction thermodynamics and kinetic competition as a function of temperature. The blue region represents kinetically hindered growth, and the red region indicates thermodynamically hindered conditions. f Optical images of MnSb2Te4 nanosheets on mica. g Atomic force microscopy (AFM) image of a ~2.4 nm MnSb2Te4 nanosheet, with the inset showing the corresponding height profile. h Energy dispersive X-ray spectroscopy (EDS) mapping of a ~450 μm region shows elemental distributions of different nanosheets. i Histogram of Mn:Sb ratios for 44 samples grown at 700 °C, where ~72.7% are close to the ideal 1:2 stoichiometry.

Figures 1d, e reveal the evolution of the chemical composition and reaction process with growth temperature, respectively, further elaborating on the mechanism of temperature in phase control. All the statistical data in Fig. 1d are collected from the EDS results of 156 triangular or hexagonal samples. It is obvious that maximum yield with the stoichiometric ratio ~1:2 of Mn: Sb can be obtained at 700 °C. According to the MnSb2Te4 phase diagram, 647 °C is the formation temperature for MnSb2Te432. However, when the flux begins to melt at ~650 °C, only a tiny fraction of precursor powder dissolves into the molten flux mixture, which reduces the probability of three types of precursor atoms encountering each other compared with the meeting of two kinds of atoms, resulting in the nucleation of bi-element compounds and phase separation. At this range, the thermodynamics of the target reaction are favorable, and the main factor leading to phase separation is the lack of compatibility in kinetics, as shown in the blue part in Fig. 1e. With the temperature rising to 700 °C, the dissolved precursor atoms dramatically increase and tend to crystallize into the target nanosheets, exhibiting both thermodynamic and kinetic compatibility. Once the growth temperature is higher than 730 °C, the reaction paths are dominated by temperature rather than the diffusion of reactant atoms, promoting the generation of thermodynamically stable Sb2Te3 and MnTe phases, as shown in the red part in Fig. 1e.

As for the thickness of the obtained samples, synthesis temperature, and the ratio of cosolvents/precursors play a crucial role, as shown in Supplementary Fig. 7. It is found that the minimum thickness detected by AFM increases with the synthesis temperature rising, which may be attributed to the increased concentration of the dissolved precursors in flux at high temperatures, leading to thermodynamically rapid vertical crystallization, which shows similar trend to the previous discussion31. Besides, the thickness can also be effectively tuned by the ratio of cosolvents and precursors. It is found that thinner samples can usually be formed when the percentage of precursors decreases. This is because in such a situation, there are insufficient atoms available to facilitate both lateral and vertical expansion31, causing small and thinner samples. Figure 1f shows the typical morphology of the obtained nanosheets in a large view, where the lateral size of the samples can be up to tens of microns. More optical images with different thicknesses are shown in Supplementary Figs. 8, 9, and 10. The minimum thickness for MnSb2Te4 is 2.4 nm in Fig. 1g, equivalent to the 2-layer thickness, with the inset showing the corresponding height profile. EDS mapping over a ~450 μm region in Fig. 1h shows elemental distributions of different nanosheets. Histogram in Fig. 1i displays the Mn:Sb ratios for 44 samples grown at 700 °C, where ~72.7% are close to the ideal 1:2 stoichiometry, while the remainder likely corresponds to phase-separation products.

Phase-control mechanism and universality

To better reveal the mechanism of the flux-assisted precipitation process, in-situ heating experiments were designed and conducted with the optical microscope as an observation and recording medium shown in Fig. 2a. Supplementary Movie 1 and optical images in Supplementary Fig. 13 record every step of the melting and precipitation process. Besides, ex-situ SEM obtained at 500, 700, and 760 °C is used to verify the chemical species at each step. First, below the melting point of the flux, all precursors and flux maintain a solid state. Then, as the temperature is increased to 650 °C, flux powder begins to melt into fluid and flow easily on the substrate. Meanwhile, the precursors dissolve into the liquid cosolvents to form the final flux mixture, as shown in Supplementary Fig. 17. With the volatilization of the cosolvent, the oversaturated flux will generate ultrathin MnSb2Te4 crystals. The main products generated at 700 °C are MnSb2Te4 single crystals. If the temperature continuously goes up, MnSb2Te4 flakes will further decompose and remelt. MnTe nanosheets then precipitate from the mixture while the rest part turns into Mn1−xSb2Te4−x (0 ≤ x ≤ 1). More ex-situ EDS images obtained at 500, 650, 680, 700, 730, 760, and 780 °C are shown in Supplementary Fig. 15–21, which indicates the state of the products at each temperature stage in detail.

Fig. 2: In-situ study of the flux-assisted crystallization mechanism.
Fig. 2: In-situ study of the flux-assisted crystallization mechanism.
Full size image

a Schematic illustrations of the in-situ growth process of the target crystals and byproducts at different temperatures. b, c Sb and Mn EDS mappings of the phase-separated area grown at 760 °C. Obvious Sb2Te3 and MnTe samples are grown in a mutually interlocked manner, indicating phase separation. d, e High-angle annular dark-field imaging (HAADF) of Sb2Te3 and MnTe products. f The parameters of the flux-assisted method in synthesizing thin-layer MB2T4 family materials. The inset crystal structures show the atomic arrangements of MB2T4 and MB4T7. SL and QL represent septuple and quintuple layers, respectively. The growth conditions include fluxes and temperature for MnSb2Te4, MnSb4Te7, MnBi2Te4, MnBi4Te7, MnBi2Se4, and MnBi4Se7, respectively, where the box area represents their corresponding appropriate temperature while the upper and lower whiskers indicate the maximum and minimum achievable synthesis temperatures, respectively. g Optical images of the obtained samples. h AFM results of the six 2D MB2T4 family materials. Error bars represent the standard deviation obtained from three independent measurements. i EDS results of these 2D MB2T4 and MB4T7 materials show a close stoichiometric ratio to their standard formula.

To further certify the phase separation products of flux-assisted precipitation, AC-TEM is conducted to characterize the final products. Figures 2b, c show the EDS mapping of the phase-separated area, where Sb2Te3 samples are grown along the edges of MnTe crystals in a mutually interlocked manner. Figures 2d, e display the high-angle annular dark-field imaging (HAADF) images of Sb2Te3 and MnTe, respectively. In the Sb2Te3 area, all the atoms show equivalent brightness due to the similar atomic numbers (Z) value of Sb and Te. As a contrast, the strong contrast in Fig. 2e for the MnTe area results from the Z difference of Mn and Te. To assess the universality of the phase control strategy in preparing various 2D MB2T4 family materials, MnSb4Te7, MnBi2Te4, MnBi4Te7, MnBi2Se4, and MnBi4Se7 bulk crystals are applied as precursors to crystallize into 2D samples with different mixed fluxes. As shown in the inset of Fig. 2f, both MnBi2Te4 and MnBi2Se4 display SL structure, while MnSb4Te7, MnBi4Te7, and MnBi4Se7 show alternating QL/SL stacking33. Figure 2f also illustrates the crystallization conditions for the listed materials, where 2D MnBi2Te4 and MnBi4Te7 can only be obtained under a strict growth condition (~590 °C) with 0.4 mol NaCl and 0.6 mol NaI as flux (melting point of ~575 °C) due to the poor thermal stability34. For the rest, NaCl–KCl mixed flux can satisfy their growth conditions. Although these species can precipitate from the mixed flux over a wide temperature range, the most appropriate growth temperatures for MnSb4Te7, MnBi2Se4, and MnBi4Se7 are quite hard to obtain pure phase products (660, 750, and 730 °C, respectively). The upper and lower whiskers in Fig. 2f indicate the maximum and minimum achievable synthesis temperatures, respectively. The obtained 2D MB2T4 family materials illustrate triangular or hexagonal shapes with lateral domain sizes of tens of microns as shown in Fig. 2g. Figure 2h shows the AFM results of the grown 2D MB2T4 family materials, where the detected minimum thickness for 2D MnSb4Te7, MnBi2Te4, MnBi4Te7, MnBi2Se4, and MnBi4Se7 is ~2.8, 6.4, 8.9, 1.9 and 2.8 nm, respectively. EDS test further confirms the chemical stoichiometric ratio of the grown 2D nanosheets in Fig. 2i, close to the ideal chemical composition. Detailed characterizations using EDS, AFM, Raman, and TEM are shown in Supplementary Figs. 25–27. The successful preparation of 2D MB2T4 family crystals proves the feasibility of flux-assisted precipitation for multi-element complex materials.

Structural characterization of MnSb2Te4

AC-TEM is further conducted to analyze the crystal structures and atomic arrangements of the as-grown nanosheets. We selected three samples of different thicknesses for cross-sectional TEM imaging. As shown in Fig. 3a, the low-magnification images display the initial sample thicknesses of approximately 49, 98, and 200 nm, with the samples sandwiched between the substrates and the deposited Pt layer. Due to the poor stability of the material, surface degradation occurred during transfer and FIB preparation, resulting in a smaller effective thickness. The atomic arrangements of the ab plane are also analyzed by AC-TEM. The uniform element distribution of Mn, Sb, and Te on the ab plane with sharp edges in Fig. 3b demonstrates the chemical uniformity. The stoichiometry ratio of the sample is indicated in Fig. 3c, close to 1:2:4, proving the feasibility of component control. Figure 3d presents cross-sectional HAADF images of MnSb2Te4 nanosheets, in which no staggered layers such as Sb2Te3 are observed, demonstrating effective control over phase purity. Complementary cross-sectional AC-TEM analyses for the other 2 thinner samples in Supplementary Fig. 29 consistently reveal a well-ordered SL-type stacking without any indication of QL-type phase separation, indicating that the stacking order remains unaffected by thickness and corroborating the effectiveness of the phase-controlled growth strategy. Figure 3e shows the HAADF images along the c-axis, indicating a single-crystal nature. The inset in Figs. 3d, e displays the selected area electron diffraction (SAED) result, where the indexed lattice plane is consistent with the lattice parameters. The atomic structure and corresponding annotations are illustrated in the image, where the pink middle atomic layers represent the Mn layer due to its smaller atomic number. The results of AC-TEM further validate the high quality of the samples, concurrently showcasing the capability of this method in phase control.

Fig. 3: Atomic arrangements and crystal structures of the obtained samples.
Fig. 3: Atomic arrangements and crystal structures of the obtained samples.
Full size image

a Cross-section aberration-corrected transmission electron microscopy (AC-TEM) of the characterized samples manipulated by the focused ion beam (FIB) with 3 different thicknesses. b Fine EDS mapping of the tested area from the ab plane. c EDS point scanning result of the obtained samples shows a stoichiometric ratio close to 1:2:4. d Cross-section AC-TEM images of MnSb2Te4 crystals, showing ordered septuple-atom layers. The inset is the corresponding selected area electron diffraction (SAED) results. On the right part, atomic lattice schematic along the [100] direction is embedded and shows excellent agreement with the AC-TEM results. e The amplified high-angle annular dark-field images of the atomic images from [001] and their corresponding atomic annotations. The inset is the corresponding SAED result, showing their single-crystal features. The atomic lattice schematic on the right part along the [001] direction is embedded on the right, matching well with the experimental data.

Magnetic properties of MnSb2Te4

SQUID and RMCD are further conducted to study the magnetic properties of the obtained nanosheets. The previous simulation and experiment results illustrate that a transition from paramagnetic to antiferromagnetic occurs below a certain temperature in pure bulk MnSb2Te4, even though the value of Néel temperature (TN) is controversial21,35. In a single-layer MnSb2Te4, Mn atoms provide the out-of-plane non-zero magnetic moments. In multiple-layer samples, spins of Mn in adjacent layers point in opposite directions, showing A-type antiferromagnetic behaviors26,35. However, Fig. 4a, b display the out-of-plane magnetization-temperature (M–T) and magnetization-magnetic field (M–H) curves, where the ferromagnetic transition occurs around 25 K, supported by the slope change in the M–T curves at this point. M–H curves at 5, 15, and 25 K display apparent hysteresis loops. When the temperature increases, the loops shrink and finally disappear. Even though the SQUID signals are obtained from many nanosheets, they still provide strong evidence for the judgment of the ferromagnetic properties of the obtained samples. To precisely characterize the magnetic properties of one single nanosheet, RMCD is further conducted on MnSb2Te4 nanosheets with different thicknesses, as Fig. 4c, d, where RMCD signals at different temperatures are collected. For the sample with a thickness of 4.8 nm (~4 layers) in Fig. 4c, the hysteresis loops start to appear below 17 K and gradually enlarge as the temperature is lowered to 2 K, which is consistent with the traditional ferromagnetic behavior and the SQUID results. As for the 17.3 nm (~14 layers) sample in Fig. 4d, a similar transition occurs at ~33 K. The reason why the obtained nanosheets exhibit ferromagnetism may be the inevitable substitution between Mn and Sb atoms, which has been widely reported in the previous literature21,23,35,36,37,38,39. The effects of various kinds of substitution types between Mn and Sb on magnetic properties have been well studied, where the replacement of Sb by Mn–Sb antisites may introduce another magnetic moment in the antiferromagnetic system, making MnSb2Te4 a ferromagnet below the Tc40.

Fig. 4: Magnetic properties of the obtained MnSb2Te4 nanosheets.
Fig. 4: Magnetic properties of the obtained MnSb2Te4 nanosheets.
Full size image

a Out-of-plane magnetization-temperature (M–T) measurement curves of the samples with zero field cooling (ZFC) and field cooling (FC), showing ferromagnetic features with Curie temperature (Tc) ~25 K. b Out-of-plane magnetization-magnetic field (M–H) measurement curves at 5, 15, 25, 50, 100, and 300 K, where obvious hysteresis loops appear below 25 K. The data for a, b are collected from superconducting quantum interference device test (SQUID). μ0 is the vacuum permeability, H is the magnetic field strength, and μ0H represents the magnetic field expressed in tesla (T). c Reflective magnetic circular dichroism (RMCD) results for MnSb2Te4 nanoflake of 4.8 nm (~4 layers) at different temperatures, where the hysteresis loop vanishes at 17 K. d RMCD results for MnSb2Te4 nanoflake of 17.3 nm (~14 layers) at different temperatures. e HAADF images of the samples with substitutions between Sb and Mn. Yellow circles mark the substituted sites. f Remanent RMCD signals of different MnSb2Te4 samples at zero magnetic field as a function of temperature, showing Tc descends as thickness decreases. Data points represent the remanent RMCD values extracted from hysteresis loops, while the solid lines are fits to the critical scaling function M = A(1-T/Tc)β, where M is the magnetization, A is a constant, T is the temperature, Tc is the Curie temperature, and β is the critical exponent. The inset is the illustration of magnetic coupling in the substituted MnSb2Te4 system.

Notably, the HAADF images of the ab plane from another sample show that dark dots exist on the periodic background, in which the substitution between Mn and Sb contributes to these atomic impurities in Fig. 4e. Generally, during the preparation process, it only takes several seconds to achieve crystallization from flux; hence, atomic misalignments will be inevitable. The substitution defects are also observed in the samples synthesized by other methods, such as solid-state reaction route, chemical vapor transport method, and molecular beam epitaxy, induced by rapid cooling and metastable thermodynamics21,35,36,37. Meanwhile, the substitution of Mn with adjacent Sb atoms introduces negligible lattice stress reflected by the similar diffraction data from the insets in Figs. 3e and 4e. Therefore, the small lattice distortion energy further diminishes the driving force for crystal recovery38. Besides, Density Functional Theory (DFT) calculations show that Mn–Sb antisite substitution has the lowest formation energy (−0.1492 eV) compared to Sb–Te substitution, demonstrating that such substitution is thermodynamically favorable in Supplementary Fig. 32. Furthermore, the M–T curves of the Chemical Vapor Transport (CVT)-grown bulk samples also exhibit a ferromagnetic signal with Tc ~27 K in Supplementary Fig. 31, which is close to the value above rather than showing antiferromagnetic behavior, thereby providing further evidence that Mn–Sb antisite substitution is present. Hence, we attribute the origin of magnetism to the atomic misalignment. Other RMCD data from the samples with different thicknesses are illustrated in Supplementary Fig. 33. Tc from these samples with different thicknesses by RMCD is statistically analyzed in Fig. 4f, where the remnant RMCD signals are extracted from hysteresis loops and fitted by the functional form A(1 − T/Tc)β (A, Tc, and β are simultaneous parameters). It is obvious that the detected Tc falls from 33.7 to 12.3 K as the thickness of the samples decreases, similar to the regularity in plenty of 2D magnets such as Fe3GeTe2. This is because the weakening of interlayer exchange interactions with thickness drops, making it hard to resist thermal fluctuations41,42,43,44. However, as shown in Supplementary Fig. 34, MnSb2Te4 exhibits a non-monotonic coercivity (Hc)-thickness trend at 2 K, in contrast to conventional 2D ferromagnets such as Fe3GeTe2 and Cr2Ge2Te6, where Hc typically increases monotonically with decreasing thickness owing to enhanced magnetic anisotropy41,45. In MnSb2Te4, although reduced dimensionality is also expected to enhance anisotropy, ferromagnetism is largely mediated by Mn–Sb antisite defects. The density of such defects may decrease in thinner flakes, weakening FM exchange interactions and lowering the magnetic moment density. In addition, ultrathin flakes are more susceptible to surface and edge disorder as well as degradation, which destabilize magnetic domains and suppress anisotropy26,41,45,46,47,48. The competition among these effects gives rise to a maximum Hc at intermediate thickness, a trend consistent with previous observations in mechanically exfoliated MnSb2Te4.

In summary, we developed a universal flux-assisted phase-controlled strategy with proper mixed salts as a flux to successfully obtain 6 kinds of ultrathin MB2T4 family crystals. Besides, the phase-separation mechanism has been thoroughly investigated via in-situ and ex-situ methods, making the thermodynamic and kinetic control of phase and thickness in 2D materials effective. The strategy may be inspiring for the controlled synthesis of other complex and multi-element 2D compounds. Additionally, in the ultrathin MnSb2Te4 nanosheets, it is found that the substitution of Mn atoms and Sb atoms can introduce ferromagnetic behavior with tunable Tc from ~12.3 to 33.7 K as the thickness increases, paving a promising way to study fundamental physics and design exotic dissipationless nanodevices.

Methods

Synthesis of bulk MB2T4 crystals

Bulk MB2T4 crystals were synthesized via a carefully controlled CVT method. MnSb2Te4 is described here as a representative example. High-purity elemental precursors, including manganese, antimony, and tellurium were weighed in a stoichiometric ratio of 1:2:4. The mixture was sealed in a quartz ampoule under high vacuum (~10−5 Torr) to prevent oxidation. To ensure complete precursor mixing and suppress the formation of secondary phases in cooler regions, the sealed ampoule was fully positioned within the high-temperature zone of tube furnace. The thermal process included several key stages: First, the temperature was rapidly ramped to 900 °C in 1 h and maintained for 2 h. Then, the system was cooled to 647 °C over 0.5 hours, establishing conditions suitable for controlled nucleation and crystal growth. The tube was then held at 647 °C for 48 h to facilitate crystal growth. Finally, the furnace was cooled to room temperature over 2 h, yielding bulk MnSb2Te4 crystals. Other MB2T4-family crystals were synthesized following the same CVT procedure, with appropriate adjustments to precursor stoichiometry and growth temperature, as summarized below: For MnSb4Te7 crystals, Mn, Sb, and Te were weighed with a molar ratio of 1:4:7. The system was first heated to 900 °C in 1 h and then cooled from 900 °C to 628 °C over 0.5 h and held at 628 °C for 48 h. For MnBi2Te4 crystals, Mn, Bi, and Te were weighed with a molar ratio of 1:2:4. The system was first heated to 900 °C and held for 2 h, then cooled from 900 °C to 590 °C over 40 min and held at 590 °C for 72 h. For MnBi4Te7 crystals, Mn, Bi, and Te were weighed with a molar ratio of 1:4:7. The system was first heated to 900 °C and held for 2 h, then cooled from 900 °C to 580 °C over 45 min and held at 580 °C for 72 h. For MnBi2Se4 crystals, Mn, Bi, and Se were weighed with a molar ratio of 1:2:4. The furnace temperature was ramped to 750 °C within 45 min and maintained for 48 h. For MnBi4Se7 crystals, Mn, Bi, and Se were weighed with a molar ratio of 1:4:7. The furnace temperature was ramped to 725 °C within 40 min and maintained for 48 h. After the growth stage, all samples were cooled to room temperature before removal from the furnace.

Synthesis of 2D MB2T4 crystals

All 2D MB2T4-family nanosheets were synthesized using a flux-assisted growth method starting from bulk MB2T4 precursors. 2D MnSb2Te4 is described below as an example. For the growth of MnSb2Te4 nanosheets, ~2 mg of bulk MnSb2Te4 crystal powder was mixed with ~4 mg of a NaCl–KCl cosolvent with an equimolar composition. The mixture was finely ground to ensure homogeneous mixing. Then the powders were uniformly spread between two freshly cleaved mica substrates, forming a mica-precursor-mica sandwich configuration. Next, the mica was placed between the self-designed equipment made of stainless steel and compressed tightly by tightening the screws. Then, the equipment was heated to 700 °C in 25 min and maintained for 3 min with 100 sccm Ar as protective gas. Finally, the samples can be taken out after the furnace and equipment cool down naturally. Other MB2T4 family nanosheets were synthesized following the same flux-assisted procedure, with appropriate adjustments to the salt composition, growth temperature, and gas atmosphere, as summarized below:

For 2D MnSb4Te7, ~2 mg MnSb4Te7 bulk powder mixed with ~4 mg of equimolar NaCl–KCl salts. The assembly was heated to 660 °C within 20 min and held for 3 min under 100 sccm Ar.

For 2D MnBi2Te4, ~2 mg MnBi2Te4 bulk crystals mixed with ~4 mg of NaCl–NaI mixed salt with a NaCl:NaI molar ratio of 2:3. The assembly was heated to 590 °C within 20 min and held for 3 min under a mixed 100 sccm Ar/H2 atmosphere.

For 2D MnBi4Te7, ~2 mg MnBi4Te7 bulk powders mixed with ~4 mg of a NaCl–NaI mixed salt (NaCl:NaI = 2:3). The assembly was heated to 585 °C within 20 min and held for 3 min under 100 sccm Ar/H2 mixed gas.

For 2D MnBi2Se4, ~2 mg MnBi2Se4 bulk powders mixed with ~4 mg of an equimolar NaCl–KCl salt. The assembly was heated to 750 °C within 25 min and held for 3 min under 100 sccm Ar.

For 2D MnBi4Se7, ~2 mg MnBi4Se7 bulk crystals mixed with ~4 mg of an equimolar NaCl–KCl salt. The assembly was heated to 730 °C within 25 min and held for 3 min under 100 sccm Ar.

AC-TEM characterization

Crystal structures were analyzed by AC-TEM, among which the samples tested along the c-axis were prepared by poly(methyl methacrylate) (PMMA) (950 PMMA A4, Micro Chem)-assisted method. First, the samples were spin-coated with PMMA and then etched with 3% hydrofluoric acid. Afterward, the peeled-off PMMA/sample was washed with deionized water and then transferred onto the copper TEM grids. Lastly, PMMA was removed by acetone. As for the preparation of cross-section samples, MnSb2Te4 nanosheets were first transferred onto SiO2/Si substrates by the PMMA-assisted method. After that, the picked MnSb2Te4 crystal was plated with a ~100 nm Pt layer and then cut along the predetermined direction to expose the cross-section crystal plane by FIB (Helios G4). Both c-axis oriented and cross-sectional AC-TEM measurements were carried out using a FEI Themis Z microscope operated at an accelerating voltage of 300 kV.

RMCD measurement

The RMCD measurements were performed using the Attocube closed-cycle cryostat (attoDRY2100), with temperatures down to 2 K and out-of-plane magnetic fields up to 9 T. The linearly polarized light of the 633-nm He–Ne laser was modulated between left and right circular polarization by a photoelastic modulator (PEM) and focused onto the sample through a high numerical aperture (NA = 0.82) objective. The reflected light was detected by a photomultiplier tube (THORLABS PMT1001/M). The magnetic reversal under the external magnetic field was detected by the RMCD signal determined by the ratio of the a.c. component of PEM at 50.05 kHz and the a.c. component of the chopper at 779 Hz, measured using a two-channel lock-in amplifier (Zurich HF2LI).