Introduction

Flexible thermoelectrics has attracted great attention because it can convert human body heat into useful electricity via the Seebeck effect, exhibiting the great potential to be used in self-powered wearable electronics1,2,3. Flexible thermoelectrics requires the thermoelectric (TE) materials with both good flexibility and high TE figure-of-merit (zT = S2σTκ−1, where Ѕ, σ, κ, and T are the Seebeck coefficient, electrical conductivity, thermal conductivity, and absolute temperature, respectively; PF = S2σ is defined as power factor)4,5,6. Beyond the organic polymers7 and inorganic films8, the recently discovered plastic/ductile inorganic semiconductors provide a new route to develop high-performance flexible TE materials9,10,11,12,13,14. Being different with the traditional brittle TE materials, the plastic/ductile inorganic TE materials possess metal-like deformability at room temperature and thus they can be easily machined into thin films or fibers to fabricate the flexible TE devices15,16,17. In the past five years, many high performance plastic/ductile inorganic TE materials (e.g. Ag2S-18,19,20,21,22,23, AgCuSe-24,25, Mg3Bi2-10,26, and SnSe2-based materials14,17,27,28) have been already developed. However, their PF and zT values are still much lower than the state-of-the-art brittle TE materials.

Comparing with the newly emerged plastic/ductile inorganic TE materials, Bi2Te3-family TE materials, including Bi2Te3, Bi2Se3, Sb2Te3, and their solid solutions, have a very long history6,29,30,31,32,33,34. The investigation can be traced back to as early as the middle of 20th century and now they are still the best TE materials near room temperature35,36,37,38,39,40. For a long time, it is widely recognized that the Bi2Te3-family TE materials are brittle and prone to break under mechanical loading41,42,43,44. However, most recently we found room-temperature exceptional plasticity in defective Bi2Te3 bulk crystals grown by the temperature gradient method45. The plastic Bi2Te3 bulk crystals can endure large deformations upon mechanical loading without fracture, similar with those of Ag2S-based plastic/ductile semiconductors reported before. Likewise, the plastic Bi2Te3 bulk crystals well maintain the excellent TE performance, with a high PF and zT about 39.2 μW cm-1 K-2 and 0.86 at 300 K, respectively, which are comparable with those of the state-of-the-art brittle TE materials45.

The coexistence of substantial BiTe and TeBi antisite defects are expected to be responsible for the exceptional plasticity in Bi2Te3 crystals45. Due to the similar atomic radius and electronegativity, substantial BiTe and TeBi antisite defects are expected to form in Bi2Te3 and their densities could be similar near the stoichiometric region due to the comparable defect formation energies46,47. The interaction, agglomeration, and motion of these BiTe and TeBi antisite defects could induce high-density, diverse microstructures (e.g. ripplocation, edge dislocation, swapped-bilayer, hyperdislocation, and lattice distortion) in bulk Bi2Te3 crystals (Fig. 1a). Under mechanical loading, these high-density, diverse microstructures can effectively relax and/or dissipate the applied stress to avoid the crack propagation, leading to the exceptional plasticity38,45,48,49,50.

Fig. 1: Room-temperature plasticity and TE performance of Bi2Te3-family crystals.
figure 1

a Schematics illustrating the atomic arrangements in Bi2Te3, Bi2Se3, and Sb2Te3 crystals. b Room-temperature power factor (PF), figure-of-merit (zT), and maximum bending strain (εmax) for Bi2(Te,Se)3 and (Bi,Sb)2Te3 bulk crystals. The PF data for Bi2(Te,Se)3 and (Bi,Sb)2Te3 crystals reported before are also included for comparison31,32.

Although Bi2Se3 and Sb2Te3 have the same crystal structure to that of Bi2Te3 (Fig. 1a), the dominated native defects in these three compounds are different51,52,53. Thus, Bi2Se3 and Sb2Te3 might possess distinct plasticity with Bi2Te3 at room temperature, but the related investigation has not been reported yet. Previously, in order to optimize the TE performance, Se and Sb are usually introduced into Bi2Te3, but the influence of Se- and Sb-alloying on the plasticity of Bi2Te3 is unclear. Furthermore, the chemical composition range where the Bi2(Te,Se)3 and (Bi,Sb)2Te3 solid solutions simultaneously possess good plasticity and TE performance is also unknown.

In this work, a series of Bi2(Te,Se)3 and (Bi,Sb)2Te3 bulk crystals are grown by the temperature gradient method to clarify the relationship among chemical composition, native defect, microstructure, and plasticity. Bi2Se3 and Sb2Te3 bulk crystals demonstrate poorer plasticity than Bi2Te3 bulk crystal, which can be well explained by the lack of diverse microstructures caused by the difficulty to form substantial antisite defects. As a result, alloying either Se or Sb in Bi2Te3 degrades the plasticity. Finally, the composition range for the Bi2(Te1-xSex)3 and (Bi1-ySby)2Te3 bulk crystals that can simultaneously achieve good plasticity and high TE performance is determined (Fig. 1b).

Results and discussion

Here the Bi2(Te,Se)3 and (Bi,Sb)2Te3 bulk crystals are grown by the temperature gradient method (Supplementary Fig. 1). The raw materials and growth process are strictly controlled to minimize the chemical stoichiometry deviation in the grown products. Figure 2a shows the optical images of as-grown Bi2Te3, Bi2Se3, and Sb2Te3 crystals. Their surfaces are smooth and shiny alike metals. The XRD patterns performed on the cleavage surfaces of these crystals only include the (00 l) (l = 3, 6, 9,…) diffraction peaks, proving that the Bi2Te3, Bi2Se3, and Sb2Te3 single crystals are successfully obtained (Supplementary Fig. 2). Energy dispersive spectroscopy (EDS) mapping indicates that all elements are homogeneously distributed inside the crystals (Supplementary Fig. 3). The average atomic ratios of Bi:Te, Bi:Se, and Sb:Te are 40.3:59.7, 39.1:60.9, and 40.8:59.2, respectively. The relative densities of Bi2Te3, Bi2Se3, and Sb2Te3 crystals are 99.6%, 99.0%, and 99.1%, respectively.

Fig. 2: Plasticity and microstructure of Bi2Te3, Bi2Se3, and Sb2Te3 crystals.
figure 2

a Optical images of the as-grown Bi2Te3, Bi2Se3, and Sb2Te3 ingots and the single crystals peeled off the ingots. Engineering stress vs. strain curves of Bi2Te3, Bi2Se3, and Sb2Te3 bulk crystals in the b three-point bending test and c tensile test. Optical images and corresponding SEM images of the d Bi2Te3, e Bi2Se3, and f Sb2Te3 crystals after bending tests. g, h HAADF-STEM images, i, j IFFT images projected on (003) plane, and k, l strain maps of Bi2Se3 and Sb2Te3 crystals, respectively. The insets in g, h are their corresponding SAED patterns. X-ray rocking curves measured on the m (0015) plane and n (006) plane of the cleaved Bi2Te3, Bi2Se3, and Sb2Te3 crystals. The data of Bi2Te3 are taken from Ref. 45.

Due to the existence of grain boundary, polycrystalline Bi2Te3, Bi2Se3, and Sb2Te3 are brittle with the engineering strains less than 0.5% in the three-point bending test (Supplementary Fig. 4). In contrast, the Bi2Te3 crystal prepared in this work has a large engineering strain over 20% along the in-plane direction in the three-point bending test (Fig. 2b). This value is much higher than those of the Bi2Se3 and Sb2Te3 crystals (less than 5%). In the tensile test, the maximum strains of Bi2Se3 and Sb2Te3 crystals along the in-plane direction are less than 1%, which are also smaller than that of Bi2Te3 crystal (about 5%) (Fig. 2c). After the three-point bending tests, obvious cracks are observed on the surface of Bi2Se3 and Sb2Te3 crystals, while no obvious cracks are observed on the surface of Bi2Te3 crystal (Fig. 2d–f).

The above results indicate that the plasticity of Bi2Se3 and Sb2Te3 bulk crystals is much poorer than that of Bi2Te3, although they possess the same crystal structure and are prepared by the same method. In order to understand the reason, we first analyze the chemical bonding of Bi2Te3, Bi2Se3, and Sb2Te3 by calculating their crystal orbital Hamilton population (COHP), charge density, and electron localization function (ELF). As shown in Supplementary Fig. 5, the interlayer Se-Se bonding energy (0.14 eV) in Bi2Se3 is much lower than those of the interlayer Te-Te bonding energy (0.4 eV) in Bi2Te3 and the interlayer Te-Te bonding energy (0.3 eV) in Sb2Te3. Likewise, the electrons of Se atoms in Bi2Se3 are more localized than those of Te atoms in Bi2Te3 and Sb2Te3 (Supplementary Fig. 6). This leads to the smaller slipping energy barriers, stacking fault energies, and cleavage energies of Bi2Se3 than those of Bi2Te3 and Sb2Te3 (Supplementary Fig. 7). However, the deformability indicator (ξ) of Bi2Te3, Bi2Se3, and Sb2Te3 are similar (Supplementary Fig. 7f), with the values much lower than those of intrinsically plastic InSe and SnSe2. This indicates that Bi2Te3, Bi2Se3, and Sb2Te3 should be intrinsically brittle, and the distinct mechanical properties observed in the experiment are not originated from the intrinsic chemical bonding.

According to our previous study, the exceptional plasticity of Bi2Te3 crystal is attributed to the existence of high-density, diverse microstructures (e.g., ripplocation, edge dislocation, swapped-bilayer, hyperdislocation, and lattice distortion), which can effectively relax and dissipate the applied stress before the crack formation and propagation45. Here, we also use the scanning transmission electron microscopy (STEM) to characterize the microstructures of Bi2Se3 and Sb2Te3 bulk crystals. As shown in Fig. 2g, h, the atomic arrangements in Bi2Se3 and Sb2Te3 crystals are very regular. No obvious ripplocations, edge dislocations, swapped-bilayers, hyperdislocations, and lattice distortions are observed, which is different from the scenario in Bi2Te3 (Supplementary Fig. 8). The diffraction spots in the selected-area electron diffraction (SAED) patterns of Bi2Se3 and Sb2Te3 are clear and sharp, which are also different from the blurry and twisted diffraction spots observed in the SAED pattern of Bi2Te3. The inverse fast Fourier transformation (IFFT) images projected on (003) in the SAED patterns (Fig. 2i, j, and Supplementary Fig. 8b) show perfect diffraction fringes in Bi2Se3 and Sb2Te3, being different from the existence of lots of interlayer cross-linking and fluctuations in Bi2Te3. The strain distributions generated by geometric phase analysis (GPA) analysis are also quite ordered in Bi2Se3 and Sb2Te3 compared with those in Bi2Te3 (Fig. 2k, l and Supplementary Fig. 8c).

The X-ray rocking curve is usually used to evaluate the quality of a single crystal. We conduct X-ray rocking curve measurements on the (0015) plane and (006) plane of Bi2Te3, Bi2Se3, and Sb2Te3 crystals (Fig. 2m, n). Being different from the broad and non-symmetric diffraction peaks of Bi2Te3, the diffraction peaks of Bi2Se3 and Sb2Te3 are very sharp and symmetric. Particularly, no obvious split peaks can be observed on their curves. These further prove that the present Bi2Se3 and Sb2Te3 have much better crystal quality with fewer defects and smaller residual strains than Bi2Te3. Due to the absence of diverse microstructures in Bi2Se3 and Sb2Te3, the applied stress cannot be timely relaxed and dissipated, and the cracks are easily formed and propagated, yielding poor plasticity.

In order to understand the different microstructure and plasticity of Bi2Te3, Bi2Se3, and Sb2Te3, the formation energy of native defects in Bi2Te3, Bi2Se3, and Sb2Te3 is calculated. The results are shown in Fig. 3a–c. For Bi2Te3, when the chemical potential of anion is tuned to −0.14 eV (Fig. 3a), BiTe and TeBi own the same formation energy of about 0.63 eV. This value is much lower than other native defects, e.g., Te vacancy (VTe) and Bi vacancy (VBi) at the same point of chemical potential (Fig. 3d), suggesting that a substantial and similar density of BiTe and TeBi antisite defects are easily formed inside the lattice. Their interaction and agglomeration then lead to the formation of high-density, diverse microstructures. This evolution process is supported by our calculations. Begining with a supercell with two isolated defects, i.e., BiTe and TeBi (Fig. 3g), these two defects agglomerate to form a defect pair (Fig. 3h), which lowers the energy to 0.36 eV. Then, these pairs attract with each other to form a line of defects (Fig. 3i), which further reduces the energy to 0.27 eV for each pair. These defect lines continue to gather to form a plane of defects (Fig. 3j), which further lowers the energy to 0.15 eV for each pair. This result suggests that it is energetically favorable for antisite defects to agglomerate into diverse microstructures in Bi2Te3.

Fig. 3: Defect calculations for Bi2Te3, Bi2Se3, and Sb2Te3.
figure 3

ac Formation energy of native defects in Bi2Te3, Bi2Se3, and Sb2Te3 with varying chemical potentials of anions. df Formation energy of native defects in Bi2Te3, Bi2Se3, and Sb2Te3 at the crosspoint denoted in ac, where the formation energy of the two antisite defects are equal. The insets show the schematics of the Te/Se(1)–Bi/Sb–Te/Se(2)–Bi/Sb–Te/Se(1) quintuple layer structure. Possible agglomeration processes of antisite defects in Bi2Te3, from g isolated antisite defects, to h antisite defect pairs, then to i a defect line, and finally agglomerated to form j a defect plane. k Formation energy of structures in gj relative to that containing isolated antisite defects, which is calculated as \({E}_{{form}}=\left(E+\left(n-1\right){E}_{{perf}}-n{E}_{{iso}}\right)/n\), where E, Eiso, and Eperf are the energy of the structure containing n defect pairs, the structure containing two isolated antisite defects, and the structure without defects, respectively.

In Bi2Se3 and Sb2Te3, the cases are different. In Bi2Se3, when the Se chemical potential is tuned to −0.21 eV as shown in Fig. 3b, BiSe and SeBi antisite defects have the same formation energies of about 1.32 eV (Fig. 3e). This value is much larger than that in Bi2Te3. Thus, the density of BiSe and SeBi antisite defects in the lattice of Bi2Se3 should be quite low. While in Sb2Te3, SbTe is the dominated native defect over a wide chemical region. Only when the anion chemical potential is tuned to the vicinity of the extreme Te-rich point as −0.04 eV, SbTe and TeSb can own the same formation energy (Fig. 3c), which is about 0.6 eV (Fig. 3f). However, in experiment, this point is difficult to be achieved. This is consistent with the fact that the strong p-type conduction behavior of Sb2Te3 is always observed. In summary, the distinct feature of native defects in Bi2Se3 and Sb2Te3 with that in Bi2Te3 is responsible for their poor plasticity.

The good plasticity of Bi2Te3 and poor plasticity of Bi2Se3 and Sb2Te3 indicate the existence of composition-dependent plastic-brittle transition in Bi2(Te,Se)3 and (Bi,Sb)2Te3 bulk crystals. To confirm this point, a series of Bi2(Te1-xSex)3 and (Bi1-ySby)2Te3 bulk crystals are grown by the temperature gradient method. The XRD patterns for the cleavage surfaces are shown in Supplementary Fig. 9, wherein only the (00 l) (l = 3, 6, 9, …) diffraction peaks can be detected. Figure 4a shows the strain-stress curves of Bi2(Te1-xSex)3 bulk crystals in the three-point bending test. With increasing Se content x in Bi2(Te1-xSex)3, the maximum bending strain that can be endured by the sample before cracking is significantly decreased. For example, when x = 0.1, the maximum bending strain is over 15%, similar to that of the pristine Bi2Te3. However, when x = 0.3, the maximum bending strain is decreased to only 3%. When x = 0.7, the maximum strain is further reduced to less 1%. After the bending test, obvious cracks are observed on the surface of Bi2(Te0.6Se0.4)3 and Bi2(Te0.1Se0.9)3 crystals (Supplementary Fig. 10a). These results prove that alloying Se in Bi2Te3 significantly degrades its plasticity. In comparison, alloying Sb in Bi2Te3 also degrades its plasticity, but to a lesser extent. As shown in Fig. 4b, when y = 0.6, the (Bi1-ySby)2Te3 crystals can still endure a large bending strain above 15% in the three-point bending test. When y > 0.6, the maximum bending strain of (Bi1-ySby)2Te3 is significantly reduced, but the value is still about 3%. After the bending test, obvious cracks are also observed on the surface of (Bi0.1Sb0.9)2Te3 crystals (Supplementary Fig. 10b).

Fig. 4: Plasticity and microstructure of Bi2Te3-family crystals.
figure 4

a Bi2(Te1-xSex)3 and b (Bi1-ySby)2Te3 bulk crystals in the three-point bending test. HAADF-STEM images (I), IFFT images projected on (003) plane (II), and strain maps (III) of c Bi2(Te0.3Se0.7)3 crystal and d (Bi0.7Sb0.3)2Te3 crystal. The red, yellow, and blue symbols represent ripplocation, swapped-bilayer, and dislocation, respectively.

The mechanical properties of the Bi2(Te1-xSex)3 and (Bi1-ySby)2Te3 bulk crystals are also related to their microstructures. Taking Bi2(Te0.3Se0.7)3 and (Bi0.7Sb0.3)2Te3 as examples, their microstructures are characterized by STEM. As shown in Fig. 4c, the atomic arrangements in Bi2(Te0.3Se0.7)3 crystal are very regular with no obvious defects and distortions. The IFFT image projected on (003) in the SAED pattern shows perfect diffraction fringes, and the strain distributions are quite ordered. These results are consistent with its poor plasticity. In contrast, there are still high-density, diverse microstructures (i.e. swapped-bilayer, ripplocation, dislocation, and lattice distortion) in (Bi0.7Sb0.3)2Te3 (Fig. 4d). The IFFT image projected on (003) in the SAED pattern shows the existence of lots of interlayer cross-linking and fluctuations, and the strain distributions are quite disordered alike that observed in pristine Bi2Te3. These results are consistent with its good plasticity.

We have measured the electrical transport properties of the Bi2(Te1-xSex)3 and (Bi1-ySby)2Te3 bulk crystals. As shown in Fig. 5, the Bi2Te3 crystal exhibits p-type conduction with positive Seebeck coefficients (S). Its room-temperature carrier concentration (nH) is about 1.09 × 1019 cm-3. Alloying Se at the Te-site of Bi2Te3 can decrease the nH. When the Se-alloying content x reaches 0.3-0.4, the conduction behavior converts from p-type to n-type. Then, with increasing Se-alloying content, the nH is gradually increased. Correspondingly, the electrical conductivity (σ) of Bi2(Te1-xSex)3 shows a non-monotonous variation trend with increasing Se-alloying content. It reaches the minimum around x = 0.3-0.4. Likewise, the sign of S is changed around x = 0.3–0.4. Alloying Sb at the Bi-site of Bi2Te3 does not change the conduction type, but just increases the nH. Thus, with increasing Sb-alloying content, the σ of (Bi1-ySby)2Te3 basically shows an increasing trend, while the S shows a decreasing trend. The reason for these variations also originated from the change of native defects, which has been clearly discussed elsewhere46,51,52,53.

Fig. 5: Electrical transport properties of Bi2Te3-family crystals.
figure 5

Room-temperature a carrier concentration (nH), b electrical conductivity (σ), and c Seebeck coefficients (S) of Bi2(Te1-xSex)3 bulk crystals. Room-temperature d nH, e σ, and f S of (Bi1-ySby)2Te3 bulk crystals.

In addition, we have measured the low-temperature resistivity (ρ) of Bi2Te3, Bi2Se3, and Sb2Te3 bulk crystals (Supplementary Fig. 11). By using the ρ values at 300 K and 4.2 K, we calculated their residual resistivity ratio (RRR = ρ(300 K)/ρ(4.2 K)) values. Interestingly, Bi2Te3 has a much higher RRR than Bi2Se3 and Sb2Te3. The reason might be that the high-density, diverse microstructures in Bi2Te3 are mainly constructed by the antisite defect pairs, which are uncharged. They have an insignificant effect on scattering electrons compared with the charged defects in Bi2Se3 and Sb2Te3 (e.g., VSe and SbTe).

Based on the measured S and σ, the room-temperature power factor (PF = S2σ) of Bi2(Te1-xSex)3 and (Bi1-ySby)2Te3 bulk crystals is calculated. As shown in Fig. 1b, the variation trend of PF as a function of the Se/Sb alloying content is roughly similar to the data reported before. In n-type materials, Bi2(Te0.4Se0.6)3 has the maximum PF around 18.8 μW cm−1K−2. In p-type materials, (Bi0.4Sb0.6)2Te3 has the maximum PF around 47.3 μW cm−1K−2. Combining the measured maximum strains in the three-point bending test, the composition range for the Bi2(Te1-xSex)3 and (Bi1-ySby)2Te3 bulk crystals that simultaneously possess exceptional plasticity with εmax > 10% and high PF > 20 μW cm−1K−2 can be determined as 0 ≤ x < 0.2 and 0 ≤ y < 0.7. Furthermore, the plastic Bi2(Te1-xSex)3 and (Bi1-ySby)2Te3 bulk crystals possess decent mechanical stability under the bending condition. For example, the σ and S of (Bi0.8Sb0.2)2Te3 and (Bi0.7Sb0.3)2Te3 are similar before and after the three-point bending test (Supplementary Fig. 12a,b). The cyclic bending test also shows that the relative resistance variation (ΔR/R0), σ, and S of the plastic (Bi0.8Sb0.2)2Te3 is scarcely changed after 300 times of bending under the bending radius of 10 mm (Supplementary Fig. 12c,d).

Figure 6a, b shows the room-temperature thermal conductivity (κ) of the Bi2(Te1-xSex)3 and (Bi1-ySby)2Te3 bulk crystals. Bi2Se3 and Sb2Te3 have much higher κ than Bi2Te3. Based on the Wiedemann–Franz law, the lattice thermal conductivity (κL) of these bulk crystals is calculated (Fig. 6c, d). In the ideal case, with increasing the Sb/Se alloying content in Bi2Te3, the κL should firstly decrease due to the strengthened point defect scattering to phonons, and then increase due to the higher κL of Sb2Te3/Bi2Se3 than that of Bi2Te3. As shown in Fig. 6d, the κL data of the (Bi1-ySby)2Te3 bulk crystals roughly agree with this trend. This scenario is reasonable because the (Bi1-ySby)2Te3 with 0 ≤ y < 0.7 possesses similar high-density, diverse microstructures to Bi2Te3. Thus, the κL variation is mainly dominated by the magnitude of point defect scattering. However, the case is more complex in the Bi2(Te1-xSex)3 bulk crystals. The κL data show two dips with increasing Se content (Fig. 6c). For the Bi2(Te1-xSex)3 prepared in this work, the content of diverse microstructures is already very few when x > 0.2. Thus, the co-effect caused by the reduced content of diverse microstructures and the enhanced Se-alloying content is responsible for the complex κL variation.

Fig. 6: Thermal transport properties of Bi2Te3-family crystals.
figure 6

Room-temperature thermal conductivity (κ) of a Bi2(Te1-xSex)3 and b (Bi1-ySby)2Te3 bulk crystals. Room-temperature lattice thermal conductivity (κL) of c Bi2(Te1-xSex)3 and d (Bi1-ySby)2Te3 bulk crystals.

Finally, Fig. 1b shows the room-temperature TE figure-of-merit (zT) of the Bi2(Te1-xSex)3 and (Bi1-ySby)2Te3 bulk crystals. In n-type materials, Bi2(Te0.4Se0.6)3 has the maximum zT around 0.42. In p-type materials, (Bi0.6Sb0.4)2Te3 and (Bi0.8Sb0.2)2Te3 have the maximum zT around 1.0. Combining the measured maximum strains in the three-point bending test, the composition range for the Bi2Te3-family bulk crystals that simultaneously possess exceptional plasticity with εmax > 10% and high zT > 0.6 can be determined as 0 ≤ y < 0.7. These compositions are good candidates for the future investigation of plastic Bi2Te3-family crystals.

In summary, this work systematically investigates the microstructure, mechanical, and TE properties of Bi2Te3, Bi2Se3, Sb2Te3, and their solid solutions. Combining with the calculations of defect formation energy, the relationship among chemical composition, native defects, microstructures, and plasticity is clarified. The composition range for the Bi2(Te1-xSex)3 and (Bi1-ySby)2Te3 bulk crystals that simultaneously possess exceptional plasticity and high PF/zT is determined. This work provides a deep understanding of the exceptional plasticity recently discovered in Bi2Te3 bulk crystal and sheds light on the future investigation of plastic Bi2Te3-family TE materials.

Methods

Synthesis

Bi2Te3, Bi2Se3, Sb2Te3, and their solid solutions were grown by the temperature gradient method. High-purity raw materials, Bi shots (99.999%, Alfa Aesar), Te shots (99.999%, Alfa Aesar), Se shots (99.999%, Alfa Aesar), and Sb shots (99.999%, Alfa Aesar) were weighed out based on the chemical stoichiometry and sealed in quartz tubes under vacuum (<10−2 Pa). The quartz tubes of Bi2(Te1-xSex)3 were put into the vertical melting furnace, heated to 1033 K with a rate of 100 K h−1, and held at this temperature for 10 h. After that, the furnace temperature was first decreased to 808 K with a cooling rate of 2 K h−1 and then decreased to room temperature at a rate of 100 K h−1. The quartz tubes of (Bi1-ySby)2Te3 were put into the vertical melting furnace, heated to 943 K with a rate of 100 K h−1, and held at this temperature for 10 h. After that, the furnace temperature was first decreased to 808 K with a cooling rate of 2 K h−1 and then decreased to room temperature at a rate of 100 K h−1. The bulk ingots obtained were cleaved along the (00 l) plane and cut into specific shapes for characterization.

Characterization

The phase composition was characterized by an X-ray diffractometer (XRD, D/max-2550 V, Rigaku, Japan). The elemental distribution was determined by Energy Dispersive Spectroscopy (EDS, Oxford, UK). The microstructures were measured by the scanning transmission electron microscopy (STEM, HF5000, HitachiTech, Japan). The specimens for the STEM characterization were prepared by using the focused ion beam technique. The bulk specimen with a flat and smooth surface was first roughly machined by using a strong Ga ion beam with a current of 0.5 nA and a voltage of 30 kV. When the specimen’s thickness was machined to a few micrometers, the current of the Ga ion beam was reduced to 30-50 pA while the voltage was not changed. When the thickness was machined to a few hundred nanometers, the voltage was reduced to 5 kV to finely polish the specimen until its thickness reached around 30–50 nm. During STEM characterization, the accelerating voltage was set to 200 kV. X-ray rocking curves were obtained by using an X-ray diffractometer with Cu Kα sources (XRD, D/max-2550 V, Rigaku, Japan). Three-point bending tests and tensile tests were performed by using a dynamic thermomechanical analyzer (DMA 850, TA Instruments, USA) with a constant loading rate of 0.1 mm min−1. The sizes of the samples used for the bending test and tensile test were about 3.0 × 0.8 × 7.0 mm3 and 13 × 3.0 × 0.25 mm3, respectively. The Seebeck coefficient (S) and electrical conductivity (σ) were measured at 300 K (ZEM-3, ULVAC, Japan). The low-temperature resistivity (ρ) and thermal conductivity (κ) were measured by Physical Property Measurement System (PPMS, Quantum Design, USA). The Hall carrier concentration (nH) was calculated by nH = 1/RHe, where RH is the Hall coefficient measured by PPMS, and e is the elementary charge.

Calculation

The first-principles calculations were carried out using the VASP program54. The PBE functional55 and PAW56 potential were adopted to optimize the atomic structures and calculate the formation energy of defects. Spin-orbit coupling was included in all the defect calculations. The properties of native defects of Bi2Te3, Bi2Se3, and Sb2Te3 have been calculated by West et al.47 and Zhang et al.46, but no van der Waals (vdW) correction was considered in these works. Here, we considered the vdW correction by DFT-D3 method in the calculation57, which is found to increase the formation energy of the vacancies while having no significant effects on the antisite defects as shown in Supplementary Table 1. A (3 × 3 × 1) 135-atom supercell was used to model the defects, and a 2 × 2 × 1 k-point grid was used to fully relax the atomic structure and calculate the energy. The formation energy of a point defect was calculated as \({E}_{{form}}=E-{E}_{{perf}}+{\sum}_{i}\Delta {n}_{i}{\mu }_{i}\) 58, where E and Eperf are the energies of structures with and without defects, and \(\Delta {n}_{i}\) is the quantity of the i-th species exchanged in forming the defect, and μi is the chemical potential of the i-th species. To model the agglomeration of the antisite defects in Bi2Te3, the 135-atom supercell was also used to construct the supercells with defects. The structure in Fig. 3g contains two antisite defects BiTe and TeBi, which are separated by more than 16 Å. The structures with defect pair, defect line, and defect plane contain one, three, and nine pairs of agglomerated antisite defect pairs, respectively. The interlayer cleavage energy was calculated by the classical slab model based on a supercell with 3 layers of [Te/Se(1)–Bi/Sb–Te/Se(2)–Bi/Sb–Te/Se(1)]55,59,60,61. The slipping energies along the (001) < 120> slipping system of the supercells of 6 layers of [Te/Se(1)–Bi/Sb–Te/Se(2)–Bi/Sb–Te/Se(1)] were calculated to obtain the generalized stacking fault energy (GSFE) surface of (001) plane. The maximum and minimum GSFE energy along the slipping path were defined as the interlayer slipping energy barrier and the stacking fault energy, respectively. The details for the charge densities, electron localization function, and crystal orbital Hamilton population analysis can be found elsewhere9,62.