Introduction

In recent years, twisted van der Waals moiré superlattices1,2,3,4,5 have garnered significant interest following the groundbreaking discovery of unconventional superconductivity (SC)6,7,8,9 in magic-angle twisted bilayer graphene10, which inspired numerous theoretical investigations on its microscopic origin11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29. Beyond graphene-based systems30,31,32,33,34,35,36,37,38,39,40,41,42,43, twisted transition metal dichalcogenides (TMDs)44,45 are regarded as a promising alternative platform to investigate many-body physics. Due to their high tunability, twisted TMDs can host a wide variety of exotic phases, including correlated insulators46,47,48,49,50,51,52, integer and fractional quantum anomalous Hall states53,54,55,56,57, and also integer and fractional quantum spin Hall states58. However, the experimental realization of SC in twisted TMD systems has remained elusive48, though many theoretical works have suggested that SC should develop in such systems59,60,61,62,63,64,65,66,67,68,69, typically through doping the correlated insulators. Very recently, two independent studies have reported the discovery of robust SC phases in twisted WSe2 (tWSe2) homobilayers70,71. The phenomenology of these SC phases differ significantly from that would arise in doped Mott insulators, which calls for further theoretical investigations, particularly regarding the possible pairing mechanism and topological properties of SC, as well as the nature of adjacent correlated insulators.

Previous theoretical studies48,59,60,61,62,63,64,65,66,72,73,74,75 of tWSe2 homobilayers are largely based on the tight-binding description of the single-band moiré Hubbard model with spin-dependent hopping phase tuned by the displacement field. By formulating the problem in real space, this approach captures the locality of screened interactions and facilitates a clearer real-space insights essential for understanding the correlated physics. However, such a simple single-band model cannot capture the potential nontrivial band topology45,76,77,78,79 and only applies to limited twist angles48. Thus, a direct Wannierization79,80,81,82,83,84,85 of the standard continuum model45 with a proper set of low-energy bands appears to be a more suitable approach to start with.

In this paper, we focus mainly on the experimental results in ref. 70, where the SC phase is observed in a 3.65° tWSe2 device at integer filling factor ν = − 1 under a small displacement field, along with a correlated insulator phase in a larger displacement field. Using the continuum model parameters provided in ref. 79, we first construct a three-band tight-binding model for 3.65° tWSe2 through direct Wannierization79,80,81,82,83,84,85. In addition to the onsite Hubbard repulsion, our model includes the next-nearest-neighbor (NNN) attraction, which can arise effectively through electron-phonon coupling11,12,13,14,15,16,17,18,19,20,21,22,23,24,86,87 or purely electronic mechanisms88,89,90,91,92,93,94,95. By keeping these key interaction terms, our model can capture the essential qualitative physics of the experiment70 and offer valuable real-space insights into the twisted WSe2 system. We then perform a thorough mean-field analysis to the interacting model and obtain the phase diagram at filling ν = − 1 under realistic interaction strengths, concluding that the SC phase is consistent with inter-valley pairing with mixed \({d}_{xy}\pm i{d}_{{x}^{2}-{y}^{2}}\) and px ∓ ipy-wave symmetry, featuring nontrivial topology under Altland-Zirnbauer A-class96,97,98 with Chern number C = ± 2, and the correlated insulator phase has 120° antiferromagnetic (AFM) order. Additionally, the mixed symmetry character of the SC phase near zero displacement field predicted by our model has experiment observable distinctions from that of moiré Hubbard model48,59,60,61,62,63,64,65,66,72,73,74,75,99,100 due to the absence of emergent spin-valley SU(2) symmetry72,73, providing new insights into the nature of the moiré TMD systems.

Results

Continuum model and symmetry analysis

We begin with the standard low-energy continuum model description of tWSe245. The non-interacting continuum Hamiltonian for the K valley is given by:

$${H}_{K}({{{\boldsymbol{r}}}})=\left(\begin{array}{cc}-\frac{{\hslash }^{2}{\left({{{\boldsymbol{k}}}}-{{{{\boldsymbol{\kappa }}}}}_{+}\right)}^{2}}{2{m}^{*}}+{\Delta }_{+}({{{\boldsymbol{r}}}})&{\Delta }_{{{{\rm{T}}}}}({{{\boldsymbol{r}}}})\\ {\Delta }_{{{{\rm{T}}}}}^{{{\dagger}} }({{{\boldsymbol{r}}}})&-\frac{{\hslash }^{2}{\left({{{\boldsymbol{k}}}}-{{{{\boldsymbol{\kappa }}}}}_{-}\right)}^{2}}{2{m}^{*}}+{\Delta }_{-}({{{\boldsymbol{r}}}})\end{array}\right)$$
(1)

where m* is the effective mass of valence band edge, κ± are located at corners of the mini Brillouin zone as shown in Fig. 1a. In the lowest-order harmonic approximation, we only keep terms with moiré wave vectors gj which are obtained by rotation of \({{{{\boldsymbol{g}}}}}_{1}=(\frac{4\pi }{\sqrt{3}{a}_{M}},0)\) by (j − 1)π/3 in moiré potential Δ±(r) and interlayer tunneling ΔT(r):

$${\Delta }_{\pm }({{{\boldsymbol{r}}}})=\pm \frac{{V}_{z}}{2}+2V{\sum }_{j=1,3,5}\cos \left({{{{\boldsymbol{g}}}}}_{j}\cdot {{{\boldsymbol{r}}}}\pm \psi \right)$$
(2)
$${\Delta }_{{{{\rm{T}}}}}({{{\boldsymbol{r}}}})=w\left(1+{e}^{-i{{{{\boldsymbol{g}}}}}_{2}\cdot {{{\boldsymbol{r}}}}}+{e}^{-i{{{{\boldsymbol{g}}}}}_{3}\cdot {{{\boldsymbol{r}}}}}\right)$$
(3)

and the Hamiltonian for  − K valley is related by time-reversal symmetry \({{{\mathcal{T}}}}\). Throughout this paper, we adopt the experimental relevant twist angle θ = 3.65°70, continuum model parameters derived by large scale ab initio simulations (V, ψ, w) = (9 meV, 128°, 18 meV)79 and the effective mass m* = 0.45me101. The moiré lattice constant is given by aM ≈ a0/θ with a0 = 3.317 Å102.

Fig. 1: Construction of the three-band tight-binding model from the continuum model by Wannierization.
Fig. 1: Construction of the three-band tight-binding model from the continuum model by Wannierization.
Full size image

a Mini Brillouin zone formed by a small twist angle θ between two layers. b Density distribution of Wannier functions on two layers, with unit cell shown as dashed line. c Comparison of the band structure between continuum model (dashed line) and tight binding model (solid line), where colors representing sublattice components.

In the absence of displacement field Vz = 0, tWSe2 system has D3 point group symmetry generated by threefold rotation C3z and twofold rotation C2y. Additionally, due to the lowest order harmonic approximation we adopted, the continuum model has additional pseudo-inversion symmetry \({{{\mathcal{I}}}}\) with σxHK(r)σx = HK( − r), enlarging the point group symmetry to D3d. When a finite displacement field Vz ≠ 0 is applied, the point group symmetry is reduced from D3d to C3v, breaking all symmetries that interchange the two layers. Apart from point group symmetries, the system also exhibits U(1) spin-valley symmetry and time-reversal symmetry \({{{\mathcal{T}}}}\) for both Vz = 0 and Vz ≠ 0. These symmetry considerations are crucial for the subsequent construction of tight binding model and the classification of SC pairing symmetries.

Wannier functions and tight-binding model

To study the low-energy physics, we first focus on the top moiré valence bands of the tWSe2 continuum model at Vz = 0. The Chern numbers for each valley share the same sign up to top five moiré bands, precluding any low-energy real-space description with correct band topology due to Wannier obstructions. Therefore, we focus on accurately reproducing the Chern numbers of the top two moiré bands, and construct a minimal three-band tight-binding model with Chern numbers (1, 1, − 2) for K valley. Following refs. 79,80,81,82, we construct sufficiently localized Wannier functions using their layer polarization properties and the location of Wannier centers. The resulting Wannier functions are shown in Fig. 1b, where the A and B orbitals are centered at XM and MX regions \((\pm {a}_{M}/\sqrt{3},0)\) with opposite layer polarization, and the C orbital is centered at MM region (0, 0) with layer hybridization. Detailed symmetry analysis81 reveals that the (A, B, C) orbitals for the K valley have C3z eigenvalues (e−i2π/3, e−i2π/3, 1), and \({C}_{2y}{{{\mathcal{T}}}}\) or \({{{\mathcal{I}}}}\) symmetries interchange A and B orbitals but leave C orbital invariant. Using these Wannier functions, we can construct the following tight-binding model:

$${H}_{0}={\sum }_{i\alpha j\beta \sigma }{t}_{i\alpha j\beta \sigma }{c}_{i\alpha \sigma }^{{{\dagger}} }{c}_{j\beta \sigma }$$
(4)

where i, j are unit cell indexes, α, β are sublattice indexes (A, B, C), and σ is spin-valley index ↑ (or K), ↓ (or  − K). By keeping up to fifth-nearest-neighbor hopping parameters (see Supplementary Section I), Fig. 1c illustrates the close matching between the band structure of the tight-binding model and the continuum model, especially for the top moiré valence band.

We then consider the case of Vz ≠ 0. Instead of repeating the above procedure for each Vz separately, we utilize the layer polarization properties of Wannier functions, modeling the effect of displacement field as:

$${H}_{D}=\frac{{{{{\mathcal{V}}}}}_{z}}{2}{\sum}_{i}({n}_{iA}-{n}_{iB})$$
(5)

where \({{{{\mathcal{V}}}}}_{z}\) is the energy difference between A and B sublattices induced by the displacement field, and \({n}_{i\alpha }={\sum }_{\sigma }{c}_{i\alpha \sigma }^{{{\dagger}} }{c}_{i\alpha \sigma }\) is the density operator. Since the energy expectation value of displacement field on the C sublattice can be chosen as 0, we neglect terms involving it in HD (see Supplementary Section II for a more detailed discussion of the displacement field effects). The Fermi surfaces of filling factor ν = − 1 with \({{{{\mathcal{V}}}}}_{z}=0\), 5, 15 and 25 meV are shown in Fig. 2a, where the Fermi surface of spin up and down are split by the displacement field \({{{{\mathcal{V}}}}}_{z}\), but related by the time-reversal symmetry \({{{\mathcal{T}}}}\). Figure 2b also illustrates the Fermi surface density of states (DOS) as a function of \({{{{\mathcal{V}}}}}_{z}\).

Fig. 2: Evolution of Fermi surfaces and density of states as a function of displacement field.
Fig. 2: Evolution of Fermi surfaces and density of states as a function of displacement field.
Full size image

a Fermi surfaces of free Hamiltonian at different displacement fields, with colors indicating the sublattice components, and thickness representing the DOS. The spin of the Fermi surfaces are labeled as solid arrows. When \({{{{\mathcal{V}}}}}_{z}=0\) meV, where spin up and down Fermi surfaces coincide, only spin down Fermi surface is shown. The approximate nesting wave vector Q between spin up and down Fermi surfaces are illustrated as dashed arrow. b The Fermi surface DOS of A, B, C sublattices as a function of displacement field \({{{{\mathcal{V}}}}}_{z}\).

To capture the many-body physics, we consider leading interactions in tWSe2. The dominant interaction should be the onsite Hubbard repulsion:

$${H}_{U}={\sum }_{i\alpha }{U}_{\alpha }{n}_{i\alpha \uparrow }{n}_{i\alpha \downarrow }$$
(6)

where α = A, B, C labels the sublattice, and \({n}_{i\alpha \sigma }={c}_{i\alpha \sigma }^{{{\dagger}} }{c}_{i\alpha \sigma }\). The positive Hubbard interaction can typically lead to magnetic ordering, providing a promising explanation of the correlated insulator phase. Longer-range Coulomb repulsions are neglected mainly because their magnitudes are much smaller (see Supplementary Section IV).

Due to the time-reversal symmetry \({{{\mathcal{T}}}}\) of the systems, the Fermi surface features Cooper instability; namely, SC instabilities can occur even with infinitesimal attractions. Considering that the Fermi surface shown in Fig. 2a favor pairing within the same sublattices and has almost no C-orbital component, a natural choice is to consider the attractions on NNN sites of the same A or B sublattices:

$${H}_{{V}_{2}}=-{V}_{2}{\sum}_{i}{\sum}_{\alpha \in \{A,B\}}{\sum}_{\delta \in \,{\mbox{NNN}}\,}{n}_{i+\delta \alpha }{n}_{i\alpha }$$
(7)

where V2 is the strength of NNN attraction, α is sublattice index of A and B, and δ represents one of the three NNN bond directions 120° apart. Besides, we have also considered the effects of nearest-neighbor (NN) attraction between A and B sublattices in Supplementary Section VI, where we have shown that it is less dominant. The relatively local NNN (or NN) attraction can be understood by a random phase approximation (RPA) analysis (see Supplementary Section V). Therefore, the interacting model which we should consider to describe the main physics of tWSe2 is denoted as \(H={H}_{0}+{H}_{D}+{H}_{U}+{H}_{{V}_{2}}\).

Mean-field analysis of SC and AFM

We start with the mean-field analysis of SC instabilities by decoupling the NNN attraction \({H}_{{V}_{2}}\) in the SC channel as:

$$\begin{array}{rcl}&&{H}_{{V}_{2}}\approx -{V}_{2}{\sum}_{i\sigma {\sigma }^{{\prime} }}{\sum}_{\alpha \in \{A,B\}}{\sum}_{\delta \in NNN}\left({\tilde{\Delta }}_{\alpha \sigma {\sigma }^{{\prime} }}^{*}(\delta ){c}_{i\alpha {\sigma }^{{\prime} }}{c}_{i+\delta \alpha \sigma }\right.\\ &&\left.+{c}_{i+\delta \alpha \sigma }^{{{\dagger}} }{c}_{i\alpha {\sigma }^{{\prime} }}^{{{\dagger}} }{\tilde{\Delta }}_{\alpha \sigma {\sigma }^{{\prime} }}(\delta )-{\tilde{\Delta }}_{\alpha \sigma {\sigma }^{{\prime} }}^{*}(\delta ){\tilde{\Delta }}_{\alpha \sigma {\sigma }^{{\prime} }}(\delta )\right),\end{array}$$
(8)

where \({\tilde{\Delta }}_{\alpha \sigma {\sigma }^{{\prime} }}(\delta )=\langle {c}_{i\alpha {\sigma }^{{\prime} }}{c}_{i+\delta \alpha \sigma }\rangle\) is the spatially uniform real-space pairing order parameter on NNN bond δ, as shown in Fig. 3a. To further determine the ansatz of the pairing order parameter \({\tilde{\Delta }}_{\alpha \sigma {\sigma }^{{\prime} }}(\delta )\), we analyze possible pairing symmetries, which should be classified by the irreducible representations of the symmetry group. Since the addition of displacement field \({{{{\mathcal{V}}}}}_{z}\) preserves time-reversal symmetry \({{{\mathcal{T}}}}\) but breaks pseudo-inversion symmetry \({{{\mathcal{I}}}}\), we only consider the Sz = 0 sector for the U(1) spin-valley symmetry (i.e. inter-valley pairing). And these Sz = 0 pairings can be further categorized into singlet pairing \({\Delta }_{\alpha }^{S}(\delta )=({\tilde{\Delta }}_{\alpha \uparrow \downarrow }(\delta )-{\tilde{\Delta }}_{\alpha \downarrow \uparrow }(\delta ))/\sqrt{2}\) and triplet pairing \({\Delta }_{\alpha }^{T}(\delta )=({\tilde{\Delta }}_{\alpha \uparrow \downarrow }(\delta )+{\tilde{\Delta }}_{\alpha \downarrow \uparrow }(\delta ))/\sqrt{2}\). For the point group symmetry C3v under finite displacement field, it is straightforward to show that only A1 (mixing s and f-wave) and E (mixing (px, py) and \(({d}_{xy},{d}_{{x}^{2}-{y}^{2}})\)–wave) representations are possible for NNN pairing. The SC pairing form factors, focusing only on chiral SC in the E representation, are illustrated in Fig. 3b, and we leave the discussion of nematic SC in the E representation in Supplementary Section VI, where we have shown it is subdominant.

Fig. 3: Illustration of the possible superconducting and antiferromagnetic order parameters.
Fig. 3: Illustration of the possible superconducting and antiferromagnetic order parameters.
Full size image

a The NNN SC order parameters ΔAA and ΔBB, where the A, B, C sites are labeled as red, green, blue dots. b The pairing form factor of s-wave, f-wave, p + ip-wave, and d − id-wave on NNN bonds, with the irreducible representation labeled. c 120° AFM pattern on a certain type of sublattice with wave vector  ± Q. d The folded Brillouin zone induced by the AFM order.

To understand the competing insulating phase, we decouple the onsite Hubbard repulsion HU in the channel of magnetic ordering in the xy-plane72,73 as:

$${H}_{U}= -{\sum}_{i\alpha }{U}_{\alpha }\left({m}_{i\alpha }{c}_{i\alpha \downarrow }^{{{\dagger}} }{c}_{i\alpha \uparrow }+{m}_{i\alpha }^{*}{c}_{i\alpha \uparrow }^{{{\dagger}} }{c}_{i\alpha \downarrow }\right)\\ +{\sum}_{i\alpha }{U}_{\alpha }| {m}_{i\alpha }{| }^{2}+\frac{1}{2}{\sum}_{i\alpha \sigma }{U}_{\alpha }{n}_{i\alpha \sigma }$$
(9)

where \({m}_{i\alpha }=\langle {c}_{i\alpha \uparrow }^{{{\dagger}} }{c}_{i\alpha \downarrow }\rangle=\langle {S}_{i\alpha }^{x}\rangle+i\langle {S}_{i\alpha }^{y}\rangle\) is the complex order parameter representing the in-plane magnetization. We further constraint the form of miα by noting that, as shown in Fig. 2a, although the Fermi surface deforms as the displacement field \({{{{\mathcal{V}}}}}_{z}\) changes, an approximate nesting between spin up and spin down Fermi surfaces persist. Moreover, the nesting wave vectors are relatively close to the commensurate ones  ± Q = (0, ± 4π/3aM) over a wide range of displacement field \({{{{\mathcal{V}}}}}_{z}\), giving raise to the 120° AFM order \({m}_{i\alpha }={m}_{\alpha }^{+}{e}^{iQ\cdot {R}_{i}}+{m}_{\alpha }^{-}{e}^{-iQ\cdot {R}_{i}}\) with two possible chiralities as shown in Fig. 3c, which are generally non-degenerate under finite \({{{{\mathcal{V}}}}}_{z}\). Such AFM order breaks translation symmetry of the system, folding the Brillouin zone as shown in Fig. 3d, with the new filling factor in terms of the folded Brillouin zone becoming \(\tilde{\nu }=-3\). Depending on details of the system, the ground state can be either metallic or insulating due to the nesting of  ± Q is non-perfect.

We then perform mean-field calculations for SC and AFM at filling ν = − 1 independently, using Hubbard repulsion UA = UB = UC = 37.5 meV and two different NNN attractions V2 = 10 meV or V2 = 12.5 meV as representing parameters. These interacting parameters can be estimated either by comparing the mean-field results with experimental observations, or by directly expanding the gate-screened Coulomb interaction onto Wannier functions (see Supplementary Sections III and IV). The resulting phase diagrams for tuning the displacement field \({{{{\mathcal{V}}}}}_{z}\) at filling ν = − 1 are summarized in Fig. 4e and f, respectively. We leave more detailed mean-field derivations and discussions in Supplementary Sections VI and VII.

Fig. 4: Mean-field analysis and phase diagrams of 3.65° tWSe2.
Fig. 4: Mean-field analysis and phase diagrams of 3.65° tWSe2.
Full size image

a The energy gain per hole of the ordered phases compared to the symmetric phase. b SC orders for V2 = 10 meV on one specific NNN bond δ0 under the gauge described in the main text. The sign of order parameters correspond to the sign of real (imaginary) part of singlet (triplet) pairings. c The AFM mean-field charge gap at filling factor \(\tilde{\nu }=-3\) in the folded Brillouin zone. d Magnitude of the AFM orders on A, B, C sublattices. e, f The phase diagrams for V2 = 10 meV and V2 = 12.5 meV respectively, indicating that under mean-field framework, the SC to AFM-I transition can either have an intermediate AFM-M phase or occur directly.

Topological superconductivity

We first focus on the SC phase in the zero- or small-displacement field regime. A comparison of the energy gain ΔE between possible SC orders and the 120° AFM order is shown in Fig. 4a, which indicates that the system is in the SC phase for \({{{{\mathcal{V}}}}}_{z} < {{{{\mathcal{V}}}}}_{z,c}\approx 2.2\) meV (3.4 meV) for V2 = 10 meV (12.5 meV), qualitatively consistent with the critical field \({{{{\mathcal{V}}}}}_{z,c}^{\exp }\approx 2.6\) meV observed in the experiment70. The pairing of this SC phase is chiral with mixed \({d}_{xy}\pm i{d}_{{x}^{2}-{y}^{2}}\) and px ∓ ipy wave in the E representation of C3v, spontaneously breaking the time-reversal symmetry \({{{\mathcal{T}}}}\)103. More importantly, such a chiral SC phase, which fits into the A-class (namely, breaking \({{{\mathcal{T}}}}\) but preserving the Sz) of the Altland-Zirnbauer tenfold classification scheme96,97,98, is topologically non-trivial with Chern number computed to be C = ± 2, suggesting tWSe2 as a promising candidate for realizing chiral topological SC. The internal structure of the SC order as a function of \({{{{\mathcal{V}}}}}_{z}\) for V2 = 10 meV is also illustrated in Fig. 4b, where the consistency constraint between time-reversal symmetry \({{{\mathcal{T}}}}\) and C3v mirror plane symmetry enables us to fix a simple gauge such that all singlet pairings \({\Delta }_{\alpha }^{S}(\delta )\) are real while all triplet pairings \({\Delta }_{\alpha }^{T}(\delta )\) are imaginary. It is worth emphasizing that, unlike the single-band moiré Hubbard model48,59,60,61,62,63,64,65,66,72,73,74,75,99,100, the mixing of singlet and triplet pairings is allowed for all \({{{{\mathcal{V}}}}}_{z}\) in our three-band model, especially for \({{{{\mathcal{V}}}}}_{z}=0\), where the emergent spin-valley SU(2) symmetry is absent and the pseudo-inversion symmetry \({{{\mathcal{I}}}}\) does not forbid such mixing. Also, the dominance of the \({d}_{xy}\pm i{d}_{{x}^{2}-{y}^{2}}\) component over the px ∓ ipy one in the SC phase is in accordance with the result of Chern number  ± 2. Moreover, SC is enhanced (suppressed) on the B (A) sublattice upon increasing the displacement field \({{{{\mathcal{V}}}}}_{z}\) from zero, closely following the trend of the Fermi surface DOS in Fig. 2b.

Antiferromagnetic correlated insulators

As the displacement field is further increased beyond a critical field \({{{{\mathcal{V}}}}}_{z,c}\), the system transitions into the 120° AFM phase. To further determine its transport properties, we compute the AFM mean-field charge gap for \(\tilde{\nu }=-3\). As shown in Fig. 4c, a broad intermediate range of the AFM insulator (AFM-I) phase appears for \({{{{\mathcal{V}}}}}_{z,c}\lesssim {{{{\mathcal{V}}}}}_{z} < {{{{\mathcal{V}}}}}_{z,c}^{{\prime} }\approx 21.5\) meV and an AFM metal (AFM-M) phase for \({{{{\mathcal{V}}}}}_{z} > {{{{\mathcal{V}}}}}_{z,c}^{{\prime} }\), closely matching the phenomenology of the experiment70. In the mean-field framework, depending on the value of V2 employed in the model, the SC to AFM-I transition either exhibits a tiny intermediate AFM-M phase as shown in Fig. 4e or occurs as a direct first-order transition as shown in Fig. 4f. To explain the transport evidence of continuous superconductor-insulator transition70, disorder could play an important role. The continuous transition into the SC phase might potentially be a percolation transition104,105 of local SC regions induced by disorders or a disorder-rounded first-order transition. We also present the magnitude of AFM orders on different sublattices as a function of \({{{{\mathcal{V}}}}}_{z}\) in Fig. 4d. Except for small \({{{{\mathcal{V}}}}}_{z}\), where the AFM order is stronger on the B sublattice due to its larger DOS (see Fig. 2b), the AFM order generally favors the A sublattice, as the holes are mostly concentrated there in response to the displacement field \({{{{\mathcal{V}}}}}_{z}\), as well as its better approximation for the commensurate nesting wave vector Q. And the sudden drop of AFM orders at large \({{{{\mathcal{V}}}}}_{z}\) regime coincides with the disappearance of B sublattice hole pockets at κ± points and the sharp decline in B sublattice DOS, as illustrated in Fig. 2, indicating that the holes on B sublattice play a crucial role in mediating the AFM order.

Discussion

In summary, we have constructed a three-band tight-binding model through direct Wannierization, and incorporated onsite Hubbard repulsion and NNN attraction to explain the SC and correlated insulator phase observed in the 3.65° tWSe2 at filling ν = − 170. Our mean-field analysis indicates that, the SC phase is an A-class topological SC with Chern number C = ± 2, featuring the inter-valley mixed \({d}_{xy}\pm i{d}_{{x}^{2}-{y}^{2}}\) and px ∓ ipy-wave pairing symmetry, and the correlated insulator phase is explained by the 120° AFM order. We further demonstrate that the metallic behavior when the filling is away from ν = − 1 can also be qualitatively understood within our mean-field framework (see Supplementary Section VIII). Compared with the 5° tWSe2 system71, the experimental features show notable similarities despite differences in low-energy band structure and interaction strength82. Recent theoretical advances106 suggest the possibility of a unifying underlying mechanism, motivating future efforts toward a unified description.

The topological band structure in our model can be crucial for understanding how SC arises for flat band systems, since non-trivial lower bounds of SC superfluid weight exist107,108,109,110,111,112 due to quantum geometric effects, which deserves more detailed future theoretical studies. Moreover, our results also suggest that the tWSe2 homobilayers could provide new possibilities for realizing topological SC, which call for more detailed experimental studies to further uncover its nature. Identifying the topological edge states by measuring quantized thermal Hall conductance113,114,115, or verifying the chiral nature of the SC pairing symmetry through the phase-sensitive Josephson junction116,117,118, will surely open new opportunities in twisted TMD systems.

Note added

In finishing the present work, we noticed that refs. 119,120,121,122,123,124 appeared, which also investigated the SC and correlated insulators in tWSe2 reported in ref. 70, although there are important differences between those studies and the present one.

Methods

We construct a three-band tight-binding model by a direct Wannierization of the continuum model of 3.65° twisted bilayer WSe2, with the resulting hopping parameters given in Supplementary Information Section I. The model incorporates onsite Hubbard repulsion and nearest- or next-nearest-neighbor attractions. A detailed estimation of the Hubbard interaction parameters is provided in Supplementary Information Sections III and IV, while a microscopic explanation of the attractive terms is presented by the random phase approximation calculation in Supplementary Information Section V. The interacting model is analyzed within the standard mean-field framework, with full derivations and self-consistent procedures provided in Supplementary Information Sections VI and  VII.