Fig. 7: Calculated nonorthogonality, effective interaction parameters, and immiscibility for two dressed spin states.

In (a–f), the calculations consider \(\left| { \uparrow \prime } \right\rangle\) and \(\left| { \downarrow \prime } \right\rangle\) located respectively at ħq_y and −ħq_y. a When Ω = 0, the nonorthogonality is zero because the two bare spin components are orthogonal. When Ω ≠ 0, either increasing Ω or decreasing q_y would increase 〈↑′|↓′〉, giving rise to stronger interference and more significant density modulations in the spatially overlapped region of the two dressed spin components. b, c Effective interspecies (g_↑′↓′) and intraspecies (g_↑′↑′, g_↓′↓′) interaction parameters versus quasimomentum at Ω = 0.1 E_r and 1.26 E_r, respectively. When Ω increases or q_y decreases, g_↑′↓′ increases while g_↑′↑′ and g_↓′↓′ almost remain at the bare values. As q_y → 0 at any finite Ω, g_↑′↓′ → 2g_↑′↑′ or 2g_↓′↓′, which is the upper bound of g_↑′↓′ (see Methods). The inset of b, c zooms out to show the maximum. d shows the immiscibility metric \(\eta = \left( {g_{ \uparrow \prime \downarrow \prime }^2 - g_{ \uparrow \prime \uparrow \prime }g_{ \downarrow \prime \downarrow \prime }} \right){\mathrm{/}}g_{ \uparrow \uparrow }^2\) in Eq. (13) (see Methods) versus ħq_y corresponding to b. η < 0 means miscible, and η > 0 means immiscible. Over the range of plotted ħq_y, d can be miscible or immiscible depending on ħq_y. The inset of d zooms in to focus on the sign change of η. The vertical dotted line in (b–d) indicates ħq_{σ min} corresponding to the Ω in each case. The calculations are performed in the two-state picture described by Eq. (1) with δ_R = 0. e, f Immiscibility metric η versus Ω for various q_y. In e, as Ω becomes larger or q_y becomes smaller, the two dressed spin components can become more immiscible until η reaches the maximum value set by the upper bound of g_↑′↓′ (see also b, c). f Zoom-in of e showing the miscible to immiscible transition (indicated by the gray dashed line at η = 0) as a function of Ω for various q_y. The red dot-dashed line corresponds to two dressed spin components located respectively at the band minima q_{σ min}, showing the well-known miscible to immiscible transition around 0.2 E_r for a stationary SO-coupled BEC. In the dynamical case studied here, BECs can be located away from the band minima and approach q_y = 0, becoming immiscible even when Ω < 0.2 E_r for small enough q_y