Introduction

A mutual coupling between various degrees of freedom such as spin, charge, lattice, and orbit gives rise to novel bound states, many-body phenomena, hybrid quasiparticle excitations, spin-orbit entangled excitons, and strong electron correlations1,2,3,4,5,6,7,8. Understanding the correlation between excitons and magnetic ordering in magnetic materials remains a fundamental challenge in the field of magneto-optics, spintronics, and quantum magnetism. In this direction, the spin-orbital entangled Zhang–Rice (ZR) state9 has emerged as a promising excitonic state, highlighting the significant correlation between magnetism and electronic/optical degrees of freedom in dictating magneto-optical phenomena10,11.

Among the various van der Waals (vdW) magnetic 2D materials, transition-metal thiophosphates MPX3 have ushered in a new era of research in 2D magnetism due to their persistence of antiferromagnetism down to the atomically thin limit12,13. Within the MPX3 family, NiPS3, a self-doped negative charge-transfer antiferromagnetic insulator (\({T}_{{\rm{N}}} \sim 150{-}155\,{\rm{K}}\))14,15 with an optical band gap of \(\sim 1{\rm{.8}}\,{\rm{eV}}\)7, offers a unique platform to explore the magneto-optical phenomena and the interplay between excitons and magnetic ordering. Recent studies have reported an extremely sharp and ultranarrow linewidth (\(\sim 200{-}400\,\mu {\rm{eV}}\)) excitonic emission in NiPS36,16,17,18, which is extremely narrower than exciton emissions observed in other 2D magnetic materials19 as well as non-magnetic transition metal dichalcogenides (e.g. MoS2 and WS2)20. This excitonic emission in NiPS3 strongly couples with cavity photons, resulting in the formation of exciton-polaritons8, and also couples with phonons, leading to the creation of exciton-phonon bound sidebands6. However, the origin of the exciton emission in NiPS3 is still controversial, with different interpretations proposed, including spin-orbital entangled ZR excitons, defect-bound excitons, Hund excitons, and spin-flip induced d-d emission4,5,11,21,22,23. Among these different scenarios, the ZR exciton can arise from the transition between ZR triplet and ZR singlet states11,16,18.

Beyond the parent compound NiPS3, recent investigations on transition-metal site alloying have shown a rapid quenching of ZR exciton emission and a significant broadening of its ultra-narrow linewidth, with only a few percentage of substitution of either non-magnetic atoms (Ni1-xCdxPS3) or magnetic atoms (Ni1-xMnxPS3) at the Ni atom site, suggesting that cation substitution drastically destabilizes the ZR exciton24,25. Given this strong dependence on the cation substitution, an ensuing question arises: Can the ZR exciton also be influenced by chalcogen (anion) substitution? Understanding the role of anion substitution could provide deeper insights into the decoherence and mechanism of excitonic and magneto-optical properties in NiPS3-based systems.

To address this question and further investigate the impact of anion substitution on the ZR exciton, in this work, we perform temperature- and polarization-dependent PL measurements on Se-substituted NiPS3-xSex (x = 0.008, 0.03, 0.06, and 0.09). For x = 0.008 Se-doped compound, we observe that the ZR exciton preserves its ultra-narrow linewidth and strong PL emission, similar to the parent NiPS3. Surprisingly, with increasing Se content above \(x=0.008\), the ZR exciton is dramatically suppressed and significantly broadens, yet it survives up to our highest Se doping level of \(x=0.09\) studied. In contrast, magnetic ground state and magnetic excitations are not much affected by increasing Se concentration. This distinct impact of anion substitution on excitonic versus magnetic properties suggests that decoherence effects are highly sensitive to electronic heterogeneity and that excitonic mechanisms based on intra-atomic interactions or localized magnetic d-orbital transitions are less consistent with our observations.

Results and discussion

Characterization: magnetic properties and magnon and phonon excitations

Bulk NiPS3 crystallizes in a monoclinic structure with space group \(C2/m(\#12)\) and point group \({C}_{2h}\). Figure 1a shows both the crystal and magnetic structures of NiPS3, in which Ni atoms form a honeycomb lattice with zigzag chains of magnetic moments. Each Ni atom is surrounded by six S atoms, forming NiS6 octahedra. Two P atoms, located above and below the Ni atom plane, are covalently bonded to six S atoms, forming a \({[{P}_{2}{S}_{6}]}^{4-}\) anion cluster. NiPS3 undergoes an antiferromagnetic phase transition below \({T}_{{\rm{N}}} \sim 150-158\,{\rm{K}}\), where the spins align ferromagnetically along zigzag chains parallel to the crystallographic a-axis with a slight out-of-plane component due to weak magnetic anisotropy along the b-axis14,15,26,27,28. These ferromagnetic zigzag chains are coupled antiferromagnetically within the ab plane, see Fig. 1a. NiPSe3 shares a similar magnetic structure as NiPS3, showing magnetic ordering below \({T}_{{\rm{N}}} \sim 212\,\,{\rm{K}}\)15.

Fig. 1: Magnetic susceptibility, XRD and Raman characterizations.
Fig. 1: Magnetic susceptibility, XRD and Raman characterizations.
Full size image

a Crystal and magnetic structure of NiPS3. Red and green arrows indicate the spin orientations of Ni atoms in the magnetically ordered phase. b XRD patterns for the NiPS3-xSex series. c Temperature dependence of the magnetic susceptibility for NiPS3-xSex along the ab plane measured under an external field of \({\mu }_{0}H=1\,{\rm{T}}\). The inset in (c) shows the temperature-dependent derivative of the magnetic susceptibility. The red dashed line in (c) and the inset marks the Néel temperature (\({T}_{{\rm{N}}} \sim 156\,{\rm{K}}\)). d Unpolarized Raman spectra of NiPS3-xSex, collected at \(3.5\,{\rm{K}}\) using \(532\,{\rm{nm}}\,(2.33\,{\rm{eV}})\) excitation. The spectra are vertically shifted for clarity, see Supplementary Fig. 1 for spectra plotted on the same scale. The observed phonon excitations in the spectral range of \(100\!-\!650\,{\rm{cm}}^{-1}\) are labeled as \({B}_{g}^{1-5}\),\({A}_{g}^{1-3}\), R and P1-P2. The gray-shaded area corresponds to the two-magnon (\(2M\)) continuum.

To verify phase purity, single-crystal X-ray diffraction (XRD) patterns of NiPS3-x Sex were collected using a Bruker-AX (D8-Advance) X-ray diffractometer with Cu Kα radiation (λ = 1.54 Å) at room temperature. Figure1b presents the XRD data collected with the X-ray beam perpendicular to the (00 l) planes, revealing the characteristic parallel planes of the monoclinic crystal structure. With increasing Se concentration at the S site, a systematic low-angle shift of the (00 l) peaks is observed, see right panel of Fig.1b, indicating a slight elongation of the c-axis due to the larger atomic radius of Se compared to S. This trend is consistent with previous reports15. An inspection of the XRD patterns (right panel of Fig. 1b) reveals two closely overlapping peaks, especially at x = 0.03, and 0.06, which are indicative of local compositional inhomogeneities likely arising from S-rich and Se-rich regions.

We also measured the temperature-dependent magnetic susceptibility \(\chi (T)\) of NiPS3-xSex, as shown in Fig. 1c with an external magnetic field of \({\mu }_{0}H=1\,{\rm{T}}\) applied along the ab plane. For all samples, we obtained the Néel temperature around \(\sim 156\,{\rm{K}}\), from the temperature-dependent derivative of the magnetic susceptibility, see inset in Fig. 1c, which shows good agreement with previous studies14,15,27,28. Notably, with increasing x, the magnetic transition becomes sharper, and the static magnetic susceptibility above TN is significantly enhanced. The substitution of Se for S replacing a smaller ligand with a larger one can enhance the superexchange interaction through increased orbital overlap between Ni and Se orbitals and suppress XY-like quantum fluctuations, accounting for the pronounced x dependence of \(\chi (T)\).

We next turn to Raman characterizations to examine magnetic and lattice excitations. Figure 1d illustrates the unpolarized Raman spectra of NiPS3-xSex collected at T = 3.5 K. We observe five \({B}_{g}\) and three \({A}_{g}\) peaks in the frequency range of \(100\,{\rm{to\,65}}0{{\,\rm{cm}}}^{-1}\), which are labeled as \({B}_{g}^{k}\,(k=1-5)\) and \({A}_{g}^{i}\,(i=1\!-\!5)\), respectively, and are in line with one-phonon modes of parent NiPS313,29,30, see Supplementary Note 1 for details on the expected phonons at the Brillouin zone center and polarization selection rules. We note that the \({B}_{g}^{k}\,(k=1{-}5)\) phonons are generally a combination of the two nearly degenerate phonon modes with the same frequency but different symmetries (\({A}_{g}\) and \({B}_{g}\)) and have also been labeled as \({A}_{g}\) and \({B}_{g}\) in earlier Raman studies on NiPS3. Here, we adopt the \({B}_{g}^{k}\,(k=1{-}5)\) notation for consistency. The \({B}_{g}^{1}\) and \({B}_{g}^{2}\) phonon modes involve the vibrations of the Ni2+ ions, while the higher-frequency phonons above \(\sim 200\,{{\rm{cm}}}^{-1}\) are expected to be associated with the vibrations of the \({[{P}_{2}{S}_{6}]}^{4-}\) cluster. Furthermore, we observed two additional phonons labeled as \({P}_{1}\) and \({P}_{2}\), which are likely related to the vibrations of the \({[{P}_{2}S{e}_{6}]}^{4-}\) cluster in NiPSe3. Additionally, a broad and pronounced continuum centered around \(\sim 530\,{{\rm{cm}}}^{-1}\) is attributed to the two-magnon (\(2M\)) excitation, which arises from double-spin–flip process with equal but opposite momenta13. A broad peak labeled as R, centered at ~446 cm-1, can be ascribed to an infrared-active mode with \({B}_{u}\) symmetry31. Additionally, we observe a very weak feature labeled as #, appearing near ~225 cm-1, which may arise from local lattice inhomogeneities (see the right panel of Fig. 1b).

Robustness of two-magnon excitations against anion substitution

Before discussing the dynamics of the ZR excitonic emission, we first address the Se substitution effect on the \(2M\) behavior and its coupling with phonons. Figure 2a–c presents the unpolarized color contour maps of the Raman intensity versus Raman shift across a temperature range of \(3.5\!-\!330\,{\rm{K}}\) for the \(x=0.03,0.0{\rm{6\,and}}\,0.09\) samples. The sharp lines correspond to the phonon excitations, while the broad continuum corresponds to the \(2M\) excitation. As the temperature increases above ~100 K, the \(2M\) excitation softens and broadens, eventually evolving into paramagnons above TN. We carefully extracted the self-energy parameters of \(2M\) such as frequency and linewidth up to \(\sim 200\,{\rm{K}}\) using the Lorentzian function after subtracting phonon excitations. Figure 2d and Supplementary Fig. 2 summarize the temperature dependence of the \(2{M}^{\text{'}}{\rm{s}}\) frequency, linewidth, and intensity. We observe no appreciable x dependence in its frequency and linewidth, although a slight suppression in its intensity occurs with increasing x. Overall, the \(2M\) excitations remain robust under Se substitution, at least up to x ~ 0.09. This is fully consistent with the absence of variation in TN up to x = 0.09.

Fig. 2: Thermal and Se-substituted evolution of two-magnon excitation.
Fig. 2: Thermal and Se-substituted evolution of two-magnon excitation.
Full size image

Unpolarized 2D color contour maps of Raman intensity versus Raman shift as a function of temperature for (a) x = \(0.03\) (b) \(0.06\) and (c) \(0.09\) of NiPS3-xSex. Sharp lines represent phonon excitations, while the broad continuum represents the \(2M\) excitation. d Frequency of \(2M\) as a function of temperature. e, f Fano resonance coupling coefficient \(1/|q|\) of the \({B}_{g}^{2}\) and \({B}_{g}^{5}\) mode as a function of temperature. The red horizontal lines in (a–c) and vertical dashed lines in (d–f) denote the antiferromagnetic phase transition temperature (TN).

To further track the \(2M\) excitations up to our highest recorded temperature, we examine the evolution Fano asymmetry of the \({B}_{g}^{2} \sim 180\,{{\rm{cm}}}^{-1}\) and \({B}_{g}^{5} \sim 560\,{{\rm{cm}}}^{-1}\) phonons, see Supplementary Fig. 3. Notably, we observe the asymmetric line shape for the \({B}_{g}^{5}\) (\({B}_{g}^{2}\)) mode at low (high) temperatures, which suggests the presence of Fano resonance32,33, resulting from the coupling between the \({B}_{g}^{5}\) (\({B}_{g}^{2}\)) phonon and the \(2M\) excitations in the respective temperature regimes. Quantitatively, the coupling strength of this coupling can be described by fitting the asymmetric \({B}_{g}^{2}\) and \({B}_{g}^{5}\) Raman peaks using the Breit–Wigner Fano function32 \({I}_{BWF}(\omega )\propto {(1+\delta /q)}^{2}/(1+{\delta })^{2}\), where \(\delta =\omega -{\omega }_{0}/\varGamma\) and \(\varGamma\) and \({\omega }_{0}\) are the linewidth and frequency of the uncoupled phonon, respectively. The asymmetric parameter \(1/|q|\) quantifies the coupling strength between the phonon and electronic/magnetic continuum excitations. In the strong-coupling limit \(1/|q|\to \infty\), the phonon line shape becomes increasingly asymmetric. As shown in Fig. 2f, \(1/|q|\) of the \({B}_{g}^{5}\) mode, located on the higher-energy side of the \(2M\) continuum, shows a gradual decrease above ~100 K. This thermal evolution resembles the temperature dependence of the \(2M\) peak frequency. Conversely, \(1/|q|\) of the \({B}_{g}^{2}\) mode, which lies on the lower-energy side of the \(2M\) continuum, starts to increase for temperatures above \({T}_{{\rm{N}}}\), as shown in Fig. 2e. This behavior suggests that the \({B}_{g}^{2}\) Fano phonon reflects the thermal damping of magnons into paramagnons as the system transitions through \({T}_{{\rm{N}}}\). Noteworthy is that the Fano parameters of both \({B}_{g}^{5}\) and \({B}_{g}^{2}\) modes exhibit little variation with x, confirming the robustness of the \(2M\) signals against anion substitution up to the concentration of x ∼ 0.09.

Anion-substitution and temperature dependence of PL emission

Next, we turn to the effect of anion substitution on the dynamics of the ZR exciton. To check the homogeneity of the samples, we collected PL spectra at T = 3.5 K (see Supplementary Fig. 6) from distinct spots that cover several micrometers across the sample. The consistent PL emission observed from all spots excludes the presence of micrometer-sized inhomogeneities. Figure 3a depicts the PL spectrum of the NiPS3-xSex collected at \(3.5\,{\rm{K}}\) using λ= \(532\,{\rm{nm}}\,(2{\rm{.33\,eV}})\) laser excitation. The PL spectrum for \(x=0.008\) shows a sharp and ultra-narrow PL emission (labeled as \({X}_{Z{R}_{2}}\)) located at \(\sim 1{\rm{.47543\,eV}}\), with a linewidth of \(\sim 389{\rm{.2}}\,{\rm{\mu }}{\rm{eV}}\). This PL peak can be attributed to the main ZR excitonic emission11,24,25. In addition to the primary ZR excitonic peak, a very weak peak labeled as \({X}_{Z{R}_{1}}\) is observed at \(\sim 1{\rm{.47309\,eV}}\) and is found to be an order of magnitude broader (\(\sim 1{\rm{.82791\,meV}}\)) compared to the \({X}_{Z{R}_{2}}\) peak, see Table 1 for detailed peak positions and linewidths/full width at half maximum (FHWM).

Fig. 3: Se-substituted modulation of ZR exciton.
Fig. 3: Se-substituted modulation of ZR exciton.
Full size image

a Unpolarized PL spectra of NiPS3-xSex, collected at \(3.5\,{\rm{K}}\) as a function of Se concentration using \({\rm{532\,nm}}\,(2{\rm{.33\,eV}})\) excitation. \({X}_{Z{R}_{2}}\) and \({X}_{Z{R}_{1}}\) correspond to the primary and secondary ZR PL emissions, respectively. b Peak energy of \({X}_{Z{R}_{2}}\) and \({X}_{Z{R}_{1}}\) versus Se content. c Energy difference between the \({X}_{Z{R}_{2}}\) and \({X}_{Z{R}_{1}}\) peaks versus Se content. d, e FWHM and integrated intensity of \({X}_{Z{R}_{2}}\) and \({X}_{Z{R}_{1}}\) as a function of Se content, respectively.

Table 1 Peak energy and FWHM of the \({X}_{Z{R}_{1}}\) and \({X}_{Z{R}_{2}}\) excitonic emission for NiPS3-xSex
Full size table

A striking difference between these excitons is their opposite energy shifts with Se substitution. The \({X}_{Z{R}_{2}}\) (\({X}_{Z{R}_{1}}\)) peak shows a blue (red) shift with increasing x, which leads to an increase in energy separation between these two excitonic features, see Fig. 3b, c. This intriguing opposite trend is further confirmed by PL spectra collected from different spots of the sample (see Supplementary Fig. 7), ruling out chemical inhomogeneities as its origin. Given the small energy difference between \({X}_{Z{R}_{1}}\) and \({X}_{Z{R}_{2}}\), the secondary peak cannot be ascribed to an exciton-phonon sideband. Rather, it appears to be an intrinsic property of the material, possibly arising from the presence of two slightly different electronic and magnetic states generated by Se substitution. As the Se concentration increases from \(x=0.008\) to \(x=0.03\), the main \({X}_{Z{R}_{2}}\) peak is drastically suppressed, followed by a gradual decrease in intensity with further substitution up to x = 0.09 (see Fig. 3a, e). In contrast to \({X}_{Z{R}_{2}}\), \({X}_{Z{R}_{1}}\) is relatively less affected by anion substitution, showing a gradual intensity decrease with increasing x, see Fig. 3e. On the other hand, both \({X}_{Z{R}_{1}}\) and \({X}_{Z{R}_{2}}\) undergo a quasi-linear broadening with increasing Se concentration, see Fig. 3d.

Next, we examine the thermal effect of the ZR exciton. Figure 4a–d shows the temperature-dependent PL color maps for NiPS3-xSex (see Supplementary Fig. 8 for the temperature evolution of the PL spectra). For \(x=0.008\), the intensity of the \({X}_{Z{R}_{2}}\) remains nearly temperature-independent up to \(40\,{\rm{K}}\) and then gradually decreases with further temperature rise and completely vanishes above \({T}^{\ast } \sim 120\,{\rm{K}}\), which is \(\sim 30\,{\rm{K}}\) lower than \({T}_{{\rm{N}}}\). The linewidth of the \({X}_{Z{R}_{2}}\) increases with temperature, accompanied by a red shift in its peak energy. Furthermore, the vanishing temperature of the \({X}_{Z{R}_{2}}\) exciton decreases progressively with increasing Se concentration: \({T}^{\ast } \sim 110{-}100\,{\rm{K}}\) for \(x=0.03\), \({T}^{\ast } \sim 100\,{\rm{K}}\) for \(x=0.06\), and \({T}^{\ast } \sim 90\,{\rm{K}}\) for \(x=0.09\). \({X}_{Z{R}_{1}}\) shows a similar temperature dependence as \({X}_{Z{R}_{2}}\), but it vanishes at a lower temperature than the primary peak. This disappearance of the ZR exciton at temperatures much below \({T}_{{\rm{N}}}\) reflects that the ZR exciton is related to the magnetic ordering, but additional mechanisms are involved in the destabilization of the ZR exciton at much lower temperatures than \({T}_{{\rm{N}}}\), which will be discussed later.

Fig. 4: Temperature-dependent dynamics of ZR exciton.
Fig. 4: Temperature-dependent dynamics of ZR exciton.
Full size image

a–d Unpolarized 2D color contour maps of PL intensity versus PL energy and as a function of temperature for NiPS3-xSex. \({T}^{\ast }\) is the temperature, above which \({X}_{Z{R}_{2}}\) becomes too weak to detect. e, f Temperature-dependent peak energy and FWHM of \({X}_{Z{R}_{2}}\) and \({X}_{Z{R}_{1}}\) for \(x=0.008\), respectively. The solid lines in (e) and (f) represent the fitted curves as described in the text.

To gain quantitative insight into the thermal effect on the ZR exciton, we plot the peak energy and linewidth as a function of temperature for \(x=0.008\) in Fig. 4e, f, respectively. Generally, the temperature dependence of excitonic emission energy in semiconductors is described using the empirical relation suggested by Donnell and Chen34, which accounts for electron (exciton)-phonon interactions in exciton energy shifts with temperature:

$$E(T)={E}_{0}-S{E}_{{\rm{p}}}\,\left[\coth \,\left(\frac{{E}_{{\rm{p}}}}{2{k}_{{\rm{B}}}T}\right)-1\right]$$
(1)

where \({E}_{0}\) is the exciton energy at \(T={\rm{0\,K}}\), \({E}_{p}=\langle \hslash \omega \rangle\) is the average phonon energy contributing to the shift in exciton energy, \({k}_{{\rm{B}}}\) is the Boltzmann constant, and \(S\) is the dimensionless Huang–Rhys factor describing the strength of exciton-phonon coupling. The solid red lines in Fig. 4e are the fitting curve using Eq. (1), with the best-fit parameters listed in Table 2. Moreover, the temperature-dependent linewidth of the exciton can also be described by considering exciton-phonon interactions. At finite temperatures, the temperature-dependent linewidth of the exciton is given as35

$$\varGamma (T)={\varGamma }_{0}+\,\frac{{\lambda }_{LO}}{(\exp \,({\Theta }_{LO}/T)-1)}$$
(2)

where \({\varGamma }_{0}\) is the exciton linewidth at \(T={\rm{0\,K}}\) which arises from scattering of excitons with impurities and imperfections. The second term is associated with the linewidth broadening due to the exciton scattering with longitudinal optical (\(LO\)) phonons, where \({\lambda }_{LO}\) represents the exciton-LO phonon coupling strength, and \({\Theta }_{LO}\) denotes the LO phonon energy. The solid red lines in Fig. 4f are the fitted curves using Eq. (2), and the fit parameters are listed in Table 2. The aforementioned models as given in Eqs. (1) and (2) are in good agreement with the experimental data, reflecting that the exciton-phonon mechanism seems to provide a consistent description of the observed energy redshift and linewidth broadening with increasing temperature. Nonetheless, it is far from clear whether the exciton-phonon mechanism alone can fully account for the thermal behavior of the ZR exciton, particularly given the inability to obtain reliable parameters for \(x=0.03-0.09\). We further highlight the anomalies observed in both the PL peak energy and FWHM, see Supplementary Fig. 9. For x = 0.03 – 0.09, the excitonic emissions exhibit splitting into \({X}_{Z{R}_{1}}\) and \({X}_{Z{R}_{2}}\) in the temperature range of 40 K and 60 K, alluding to the presence of an additional energy scale below the magnetic ordering temperature.

Table 2 List of fitting parameters obtained from the fitting of temperature-dependent exciton energy and linewidth for \(x=0.008\) as described in the text
Full size table

Linearly polarization-dependent PL

As aforementioned, the ferromagnetic zigzag spin chains along with the Néel vector align along the a-axis. It is extremely important to figure out the correlation between the ZR excitonic emission and the spin structure in the chalcogen-substituted NiPS3. An intriguing characteristic of the ZR excitonic emission in NiPS3 is that it shows a maximum intensity when the collection polarization is perpendicular to the crystallographic a-axis (Néel vector), indicating a highly anisotropic linear polarization of the ZR excitonic emission17. To explore this correlation in Se-mixed NiPS3, we conducted linear polarization-dependent PL measurements on the \(x=0.008\) and \(x=0.03\) samples in a parallel configuration. Simultaneously, we also performed linearly polarized Raman measurements on the same sample spot to determine the crystal symmetry and the crystallographic axis. For the higher-x samples, the limited statistics hinder analysis of the detailed angular dependence.

Figure 5a sketches the schematic representation of our experimental setup for linearly polarized Raman and PL measurements. Figure 5b, c shows the intensity polar plots for Raman-active phonons \({B}_{g}^{1}\) and \({B}_{g}^{2}\) for \(x=0.03\) collected at \(3.5\,{\rm{K}}\) in the parallel configuration (\({\hat{e}}_{i}\parallel {\hat{e}}_{s}\)) after correcting the initial orientation of the crystal axis, see Supplementary Note 1 for the Raman selection rules and as-measured uncorrected polarized Raman spectra, as well as intensity polar plots of the phonon excitations in Supplementary Fig. 4 for \(x=0.03\) and Supplementary Fig. 5 for \(x=0.008\). The observed splitting of \({B}_{g}^{2}\) into two components is not surprising, as such splitting is expected in the magnetically ordered phase due to the breaking of the three-fold rotation symmetry of the lattice13. Both split components show a four-lobed symmetry, which are in good agreement with the earlier linearly polarized Raman studies6. Figure 5d shows the PL spectra at 3.5 K for two configurations, i.e., parallel (\({\hat{e}}_{i}\parallel {\hat{e}}_{s}\)) and perpendicular (\({\hat{e}}_{i}\perp {\hat{e}}_{s}\)) configurations. We observe the highly anisotropic excitonic emission with a nearly unity degree of linear polarization, implying a strong coupling between the ZR exciton and zig-zag spin chain structure6. Unlike in NiPS3 nanoflakes36, we find no evidence of three distinct magnetic domains in the Se-substituted compounds. In this context, the polarization anisotropy observed in the ZR exciton should be understood as a net signal, likely dominated by a majority domain, which may be preferentially stabilized due to monoclinic stacking or defects. Next, we plot the intensity polar plot for \({X}_{Z{R}_{1}}\) and \({X}_{Z{R}_{2}}\) as a function of polarization angle in Fig. 5e, f, measured in the parallel configuration, after correcting the orientation of the crystal axis according to the intensity polar plot of the \({B}_{g}^{1}\) and \({B}_{g}^{2}\) modes, see Supplementary Fig. 10b for the uncorrected polarization-angle-dependent PL spectrum. The linear polarization PL results for \(x=0.008\) are shown in Supplementary Fig. 10a and Supplementary Fig. 11. For \(x=0.008\) and \(x=0.03\), both of the \({X}_{Z{R}_{1}}\) and \({X}_{Z{R}_{2}}\) excitonic emissions show the maximum (minimum) intensity along the b-axis (a-axis/Néel vector), being in line with what was observed in parent NiPS36,17. In conjunction with the temperature dependence of \({X}_{Z{R}_{1}}\) and \({X}_{Z{R}_{2}}\), we infer that the two excitons are closely tied to the underlying spin structure, confirming their magnetic origin.

Fig. 5: Linear polarization-dependent Raman and PL results for x = 0.03.
Fig. 5: Linear polarization-dependent Raman and PL results for x = 0.03.
Full size image

a Schematic representation of the experimental setup for linearly polarized Raman and PL measurements with polarization directions of incident and scattered light. b, c Intensity polar plot for the Raman phonons \({B}_{g}^{1}\) and \({B}_{g}^{2}(1,2)\) for \(x=0.03\), collected at \(3.5\,{\rm{K}}\) in the parallel configuration (\({\hat{e}}_{i}\parallel {\hat{e}}_{s}\)), after the corrections of the initial orientation of the crystal axis, respectively. d PL spectra for \({\hat{e}}_{i}\parallel {\hat{e}}_{s}\) (red) and \({\hat{e}}_{i}\perp {\hat{e}}_{s}\) (black) for \(x=0.03\) at \(3.5\,{\rm{K}}\). e, f Intensity polar plot of the excitonic peaks \({X}_{Z{R}_{2}}\) and \({X}_{Z{R}_{1}}\) for \(x=0.03\) in the parallel configuration at 3.5 K, after correcting the initial crystal axis orientation, respectively. The red arrows in (f) and (e) represent the Néel vector, which aligns along the a-axis. The angular dependence of the \({B}_{g}^{2}(1)\) an \({B}_{g}^{2}(2)\) intensities are described by \(I(\theta )={e}^{2}{\cos }^{2}(2\theta )\) and \(I(\theta )={e}^{2}{\sin }^{2}(2\theta )\), respectively, while the excitonic peaks \({X}_{Z{R}_{2}}\) and \({X}_{Z{R}_{1}}\) follow \(I(\theta )={I}_{0}{\sin }^{2}(\theta )\).

Discussion

We have investigated the dynamics of the ZR exciton by combining temperature- and linearly polarization-dependent PL measurements for NiPS3-xSex. For \(x=0.008\), the primary ZR exciton is found to have similar characteristics (ultra-narrow linewidth and intense PL emission) as observed in the parent NiPS3. Surprisingly, with increasing Se concentration, the ZR exciton undergoes a rapid quenching in its PL intensity and significant linewidth broadening above \(x=0.03\). This drastic disruption of the ZR exciton induced by anion substitution closely resembles the behavior reported in cation-substituted compounds Ni1-xCdxPS3 or Ni1-xMnxPS324,25. Our findings reveal that even a small percentage of heterogeneous metal or anion substitutions are detrimental to the ZR excitons. Nonetheless, the decoherence mechanism may differ. For example, in the case of magnetic metal-ion substitution, a double-spin-flip process plays a significant role in the thermal behavior of ZR excitons.

In contrast to the ZR exciton, the magnetic ground state and \(2M\) excitations are not much affected by the anion substitution in the investigated range of x = 0.008-0.09. Furthermore, temperature-dependent measurements reveal that the 2 M excitations remain robust up to \(\sim 0.5{T}_{{\rm{N}}}\) irrespective of x, while the ZR exciton becomes destabilized at much lower temperatures of \(\sim 0.25{T}_{{\rm{N}}}\), reflecting that the magnetic ground state and magnon dynamics are not sufficient to explain the rapid quenching of the ZR exciton with anion substitution or the observed thermal effects. Additionally, the main ZR shows a blueshift in its peak energy with an increase in Se concentration, which contradicts the expected band gap trend, as the band gap is advocated to decrease from NiPS3 to NiPSe337,38. This unexpected increase in the primary ZR energy with increasing Se concentration is not consistent with changes in the band gap. On the other hand, a second PL peak appears on the lower energy side of the main ZR PL peak and exhibits the identical anisotropic polarization characteristic as the main ZR peak. However, in contrast to the main ZR peak, this low-energy peak shows the following differences: (i) It is very weak and is an order of magnitude broader for \(x=0.008\), but it becomes equally intense as the main ZR peak for \(x\ge 0.03\). (ii) It shows a red-shift trend with an increase in Se concentration. (iii) This peak vanishes at a temperature much lower than the main ZR peak’s destruction temperature. Overall, the temperature dependence of the exciton emission cannot be described by typical exciton-phonon behavior. Rather, for x = 0.03 -0.09, the splitting of the excitonic feature at T = 40–60 K (well below TN) alludes to the presence of additional mechanisms within the magnetically ordered state. Given the pronounced sensitivity of the exciton emission to p-cation inhomogeneity, we conclude that the underlying exciton mechanisms may go beyond scenarios based solely on intra-atomic interactions or magnetic d orbitals. Instead, our observations lend more support to the ZR exciton scenario, which originates from the spin and charge entanglement between the Ni d orbital and the ligand (S/Se) p orbital. The appearance of two distinct PL peaks upon Se substitution can be attributed to variations in charge transfer between the p orbitals of S and Se, thereby engendering the self-organization of two different spin- and charge-entangled states.

In summary, our comprehensive study of the temperature, composition, and angle-resolved PL in NiPS3-xSex (x = 0.008-0.09) revealed that even a slight degree of electronic heterogeneity in the p orbitals rapidly destabilizes the primary spin-entangled exciton, leading to the emergence of a secondary exciton. These findings highlight the decisive role of electronic uniformity in stabilizing entangled magneto-excitons.

Methods

Sample growth

High-quality NiPS3-xSeₓ (x = 0.008, 0.03, 0.06, and 0.09) single crystals were grown using the chemical vapor transport (CVT) method with iodine as the transport agent. Initially, polycrystalline powders were synthesized via solid-state reaction under high-vacuum conditions. High-purity starting materials—nickel powder (99.999%), phosphorus powder (99.999%), sulfur powder (99.999%), and selenium powder (99.999%)—were weighed in the nominal stoichiometric ratio and sealed in a quartz tube (22 mm in diameter) under a vacuum of 10⁻³ Torr. The mixture was then subjected to two-stage heating at 400 °C and 600 °C, with intermittent grinding, to ensure the formation of a single-phase compound. For crystal growth, 200 mg of iodine was added to the synthesized polycrystalline powder, which was then sealed in a quartz tube (dimensions: 20 mm × 22 mm × 400 mm) under a vacuum of 10⁻³ Torr. The tube was placed in a two-zone furnace, with temperature zones set at 700 °C and 600 °C, and maintained for 200 h. After the growth process, the furnace was cooled to room temperature at a rate of 2 °C/min. The quartz tube was subsequently broken inside an argon-filled glovebox, and high-quality single crystals were collected.

XRD and magnetic susceptibility measurements

Single-crystal X-ray diffraction (XRD) patterns were collected using a Bruker-AX (D8-Advance) X-ray diffractometer with Cu Kα radiation (λ = 1.54 Å) at room temperature. Temperature-dependent magnetic susceptibility measurements were performed using a superconducting quantum interference device vibrating sample magnetometer (SQUID VSM, Quantum Design) with an applied magnetic field of \({\mu }_{0}H=1\,{\rm{T}}\) along the ab plane.

Temperature-and polarization-dependent Raman and photoluminescence measurements

Raman and photoluminescence spectroscopic measurements were done using the Princeton SpectraPro HRS-750 spectrometer in backscattering geometry. A 532 nm (2.33 eV) laser excitation source was used to excite the Raman and photoluminescence spectra. Laser power on the sample was kept below ~500 μW to avoid any local heating and to prevent sample damage. A 50x objective lens was used to focus the incident light beam onto the sample and the same objective lens was used to collect the scattered light beam from the sample. The scattered light was dispersed using a 1200 grating coupled with an electrically cooled BLAZE charge coupled device detector. Temperature-dependent Raman and photoluminescence measurements were carried out using a closed-cycle cryostat (Montana Cryostat) by varying the temperature from 3.5 to 330 K, under high vacuum with a temperature accuracy of ±0.1 K. To ensure thermal equilibrium, a waiting time of ~15 min was maintained before acquiring each Raman/PL spectrum. A set of linear polarizers and a half-wave plate were used to perform polarization-dependent PL measurements. A vertical analyzer/polarizer was installed in front of the spectrometer to keep the consistent signal with respect to the grating orientation (see Fig. 5, in the main text for a schematic representation). A half-wave plate was installed to rotate the polarization angle in the parallel configuration.