Introduction

Of all the fundamental aspects of quantum mechanics (QM), perhaps the most bizarre is the non-local nature of an entangled system consisting of two or more components, whose quantum state cannot be factored into the tensor product of the quantum state of each individual member. “Local” and “locality” mean that an object is influenced only by its surroundings and that any influence cannot travel faster than the speed of light. Consider two observers, Alice and Bob, who are spatially far apart from each other and possess half of an entangled quanta pair. According to QM, a measurement by Alice can instantaneously affect the state of her partner, and vice versa. This entanglement between observers is non-local, superluminal, and seemingly incompatible with Special Relativity. Einstein called this “spooky action at a distance”. In 1935, Einstein, Podolsky, and Rosen devised a thought experiment, known as the EPR paradox1. The paradox led EPR to conclude that “the description of reality given by the wave function in QM is not complete”1, suggesting the existence of an local hidden variable theory (LHVT).

In 1951, Bohm modified the EPR paradox to make it experimentally accessible2,3. In Bohm’s scheme, a pair of entangled spin-1/2 particles in a spin-singlet state is used. If Alice measures the spin of her particle to be along the z direction, Bob should find the spin of his along the—z direction. In 1964, Bell developed his own variant of the EPR paradox4. Bell assumed that the measurement results of Alice and Bob could be described by two families of variables. The outcomes of their measurements in space-like separation do not affect each other and are mutually independent (local). Under these assumptions, he provided an upper bound for hidden-variable correlation according to the Bell inequality, which QM can violate in specific regions of parameter space. Bell showed that QM is incompatible with LHVT, which was known as the Bell Theorem. The triumph of the Bell inequality lies in transforming the philosophical debates on the completeness of QM into an experimental criterion.

Numerous optical experiments using entangled photons have been conducted to test Bell inequalities5,6,7,8,9,10,11. However, most experiments relied on additional assumptions in order to exclude LHVTs, therefore not closing all the so-called loopholes. There are three commonly admitted loopholes12. The locality loophole means the separation of Alice and Bob is not space-like. By increasing the distance between them and shortening the interval of successive measurements, the space-like separation requirement can be met, ensuring no physical information is exchanged between Alice and Bob, even by light. The freedom-of-choice loophole addresses whether Alice and Bob can freely and independently decide what to measure. LHVT postulates that measurements can be performed with mutual independence13. The third loophole, known as the fair sampling loophole (or detection loophole)14,15,16, can occur when a subset of detected particles violates a Bell inequality while the total group does not. If an experiment only detects this subset and assumes it represents the entire particels, a loophole is exploited. Closing this loophole is possible by detecting the particles with sufficient efficiency, which was realized in optical experiments11,17. Optical experiments designed to test Bell inequalities have made significant progress in closing potential loopholes15,16 since the pioneering work18. These experiments, both with specific loopholes and without, have produced results that clearly violate the Bell inequalities5,6,7,8,9,10,11.

Unlike experiments with entangled photons, studies utilizing entangled-massive particles to investigate local realism9,19,20,21 are uncommon, and these experiments in low energy region often do not address all three loopholes simultaneously. The detection loophole was addressed in the entangled ion experiments9,19 with significant results. The entangled states are prepared with specific ion traps, or nuclear reactions. Here, we present an experiment testing realism with entangled \(\Lambda \bar{\Lambda }\) particles. It has three eminent features: the first is that the entangled \(\Lambda \bar{\Lambda }\) particles are realized over extremely short distances, accompanied by strong and weak interactions in the decays, severing as a spin self-analyzer. The second feature is that the \(\Lambda \bar{\Lambda }\) particles is the maximally entangled states22 produced from the ηc decays. The third is concerning the locality loophole, which can be addressed by requiring the space-like criteria applied to the decayed \(\Lambda \bar{\Lambda }\) particles. Our experiment does not close the detection loophole, since having a fair sampling of the recorded events is assumed. However, it is a widely accepted fundamental assumption in collider physics23.

Testing LHVT in high energy physics are challenging, and has garnered substantial attention over the past two decades24,25,26,27,28,29,30,31,32,33,34,35,36,37. The proposed experiments can be categorized into two groups, quark flavor entanglement (also known as quasi-spin) experiments30,34,35 and particle spin entanglement experiments25,26,27,28,29,31,32,33,37,38. Tornquist proposed to examine the non-locality of quantum mechanical prediction using the spin-entangled \(\Lambda \bar{\Lambda }\) system39. The decay \(\Lambda (\bar{\Lambda })\to p{\pi }^{-}(\bar{p}{\pi }^{+})\) can serve as a spin analyzer for inferring the hyperon spins40. Specifically, the spins of Λ and \(\bar{\Lambda }\) in the \({\eta }_{c}\to \Lambda \bar{\Lambda }\) process are always opposite and possess a total spin of 0, i.e., \(\left\vert S\right\rangle=1/\sqrt{2}(\left\vert \uparrow \downarrow \right\rangle -\left\vert \downarrow \uparrow \right\rangle )\), where ↑ and ↓ denote the spin projections of \(\Lambda (\bar{\Lambda })\). Due to parity violation in hyperon weak decays, the outgoing proton (anti-proton) exhibits a preference to travel along (against) the polarization direction of the hyperon. Alice and Bob arrange to measure the spins of their respective particles at the Λ and \(\bar{\Lambda }\) decay vertices, with their measurement axes setting aligned along the momentum directions of the proton and antiproton. The correlation function between these measurements can be expressed as39:

$$E({\overrightarrow{n}}_{1},{\overrightarrow{n}}_{2})\equiv \left\langle S\right\vert \sigma \cdot {\overrightarrow{n}}_{1}\sigma \cdot {\overrightarrow{n}}_{2}\left\vert S\right\rangle=-{\overrightarrow{n}}_{1}\cdot {\overrightarrow{n}}_{2}\equiv -\cos {\theta }_{p\bar{p}},$$
(1)

where the operator \(\sigma \cdot {\overrightarrow{n}}_{1}(\sigma \cdot {\overrightarrow{n}}_{2})\) represents the measurement of \(\Lambda (\bar{\Lambda })\) spin projection along the guide axis n1(n2), which coincides with the proton (antiproton) momentum direction in the Λ (\(\bar{\Lambda }\)) rest frame. This direction is obtained by boosting the proton(antiproton) momentum to the \(\Lambda (\bar{\Lambda })\) center-of-mass system. Here \({\theta }_{p\bar{p}}\) is the opening angle between \({\overrightarrow{n}}_{1}\) and \({\overrightarrow{n}}_{2}\) with their reference frames superimposed. Thus, LHVT can be experimentally tested by measuring the distribution \(I({\theta }_{p\bar{p}})\) of angles between the momenta of p and \(\bar{p}\)39,

$$I({\theta }_{p\bar{p}})=1+{\alpha }^{2}\cos {\theta }_{p\bar{p}},$$
(2)

where α = αΛ = 0.750 ± 0.009 ± 0.004 is the asymmetry parameter of the Λ → pπ− decay, which can be precisely measured in \(J/\psi \to \Lambda (p{\pi }^{-})\bar{\Lambda }(\bar{p}{\pi }^{+})\)41,42. If a hidden measurement of Λ polarization is carried out before its decay, this reduces to \({\alpha }^{2}={\alpha }_{\Lambda }^{2}/3\). When the Bell inequality is applied to the decay \({\eta }_{c}\to \Lambda \bar{\Lambda }\), one may get a bound39 as \(| E({\overrightarrow{n}}_{1},{\overrightarrow{n}}_{2})| \le 1-\frac{2{\theta }_{p\bar{p}}}{\pi }\). Substituting with \(I({\theta }_{p\bar{p}})\), we obtain:

$$| I({\theta }_{p\bar{p}})-1| \le {\alpha }_{\Lambda }^{2}(1-\frac{2}{\pi }{\theta }_{p\bar{p}}),$$
(3)

which defines the domain satisfying the Bell inequality. If experimental measurement of the angular distribution \(\cos {\theta }_{p\bar{p}}\) lies outside the region allowed by Bell’s inequality, then a violation of Bell inequality is established.

Later, a freedom-of-choice loophole was identified when testing the Bell inequality using \(\Lambda \bar{\Lambda }\) spin entanglement produced in ηc decays40,43. Compared to optical experiments, the decays of Λ and \(\bar{\Lambda }\) occur spontaneously, rather being artificially controlled by experimenters at will. The assumption of independence in the decay of Λ and \(\bar{\Lambda }\) particles is utilized during realism testing. Closing this loophole in high-energy experiments is challenging, as suggested in ref. 30, requiring dedicated devices for active measurements. Although ref. 23 suggests that measurements of Λ and \(\bar{\Lambda }\) decays in the detector could serve as an ideal random generator, the freedom-of-choice loophole remains a significant challenge in high-energy experiments. Nevertheless, we could introduce a weak assumption that the sample that survived from the free-will choice should have the same distribution as the detected sample. Thus the presence or absence of free-will choice only affects the sensitivity to test realism, but does not alter the acceptance or rejection of realism conclusions in the high luminosity of collider experiment.

It should be noted that in previous scheme the \(\Lambda (\bar{\Lambda })\) decay parameter \({\alpha }_{\Lambda }({\alpha }_{\bar{\Lambda }})\) is introduced in the test of Bell inequality, which will bring about a new loophole. In order to overcome the QM dependence, a new inequality, a Clauser-Horne (CH)-type inequality, was developed25,44. For the \({\eta }_{c}\to \Lambda \bar{\Lambda }\) decay, the CH inequality can be generalized as25

$$P({\overrightarrow{n}}_{a},\overrightarrow{n}^{{\prime}}_{b}) -P({\overrightarrow{n}}_{a},\overrightarrow{n}^{{\prime}}_{d})+P({\overrightarrow{n}}_{c},\overrightarrow{n}^{{\prime}}_{b})+P({\overrightarrow{n}}_{c},\overrightarrow{n}^{{\prime}}_{d}) \\ -P({\overrightarrow{n}}_{c})-P(\overrightarrow{n}^{{\prime}}_{b})+\frac{1-{\alpha }_{\Lambda }^{2}}{2}\le (1-{\beta }_{\Lambda })\frac{{\alpha }_{\Lambda }^{2}}{2}.$$
(4)

Here unit vectors \({\overrightarrow{n}}_{a,c}\) and \(\overrightarrow{n}^{{\prime}}_{b,d}\) denote the directions of chosen guide axes used to detect proton and antiproton, respectively. This can be viewed as a generalization of polarizer settings in optical experiments. \(P(\overrightarrow{n},\overrightarrow{n}^{\prime} )=\frac{1}{4}(1+{\alpha }_{\Lambda }^{2}\overrightarrow{n}\cdot \overrightarrow{n}^{\prime} )\) represents the probability to detect an event of proton and antiproton at direction \(\overrightarrow{n}\) and \(\overrightarrow{n}^{\prime}\) respectively, coinciding with a \({\eta }_{c}\to \Lambda (p{\pi }^{-})\bar{\Lambda }(\bar{p}{\pi }^{+})\) decay. While \(P({\overrightarrow{n}}_{c})=P(\overrightarrow{n}^{{\prime}}_{b})=\frac{1}{2}\) indicates the probability to detect proton or antiproton alone. Just as the CH inequality is tested in optical experiments, these direction settings are used to characterize the polarization of \(\Lambda (\bar{\Lambda })\). In the above inequality, βΛ = P/E, is the velocity of Λ, and P and E are the Λ momentum and energy, respectively. The introduction of βΛ is necessary to account for the requirement of space-like separation, which decreases the upper bound. The nonzero upper bound of the CH inequality is due to the requirement, described below, of excluding any possible classical communication between the Λ and \(\bar{\Lambda }\). To directly verify the contradiction between locality and QM, one can obtain the CH inequality by substituting the QM predictions into the above equation. Specifically, the CH inequality is given by:

$$\frac{{\alpha }_{\Lambda }^{2}}{4}(\cos {\theta }_{ab}-\cos {\theta }_{ad}+\cos {\theta }_{cb}+\cos {\theta }_{cd})-\frac{{\alpha }_{\Lambda }^{2}}{2}\le (1-{\beta }_{\Lambda })\frac{{\alpha }_{\Lambda }^{2}}{2},$$
(5)

where θij are the angles between \({\overrightarrow{n}}_{i}\) and \(\overrightarrow{n}^{{\prime} }_{j}\). To highlight the specific region where the violation of the CH inequality occurs, one selects \({\theta }_{ab}={\theta }_{cb}={\theta }_{cd}={\theta }_{p\bar{p}}\) and \({\theta }_{ad}=3{\theta }_{p\bar{p}}\), resulting in the following expression25:

$$CH({\theta }_{p\bar{p}})\equiv {\alpha }_{\Lambda }^{2}\left[\frac{3\cos {\theta }_{p\bar{p}}-\cos (3{\theta }_{p\bar{p}})}{4}-\frac{1}{2}\right]\le (1-{\beta }_{\Lambda })\frac{{\alpha }_{\Lambda }^{2}}{2}.$$
(6)

Since \({\alpha }_{\Lambda }^{2} \, > \, 0\), the above inequality may test the LHVT evidently independent of αΛ, superior to inequality (3). From the generalized CH inequality, the maximum violation of the inequality, \({\alpha }_{\Lambda }^{2}(\frac{\sqrt{2}}{2}-\frac{1}{2})\, \le \, (1-{\beta }_{\Lambda })\frac{{\alpha }_{\Lambda }^{2}}{2}\) with βΛ ~ 0.7, is achieved when \({\theta }_{p\bar{p}}=\pi /4\). If a significant number of events can be observed near \({\theta }_{p\bar{p}}=\pi /4\) with \(CH({\theta }_{p\bar{p}}) > (1-{\beta }_{\Lambda })\frac{{\alpha }_{\Lambda }^{2}}{2}\), the prediction of quantum theory is inconsistent with the locality.

Results and discussion

BESIII detector and events candidates selection

The BESIII detector45 records symmetric e+e− collisions provided by the BEPCII storage ring46, which operates with a peak luminosity of 1 × 1033 cm−2s−1 in the center-of-mass energy range from 2.0 GeV to 4.95 GeV. The cylindrical core of the BESIII detector covers 93% of the full solid angle and consists of a helium-based multilayer drift chamber (MDC), a plastic scintillator time-of-flight system (TOF), and a CsI(Tl) electromagnetic calorimeter (EMC), which are all enclosed in a superconducting solenoidal magnet providing a 1.0 T (0.9 T for 2012 J/ψ data) magnetic field. The tracks of charged particles are reconstructed and their momenta are determined in the MDC, while showers from photons are reconstructed and their energy deposits are measured in the EMC.

In this work, an experimental test of the non-local correlation in the \(\Lambda \bar{\Lambda }\) system is performed using (10.087 ± 0.044) × 109 J/ψ events collected by the BESIII detector at the BEPCII e+e− collider47 through \(J/\psi \to \gamma {\eta }_{c}\to \gamma \Lambda (p{\pi }^{-})\bar{\Lambda }(\bar{p}{\pi }^{+})\) decays, and the event selection criteria are described in the Methods below. The locality loophole is closed by applying a requirement on the hyperon decay length to guarantee the spatial separation between their decays. From the e+e− interaction point, where ηc decays into Λ and \(\bar{\Lambda }\) instantaneously, the average flight distance of Λ (\(\bar{\Lambda }\)) to the decay point is about 6.95 cm. The Λ and \(\bar{\Lambda }\) candidates are identified by fitting the secondary vertices to pπ− and \(\bar{p}{\pi }^{+}\) final states, respectively. The fits yield the decay lengths L1 and L2, which are the flight distances of the Λ and \(\bar{\Lambda }\) from the beam interaction point, respectively. Testing LHVT requires space-like separation of Λ and \(\bar{\Lambda }\), and L1 and L2 are used to select separated events25,39, i.e.

$$\frac{1}{S}\le \frac{{L}_{1}}{{L}_{2}}\le S,\,{\mbox{with}}\,S=\frac{1+{\beta }_{\Lambda }}{1-{\beta }_{\Lambda }}.$$
(7)

After applying the above selection criteria, 23,313 events survived. The detection efficiency was determined to be 8.2%. The number of background events was estimated to be 4319, mainly from the decays of \(J/\psi \to \bar{\Lambda }{\Sigma }^{0}(\gamma \Lambda )+c.c.,\,\gamma \bar{\Lambda }{\Sigma }^{0}(\gamma \Lambda )+c.c.,\,{\Sigma }^{0}(\gamma \Lambda ){\bar{\Sigma }}^{0}(\gamma \bar{\Lambda }),\,{{\mbox{and}}}\,\,{\pi }^{0}(2\gamma )\Lambda \bar{\Lambda }\).

The transverse view of the BESIII detector in Fig. 1 shows an event with \(\Lambda \bar{\Lambda }\) space-like separation. The neutral hyperon pair originates from the ηc decay at the primary vertex. The decay-length ratio is L1/L2 = 2.711 for this event. The \(p\bar{p}\) daughter particles fly along curves with large radii, and the π± along curves with smaller radii. The momenta of the hyperons are calculated from those of their daughter particles, and S = 4.854 here, satisfying the space-like separation criterion.

Fig. 1: Transverse view of a \(\Lambda \bar{\Lambda }\) space-like separation event in the detector.
Fig. 1: Transverse view of a $$\Lambda \bar{\Lambda }$$ Λ Λ ¯ space-like separation event in the detector.The alternative text for this image may have been generated using AI.
Full size image

The center of the detector corresponds to the e+e− interaction point, where the J/ψ is produced and decays instantaneously into a photon and the ηc meson. The ηc itself decays also instantaneously into a hyperon pair \(\Lambda \bar{\Lambda }\), and the \(\Lambda (\bar{\Lambda })\) decays into the charged particles p and π− (\(\bar{p}\) and π+). The trajectories of the charged tracks can be seen as black curves. The areas colored beige, yellow and light blue correspond to the MDC, TOF and EMC of the BESIII detector, respectively.

Amplitude analysis

An amplitude analysis is used to calculate the probability of events with \(\gamma \Lambda \bar{\Lambda }\) final states for the selected events, which are divided into three categories. The first category refers to background events in which the final states are misidentified as \(\gamma \Lambda \bar{\Lambda }\); the second corresponds to spin-entangelment signal events including the intermediate state ηc and the non-resonant (NR) case with quantum numbers JP = 0−, denoted by NR(0−); the last category is due to spin-entanglement background events, i.e., \(J/\psi \to \gamma NR({0}^{+},{1}^{+},{2}^{+})\to \gamma \Lambda \bar{\Lambda }\) decays. The amplitudes corresponding to NR(0±, 1+, 2+) with JP are denoted by \({M}^{{J}^{P}}\), which are obtained by multiplying the helicity amplitudes of all steps of the cascade decay.

An amplitude model is used to isolate the signal events from the J/ψ cascade decay, in which the probability of finding the intermediate state JP is calculated by evaluating the weight factor of the Monte-Carlo (MC) event i, which is defined as,

$${W}_{i}(\overrightarrow{\xi },{\overrightarrow{\omega }}_{i})=\frac{1}{{{{\mathcal{N}}}}}\overline{\sum }| {M}^{{J}^{P}}(\overrightarrow{\xi },\overrightarrow{\lambda },{\overrightarrow{\omega }}_{i}){| }^{2}\frac{{N}_{{{{\rm{dt}}}}}-{N}_{{{{\rm{bg}}}}}}{{N}_{{{{\rm{MC}}}}}},$$
(8)

where \(\overline{\sum }\) denotes the summation over the helicities of the photon, proton and antiproton, and taking the average over the spin third-component of the J/ψ. The variables Ndt, Nbg and NMC denote the numbers of data, background and MC phase-space events, respectively. \({{{\mathcal{N}}}}\) is a normalization factor calculated as the amplitude squared average of NMC events. The vector \(\overrightarrow{\lambda }\) denotes the helicities of the particles involved. The parameter \(\overrightarrow{\xi }\) is determined by fitting the amplitudes to the data events with 9-dimensional vectors \({\overrightarrow{\omega }}_{i}\).

The mass spectra of \(\gamma \Lambda /\gamma \bar{\Lambda }\) and \(\Lambda \bar{\Lambda }\) can be well described by NR in conjunction with the ηc signal, and the statistical significance of ηc is determined to be more than 5σ. The criterion to include NR is that its statistical significance is more than 5σ, and the spin and parity of these states surviving the event selection criteria are determined to be JP = 0±, 1+, or 2+. However, only the \(\Lambda \bar{\Lambda }\) pairs produced by the decays of ηc and NR(0−), totaling 14716 observed events, are in the spin entangled state being studied. The contributions from other non-resonant states with JP = (0+, 1+, 2+) constitute the entanglement background. (More details could be found in the Methods).

Test CH inequalities

To test the realism, the distribution of \(CH({\theta }_{p\bar{p}})\) for the \(\Lambda \bar{\Lambda }\) spin-entangled events are measured, where the background and entanglement background are subtracted using simulated events weighted to agree with the amplitude analysis solution. More detail can be found in the Methods. The distribution of \(CH({\theta }_{p\bar{p}})/{\alpha }_{\Lambda }^{2}\) for the \(\Lambda \bar{\Lambda }\) spin-entanglement events is shown in Fig. 2. The points with total error bars, corresponding to signal, are the numbers of events with simulated background and weighted NR(0+, 1+, 2+) events subtracted in each bin, corrected by the detection efficiency times the value of \(CH({\theta }_{p\bar{p}})/{\alpha }_{\Lambda }^{2}\). For comparison with the theoretical distribution, the points are further scaled by the ratio of the area under the signal \(CH({\theta }_{p\bar{p}})/{\alpha }_{\Lambda }^{2}\) distribution divided by that under the theoretical \(CH({\theta }_{p\bar{p}})/{\alpha }_{\Lambda }^{2}\) distribution. The events in the shaded region above \(\frac{(1-{\beta }_{\Lambda })}{2}\) line are consistent with the QM prediction25, and they are located above the upper bound of the LHVT prediction, indicating that the CH inequality is significantly violated. We obtain \({\chi }^{2}={\sum }_{i=1}^{4}{({N}_{i}-{U}_{b})}^{2}/{\sigma }_{i}^{2}=30.9\) for the two bins in this interval, where Ni(σi) denotes the measurement (total uncertainty) of the ith bin, and Ub is the upper boundary of LHVT25. This shows the significance of rejecting the CH inequality is determined to be 5.2σ.

Fig. 2: The distribution of \(CH({\theta }_{p{\overline{p}}})/{\alpha }_{\Lambda }^{2}=[\frac{3 \cos {\theta }_{p{\overline{p}}}-\cos (3{\theta }_{p \overline{p}})}{4}-\frac{1}{2}]\).
Fig. 2: The distribution of $$CH({\theta }_{p{\overline{p}}})/{\alpha }_{\Lambda }^{2}=[\frac{3 \cos {\theta }_{p{\overline{p}}}-\cos (3{\theta }_{p \overline{p}})}{4}-\frac{1}{2}]$$ C H ( θ p p ¯ ) / α Λ 2 = [ 3 cos θ p p ¯ − cos ( 3 θ p p ¯ ) 4 − 1 2 ] .The alternative text for this image may have been generated using AI.
Full size image

The points with total error bars are the measurements, the solid line is the QM prediction, and the dashed line is the upper bound of the LHVT prediction. The shaded area above the dashed line indicates the violation of the \(CH({\theta }_{p\bar{p}})\) inequality.

We also test the Bell and CHSH inequalities in two QM dependent schemes, with further details provided in the Supplementary Note 1. To measure the \(p\bar{p}\) angular distribution, the significance to exclude the Bell inequality region is determined to be 8.9σ. The measurements are consistent with the QM predictions and clearly contradict the predictions of the LHVT. We check the CHSH inequality by calculating the Cij tensor using a TOY MC method based on amplitude analysis. A χ2 test shows that excluding the LHVT is significant at a level exceeding 10σ.

Using (10.087 ± 0.044) × 109 J/ψ events collected with the BESIII experiment, a non-local correlation test for the spin entanglement of a hyperon system in \(J/\psi \to \gamma {\eta }_{c},{\eta }_{c}\to \Lambda \bar{\Lambda }\) decays is carried out for the first time. The decay lengths of the Λ and \(\bar{\Lambda }\) particles, detectable at a macroscopic scale, are used to control the selection of space-like separation events, which ensures that the locality loophole is closed. The \(CH({\theta }_{p\bar{p}})\) angular distributions between the proton and antiproton are in agreement with the QM predictions within their uncertainties. The Bell and CHSH inequalities are also tested, which however are QM dependent. The three tests have significantly excluded LHVT, confirming the existence of non-local quantum correlations, with the significances of 5.2σ, 8.9σ and larger than 10σ for the CH, Bell and CHSH inequalities, respectively. These results confirm the existence of quantum entanglement and the violation of the Bell inequality in the presence of strong and weak interactions. It tells that the entanglement emerging from these fundamental interactions also exhibits quantum correlation and nonlocality, which deepens our understanding of the physical reality.

Methods

Detector simulation and MC events

MC simulations are used to optimize the event selection criteria and estimate the background sources, as well as to determine the efficiency. geant448 based MC software, including the geometric description of the BESIII detector49,50 and its response, is used to simulate the MC samples. The inclusive MC sample includes the production of vector charmonium(-like) states and the continuum processes incorporated in kkmc51. All particle decays are modeled with evtgen52,53 using the branching fractions either taken from the Particle Data Group54, or otherwise estimated with lundcharm55,56.

Selection criteria

The final state of a candidate event is required to contain four charged tracks (\(p,\bar{p},{\pi }^{+}\) and π−) and at least one good photon (γ). The charged tracks reconstructed in the MDC are required to satisfy \(| \cos \theta | \le 0.93\), where θ is defined with respect to the z-axis which is the symmetry axis of the MDC. The momentum distributions of protons and pions from the signal process are well separated and do not overlap, as shown in Supplementary Fig. 3. Therefore, a simple momentum criterion is applied: if the momentum of the particle is larger than 0.4 GeV/c, it will be identified as a proton, otherwise it will be identified as a pion. Next, the \({\Lambda }\to {p}{\pi }^{ - }\) and \(\bar{\Lambda }\to \bar{p}{\pi }^{+}\) decays are reconstructed, requiring for each that two tracks with opposite charges can be successfully fit to a secondary vertex and requiring the invariant mass of the hyperon lies in the interval of [1.008,1.124] GeV/c2. The hyperon decay length distributions of the MC events are in good agreement with the data (shown in Supplementary Figs. 4 a and b).

Photon candidates are reconstructed from showers in the EMC within 700 ns from the event start time. The deposited energy of each shower is required to be greater than 25 MeV in the barrel region (\(| \cos \theta | \le 0.8\)) or 50 MeV in the end cap region (\(0.86\le | \cos \theta | \le 0.92\)), and the minimum opening angle between the shower and the pion or nucleon (antinucleon) is required to be greater than 20°(30°). The radiative photon is selected through a four constraint (4C) kinematic fit requiring energy and momentum conservation in the decay \(J/\psi \to \gamma \Lambda \bar{\Lambda }\), and events with \({\chi }_{{{{\rm{4}}}}C}^{2}(\gamma \Lambda \bar{\Lambda }) \, < \, {\chi }_{{{{\rm{4}}}}C}^{2}(\gamma \gamma \Lambda \bar{\Lambda })\) and \({\chi }_{{{{\rm{4}}}}C}^{2}(\gamma \Lambda \bar{\Lambda }) \, < \, 30\) are retained for further analysis, where \({\chi }_{{{{\rm{4C}}}}}^{2}\) is the goodness of fit of the kinematic fit. In order to remove background events containing \(J/\psi \to {\Sigma }^{0}{\bar{\Sigma }}^{0},\Lambda {\bar{\Sigma }}^{0}+c.c.\) decays, the invariant mass of \(\gamma \Lambda /\gamma \bar{\Lambda }\) is required to satisfy \(| {M}_{\gamma \Lambda /\gamma \bar{\Lambda }}-{M}_{{\Sigma }^{0}}| \, > \, 0.009\) GeV/c2, where \({M}_{{\Sigma }^{0}}\) is the known mass of Σ054.

Amplitude analysis using maximum likelihood fit

The amplitude of the ηc or non-resonant (NR) decays depends on a 9-dimensional vector \(\overrightarrow{\omega }=({M}_{\Lambda \bar{\Lambda }},{\theta }_{\gamma },{\phi }_{\gamma },{\theta }_{\Lambda },{\phi }_{\Lambda },{\theta }_{p},{\phi }_{p},{\theta }_{\bar{p}},{\phi }_{\bar{p}})\), where \({M}_{\Lambda \bar{\Lambda }}\) is the invariant mass of the \(\Lambda \bar{\Lambda }\) system and θi and \({\phi }_{i}(i=\gamma,\Lambda,p,\bar{p})\) denote, respectively, the polar and azimuthal angles of particle i in their helicity coordinate systems, which are illustrated in Fig. 3. For the decay A → B + C, the polar angle θ is defined as the angle between the momentum vectors \({\overrightarrow{p}}_{A}\) and \({\overrightarrow{p}}_{B}\), which are defined in the rest frame of the mother particle. The azimuthal angle ϕ is defined as the angle between the production and decay planes of particle A.

Fig. 3: Illustration of helicity angles defined in each step of a decay chain.
Fig. 3: Illustration of helicity angles defined in each step of a decay chain.The alternative text for this image may have been generated using AI.
Full size image

Helicity angles in \(J/\psi \to \gamma {\eta }_{c},{\eta }_{c}\to \Lambda \bar{\Lambda }\) decays, and \({\eta }_{c}\to \Lambda \bar{\Lambda }\to p\bar{p}{\pi }^{+}{\pi }^{-}\) decays.

For the cascade decays \({{{\rm{J}}}}/{{{\mathrm{\psi}}}} ({{{\rm{m}}}})\to \gamma ({\lambda }_{\gamma })R({\lambda }_{0}),\,R\to \Lambda ({\lambda }_{1})\bar{\Lambda }({\lambda }_{2}),\,\Lambda \to p({\lambda }_{3}){\pi }^{-}\) and \(\bar{\Lambda }\to \bar{p}({\lambda }_{4}){\pi }^{+}\), where m and λi denote the third component of the J/ψ spin and the helicity of the particle i, respectively, the amplitudes are expressed as

$${M}^{{J}^{P}}(\overrightarrow{\xi },\overrightarrow{\lambda },\overrightarrow{\omega })= {\sum}_{{\lambda }_{0},{\lambda }_{1},{\lambda }_{2}}{D}_{m,{\lambda }_{\gamma }-{\lambda }_{0}}^{1*}({\phi }_{\gamma },{\theta }_{\gamma },0){D}_{{\lambda }_{0},{\lambda }_{1}-{\lambda }_{2}}^{J*}({\phi }_{\Lambda },{\theta }_{\Lambda },0){D}_{{\lambda }_{1},{\lambda }_{3}}^{1/2*} \\ ({\phi }_{p},{\theta }_{p},0){D}_{{\lambda }_{2},{\lambda }_{4}}^{1/2*}({\phi }_{\bar{p}},{\theta }_{\bar{p}},0) \times BW({M}_{\Lambda \bar{\Lambda }},{M}_{0},{{\Gamma }}){H}_{{\lambda }_{\gamma },{\lambda }_{0}}^{J/\psi }\\ (\overrightarrow{\xi }){H}_{{\lambda }_{1},{\lambda }_{2}}^{R}(\overrightarrow{\xi }){H}_{{\lambda }_{3},0}^{\Lambda }(\overrightarrow{\xi }){H}_{{\lambda }_{4},0}^{\bar{\Lambda }}(\overrightarrow{\xi }),$$
(9)

where \(BW({M}_{\Lambda \bar{\Lambda }},{M}_{0},{{\Gamma }})\) is the relativistic Breit-Wigner (BW) function describing the ηc resonance with a mass of M0 and a width of Γ. For non-resonant transitions, this BW factor is set to 1. \({D}_{n,h}^{J}(\phi,\theta,0)\) are elements of the Wigner-D matrix, where J is the spin of R and n and h correspond to helicities. \({H}_{{\lambda }_{B},{\lambda }_{C}}^{A}\) denotes the helicity amplitude of the decay A → B(λB)C(λC), which is expanded into partial waves in terms of the orbital angular momentum L and the total spin S of the decay, and combined linearly with the L-S coupling parameter \(\overrightarrow{\xi }\)57,58. For \({H}_{{\lambda }_{\gamma },{\lambda }_{0}}^{J/\psi }\) and \({H}_{{\lambda }_{1},{\lambda }_{2}}^{R}\), the number of partial waves is restricted by parity conservation. For the \(\Lambda (\bar{\Lambda })\to p{\pi }^{-}(\bar{p}{\pi }^{+})\) weak decays, their amplitudes, \({H}_{{\lambda }_{3},0}^{\Lambda }\) and \({H}_{{\lambda }_{4},0}^{\bar{\Lambda }}\), are expanded in terms of S- and P-wave amplitudes. Because the two decays approximately conserve the CP quantum numbers41, the sign of the S-waves stays the same, while the P-waves change sign in the charge conjugated decays. The L-S coupling constants involved in all partial wave amplitudes are set as parameters to be determined by fitting the data.

The probability density of event i is obtained by coherently adding the amplitudes of all intermediate states, and taking the modulo squared:

$${\sigma }_{i}(\overrightarrow{\xi },\overrightarrow{\omega })=\mathop{\sum}_{m,{\lambda }_{\gamma },{\lambda }_{3},{\lambda }_{4}}\left| {\sum}_{{J}^{P}}{M}^{{J}^{P}}(\overrightarrow{\xi },\overrightarrow{\lambda },\overrightarrow{\omega })\right|^{2}.$$
(10)

Here JP is summed over all resonant and non-resonant states.

We determine the coupling parameters, \(\overrightarrow{\xi }\), from a maximum likelihood fit to data. The likelihood function of an ensemble with N events is defined as

$${{{\mathcal{L}}}}(\overrightarrow{\xi },\overrightarrow{\omega })={\prod }_{i=1}^{N}{{{{\mathcal{P}}}}}_{i}(\overrightarrow{\xi },\overrightarrow{\omega })={\prod }_{i=1}^{N}\frac{{\sigma }_{i}(\overrightarrow{\xi },\overrightarrow{\omega })}{{{{\mathcal{N}}}}(\overrightarrow{\xi })},$$
(11)

where \({{{\mathcal{N}}}}=\int{\sigma }_{i}(\overrightarrow{\xi },\overrightarrow{\omega })d\overrightarrow{\omega }\) is the normalization factor, which accounts for the detection and reconstruction efficiency and is approximated as the MC integral i.e., the average value of the integrand is estimated with a sufficiently large number of MC events. Here the MC events are generated with a phase-space model, subjected to detector simulation, and required to survive the event selection criteria. The minimum of the objective function

$$S=-[\ln {{{\mathcal{L}}}}(\overrightarrow{\xi },{\overrightarrow{\omega }}_{{{{\rm{dt}}}}})-\ln {{{\mathcal{L}}}}(\overrightarrow{\xi },{\overrightarrow{\omega }}_{{{{\rm{bg}}}}})]$$
(12)

corresponds to the maximum of the likelihood function \({{{\mathcal{L}}}}\). To obtain the coupling parameters \(\overrightarrow{\xi }\) in the amplitude analysis, S is minimized with minuit259, and the contribution from the background events obtained from the exclusive MC samples, \(\ln {{{\mathcal{L}}}}(\overrightarrow{\xi },{\overrightarrow{\omega }}_{{{{\rm{bg}}}}})\), is subtracted from the objective function of the data \(\ln {{{\mathcal{L}}}}(\overrightarrow{\xi },{\overrightarrow{\omega }}_{{{{\rm{dt}}}}})\). The dominant background mainly consists of \(J/\psi \to \bar{\Lambda }{\Sigma }^{0}\) + c. c. , \(J/\psi \to \bar{\Lambda }(1520)\Lambda \to \gamma \bar{\Lambda }\Lambda\) + c. c., and \(J/\psi \to {\bar{\Sigma }}^{0}(\gamma \bar{\Lambda }){\Sigma }^{0}(\gamma \Lambda )\), with their contributions estimated using MC samples. The efficiency of the data obtained in 2012 is higher than that of the other runs by 14%, since the MDC magnetic field setting was lower than that in other years. Consequently, the dataset of 10 billion data events is divided into two sub-samples and then fitted simultaneously to determine the parameters. ~11% of the data was obtained with the lower magnetic field.

To have the signal and NR samples agree with the solution obtained from the amplitude model, phase space MC events are weighted by \({{{{\mathcal{P}}}}}_{i}(\overrightarrow{\xi },\overrightarrow{\omega })\), with parameters obtained from the maximum likelihood fit. Background is made up of inclusive MC events. The numbers of simulated events used is obtained from the maximum likelihood fit. The fit results are displayed in Supplementary Figs. 5 through 8.

The invariant mass spectra of γΛ(\(\gamma \bar{\Lambda }\)) and \(\Lambda \bar{\Lambda }\) are displayed in Supplementary Figs.5 and 6. The signal ηc and NR(0±, 1+, 2+) components are parameterized by weighted phase space.

Supplementary Fig. 7 shows the \(\cos {\theta }_{p\bar{p}}\) distribution. The total histogram is the sum of weighted simulated samples of signal entangled events (ηc and NR(0−)), NR(0+, 1+, 2+), and background. Supplementary Fig. 8 shows the \(CH({\theta }_{p\bar{p}})\) distribution multiplied by the number of data events. The entries are the number of ηc signal events in each bin times the value of CH for that bin. The number of signal events is given by the data minus NR(0+, 1+, 2+) and background.

The yields of JP components are determined according to the MC event weights, namely, the ratio of the cross section for the JP component over the total cross section. Supplementary Table 1 shows the number of yields in data for the NR(0+, 0−, 1+, 2+) and ηc components.

Systematic uncertainties

The systematic uncertainties considered here include: tracking efficiency, photon detection efficiency, space-like separation criteria, kinematic fit, background estimation, and the mass and width of ηc. To account for potential correlations among the systematic sources, all systematic sources, with the exception of the space-like separation, are collectively considered in an alternative fit, rather than being treated separately. Initially, we adjust the MC sample to accommodate the kinematic fit, followed by corrections for tracking efficiency, photon efficiency, and \(\Lambda /\bar{\Lambda }\) reconstruction, achieved by multiplying their correction factors to the predicted amplitude squared. The uncertainties in the mass and width of ηc are factored in when calculating its Breit-Wigner amplitude, by randomly smearing its mass and width. During the subtraction of the background event contribution from the log-likelihood, the weighted factor is smeared in accordance with the statistical uncertainty of the background. The difference in the \(\cos {\theta }_{p\bar{p}}\) distribution between the nominal and alternative fit is considered as the systematic uncertainties. The systematic uncertainties for the \(\cos {\theta }_{p\bar{p}}\) distribution are negligible. However, the systematic uncertainty of the CH inequality distribution is primarily influenced by the space-like separation requirement. The detail of the systematic uncertainties are listed in Supplementary Note 4.