Introduction

Hallucinations are complex and heterogenous phenomena during which an individual has an aberrant perceptual experience in the absence of any corresponding external stimulus1. While hallucinations can be experienced by healthy individuals2, with increased prevalence in the elderly3,4 and during bereavement3,5, they have major clinical relevance in psychiatric conditions such as schizophrenia6 or neurodegenerative diseases such as Lewy body dementia7 and Parkinson’s disease8,9,10. Despite this, hallucinations remain difficult to study in laboratory conditions, given their spontaneous appearance, variable content, and lack of specific induction protocols. Methods such as Ganzfeld11 and flicker-induced hallucinations12 have been developed for this purpose, but cannot induce clinically relevant hallucinations similar to those experienced by patients, lack real-time control over their induction and content, and miss appropriate control experimental conditions. Psychoactive drugs can induce vivid hallucinations, but these are often modality unspecific, have highly variable content, and are often accompanied by significant alterations of consciousness13,14.

Recently, the integration of robotics technology with insights from cognitive neuroscience and clinical research, allowed the repeated induction of a specific and clinically-relevant hallucination: presence hallucination (PH), which is the convincing sensation that someone is close by when no one is there15. PH is frequent in, Lewy body dementia16,17, Parkinson’s disease18, and has also been reported in schizophrenia19,20,21, stroke22, epilepsy23, as well as healthy individuals24. PH is one of the most common hallucinations in Parkinson’s disease, being experienced by approximately 40% of patients25,26, occurring early in the disease27, and is associated with faster disease progression leading to structured hallucinations, persistent psychosis, and cognitive decline28,29, which in turn are associated with higher risk of mortality30,31. The aforementioned robotic setup has been shown to induce a PH-like state (robot-induced PH: riPH) comparable to symptomatic PH, in healthy individuals32,33,34,35,36, first-episode psychosis patients37 and in patients with Parkinson’s disease38, by exposing them to different sensorimotor conflicts. Specifically, participants manipulate a front-robot, whilst a back-robot provides tactile feedback to their back by reproducing the participant’s front movements with a temporal delay. Such asynchronous stimulation triggers riPH (PH-inducing condition), whereas a control condition without the temporal delay, but otherwise identical stimulation, does not. Recently, this robotic setup was extended to MRI36 in healthy individuals38,39, and revealed that both the posterior superior temporal sulcus (pSTS) and the inferior frontal gyrus (IFG) are generally more active in the PH-inducing condition than in the control condition38. In addition, participants who were sensitive to riPH were shown to have altered brain network dynamics, with a significant increase of transitions to a specific brain network (PH-network) during the induction condition40.

Given the clinical relevance of PH18,29, the ability to induce it in the MR-scanner and the identification of abnormal network dynamics related to the PH-network in healthy individuals40, here, we investigated: (1) whether healthy participants could achieve volitional control of the PH-network, (2) whether successful regulation would increase sensitivity to riPH, and (3) whether regulation would trigger changes in the dynamics of the PH-network. Together, such findings in healthy individuals would allow to draw a more causal inference regarding the role of this brain network in PH. To that aim, we combined the MR-compatible robot capable of triggering riPH36 with real-time fMRI neurofeedback (fMRI-NF)41.

Real-time fMRI-NF is a technique to “close the loop” between brain activity and behavior, by providing participants with real-time feedback about specific features of their brain activity, with the aim of enabling the search for a mental strategy capable of successfully regulating said brain activity and modulating behavior41,42,43. Targeting functional connectivity through fMRI-NF is currently the most promising approach to translate control of brain activity into modulation of neuropsychiatric symptoms42,44,45. While modulation of hallucinations has not yet been fully demonstrated using fMRI-NF, regulating average properties of large-scale brain networks comparable to the PH-network has been demonstrated for attention46 and stress47. For the present experiment, we developed a paradigm where participants underwent five fMRI sessions (each on a different day), spanning over the course of two weeks (Fig. 1A): one session for pre-training assessment of proneness to riPH, three sessions of fMRI-NF training targeting the PH-network, and one for post-training assessment of proneness to riPH. During training sessions, participants manipulated the robotic system while attending a continuous auditory feedback score based on the correlation between their ongoing brain activity and the PH-network. With this, we trained 20 participants to up-regulate the PH-network in a total of 176 training runs and then assessed their regulation performance using a non-parametric procedure based solely on temporal properties of the feedback signal. Brain activity in both training and pre/post sessions were then investigated using clustering techniques capable of defining relevant networks of brain activity for the present protocol. These were followed up with regression analysis on temporal metrics of the networks (occurrences and transition probabilities), to investigate the underpinnings of the neural regulation mechanisms and neural changes between pre and post sessions caused by the training. In line with our research questions, our results show that: (1) participants were successful in up-regulating the PH-network by increasing transition probabilities to this network; three sessions of fMRI-NF training on different days (2) led to an increase in sensitivity to riPH after training; and (3) further led to neural changes observed on the last day of testing, as participants who became sensitive to riPH after training presented higher occurrences of the PH-network during the induction of riPH.

Fig. 1: Experimental paradigm.
figure 1

A Diagram of the fMRI experiment. On average participants (N = 20) completed the experiment in around two weeks, with a minimum break of one day between each session (neurofeedback: NF). B One block of neurofeedback training is shown. In total six were done per run. Participants start at rest. After one auditory cue the baseline condition starts and participants move the robot in the synchronous mode, while listening to the minimal level of auditory feedback (it did not vary with brain activity at this point). Participants were told they shouldn’t engage in a mental strategy here and should rather “clear their minds”. After two auditory cues the feedback condition started, and participants continued moving the robot which changed to the asynchronous mode. The auditory feedback (resembling a looming and receding wave) became now variable with the spatial correlation between ongoing brain activity and the PH-network (visually represented three times in the figure but it varied every 1.5 s). Participants knew they should deploy a mental strategy to boost the feedback to the maximum level. This condition ended with three auditory cues. Participants were asked about preferred strategies at the end of each session (and reassured some strategies might not be verbalizable). At the end of the last training sessions a more general briefing occurred. C Two blocks of the riPH task are shown in sequence of a total of 16 blocks per run. Blocks were composed of rest followed by an auditory cue which indicated the start of the condition were participants moved the robot in asynchronous or synchronous mode. Two auditory cues indicated the end of the riPH task. No more than two blocks in sequence could be of the same condition. At the end of the second run of the riPH task, participants performed two individual blocks of robot manipulation (one asynchronous, one synchronous), each followed by a questionnaire assessing PH induction sensitivity amongst other sensations and control questions.

Results

Neurofeedback regulation performance

We first investigated whether there was evidence for any regulation of brain activity during fMRI-NF sessions. This was done by examining the feedback scores presented to the participants during the feedback condition, given that these indicated instances of modulation of the PH-network between the feedback and baseline conditions (Fig. 2A; see Methods). We used a non-parametric surrogate-data procedure to generate a null distribution of the average feedback scores for each neurofeedback training run, and then considered a neurofeedback run to exhibit high regulation when the average feedback score was one standard deviation above the mean (percentile threshold: 84.1%) of the neurofeedback run’s surrogate distribution feedback score (Fig. 2B; see Methods). This revealed a total of 52 out of 176 neurofeedback runs that achieved high regulation (~30%; median per participant: 2; MAD: 1.7; range: 0–6), which was highly significant according to binomial testing (P < 0.00001) (Fig. 2C). Importantly, repeating this procedure for every selection threshold between 50% and 97.5% (in steps of 2.5%), confirmed that regulation was always significant regardless of the chosen threshold (Fig. 2D). To further confirm regulation performance, we sought to assess whether high regulation was maintained in runs during which participants received no feedback, but that were identical to neurofeedback runs in all other aspects. These transfer runs were performed at the end of each training day (Fig. 1A). Using the same procedure described above, we identified 18 out of 56 transfer runs with high regulation (~32%; median per participant: 1; MAD: 0.63; range: 0–2; P < 0.00001), and importantly, showed that the number of high-regulation transfer runs was significantly correlated to the number of high-regulation neurofeedback runs (\(\rho\) = 0.49; P = 0.02750). Participants were instructed to not change respiration patterns between conditions, but nonetheless cardiac and respiration confounds were regressed out in the analysis to ensure that these would not drive the achieved regulation between the conditions. Put together, these results show that participants successfully achieved control over the PH-network during fMRI-NF training, and that they were able to maintain such control even in the absence of feedback (i.e., during transfer runs).

Fig. 2: Assessment of neurofeedback regulation performance.
figure 2

A An exemplary run illustrates how the feedback score presented the feedback condition is calculated (vertical dashed lines delineate baseline and feedback conditions). Computing the feedback score of a time point during the feedback condition is performed by subtracting the median NF-signal of the previous baseline condition (in orange) to the current NF-signal and normalizing by the maximum dynamic range over a trial moving window (dashed horizontal lines). B Non-parametric amplitude adjusted surrogate data procedure to identify runs with high regulation. The NF-signal of a run is extracted and transformed into Fourier space. Surrogates with identical temporal properties are generated by randomizing the phase and doing an inverse transformation back into time domain (100,000 times). By virtually computing the average feedback score of each surrogate, a null distribution of average feedback scores of the surrogates is obtained, and the observed feedback score can then be compared against this null distribution based on an 84.1% percentile threshold. C Examples of null distributions of different runs are shown on the left. On the right, it is shown how the probability of obtaining the observed number of high regulation runs (52) or more, was obtained. I.e., by evaluating the area under the binomial distribution curve with parameters X ~ Bin(176, 1–0.841). D The number of high regulation runs is shown on the left, in function of the choice of threshold. On the right the probability of obtaining such number of high regulation runs (or more) is shown, taking into account the different percentile thresholds. Independently of the chosen threshold, regulation was successful in such a manner that the probability of obtaining the observed numbers of high regulation runs is always extremely low, even when considering null hypotheses with extremely stringent percentile thresholds.

Brain networks underlying NF and riPH sessions

In parallel and independently of the regulation performance analyses, we investigated brain networks active during training sessions and pre/post training sessions. Capturing brain networks active during these sessions was done using co-activation pattern analysis (CAPs)48, a clustering technique that resolves dynamics of functional connectivity in terms of large-scale activation patterns occurring when predefined seed regions exhibit high activity (see Methods). We focused on periods of high activity of the pSTS and IFG, as these are related to PH-induction38, and can capture an activation pattern that underlies sensitivity to riPH: the PH-network40. Periods when none of these regions were active, are referred to as network0. Using this method, we recovered eight brain networks that were active during fMRI-NF training and the pre/post training sessions. These were named based on their spatial pattern and included: the PH-network (Fig. 3A–C), the sensorimotor+ network (Fig. 3D), its opposite pattern (i.e., sensorimotor- network) (Fig. 3E), and a default-mode network- (DMN-) characterized by deactivated regions of the DMN49 (Fig. 3F). The sensorimotor- and the DMN- networks had large overlap with the PH-network, and could represent subnetworks of the PH-network as CAP analysis is particularly apt at recovering subnetworks48,50 (other networks are shown in the Supplementary Fig. S1 and cluster peaks in Supplementary Table S1). These findings corroborate previous results that recovered the PH-network during induction of riPH, but now extend them to the fMRI-NF context.

Fig. 3: Anatomy of PH-network and of other identified networks.
figure 3

All brain networks are shown after spatial standardization. A PH-network identified in previous work34 and used here to compute online NF-signal. Major clusters are seen on the right posterior superior temporal sulcus (pSTS), the bilateral inferior parietal lobule (IPL) with focus on the supramarginal gyri (SMG), the right dorso-lateral prefrontal cortex (dlPFC), the middle prefrontal cortex (including part of the supplemental motor area: SMA), the left dlPFC, the left precentral gyrus (PrC), the body of the caudate on the right, and in the cerebellum, the left Crus I and II. Deactivations are observed over the cuneus and occipital gyrus. B PH-network as identified by CAP analysis in the current data. C Overlap between the PH-network as recovered previously34 and the PH-network recovered here, showing that the latter recovers every major cluster of the former. D The sensorimotor+ network represents mostly sensory processing. Major activation clusters can be seen bilaterally over the sensory cortex (post central gyri, PsC), SMA, pre-motor cortex (pmC) pSTS, and supra-marginal gyri (SM). Major cluster deactivations are also seen bilaterally for the middle temporal gyri (MTG), medial and superior frontal gyri (mSFG, SFG), angular gyri, and posterior cingulate cortex (PCC). E The sensorimotor- network is characterized by the pairing of bilateral deactivations of the sensory cortex, and by activations over four main cluster of the PH-network all on the right: the pSTS, the dlPFC, the mPFC, and the AG. F The DMN- network is characterized by deactivations of the PCC, bilateral AG, bilateral superior frontal gyri, and mPFC. It is also accompanied by activations over the PH-network clusters, including: SMA, and dlPFC (more prominently on the right).

Up-regulation of the PH-network during the feedback condition

Next, in order to characterize the brain mechanisms underlying high regulation, we computed network transition and occurrence probabilities, two metrics which had been previously associated with PH-induction40. To study network dynamics, these were then analysed with Linear Mixed Models (LMM) followed by post-hoc analyses to decompose significant interactions (multiple comparisons corrected, see Methods). Our results identified an up-regulation mechanism underlying high-regulation runs based on changes in transition probabilities and characterized by re-instantiation of the PH-network during the feedback condition.

We hypothesized that successful participants would learn to boost the auditory feedback by changing transition probabilities across networks, in a manner that favored the PH-network. To test this, we first demonstrated that transition probabilities between brain networks changed depending on the brain networks assessed (i.e., transitions coded with two predictors, initial and final network; see Methods), conditions (feedback, baseline, and rest), and on whether high regulation was achieved (three-way interaction: P = 0.00058, Supplementary Table S2). Then, through post-hoc analyses (detailed in Supplementary Tables S3, S4), we showed that achieving high regulation modulated the transitions departing from specific networks to all networks in general, and that furthermore this effect of high regulation was also dependent on the final network of the transition (interaction between the high-regulation and final network). The networks that exhibited these changes in transitions to all others were: network0 (P < 0.00001, P = 0.01371, respectively for the high regulation effect and for the interaction), the PH-network (P < 0.00001, P = 0.00699), the sensorimotor- network (P < 0.00001, P = 0.00036), the DMN- network (P < 0.00001, P = 0.00047), and the DMN+ network (P < 0.00001, P = 0.01910). These significant changes reflected increases in transition probabilities towards the PH-network and its subnetworks (i.e., sensorimotor- and DMN- networks; Fig. 4, Supplementary Fig. S2). The same analysis performed for transfer runs (i.e., without auditory feedback) detected one of these significant changes, specifically an increase in transitions from the PH-network to the DMN- (see Supplementary: Tables S5, S6, Fig. S3), implying that participants had partial retention of this up-regulation strategy developed in the feedback condition.

Fig. 4: Re-instatement mechanism during the feedback conditions of runs with high regulation.
figure 4

Linear mixed model analysis on the networks’ transition probabilities identified a triple interaction between condition, high-regulation, and initial network of the transition (p = 0.00058). Subsequent analysis revealed that in the feedback condition of runs with high regulation, three networks (PH, DMN-, sensorimotor-) significantly changed their transitions to other networks as compared to the feedback condition of runs where high regulation was not achieved. These changes in transition probabilities formed a reinforcement loop to the PH-network through networks that were similar to it. They were predominantly marked by increases in the transitions from the PH-network, to two networks which include activations over major PH-network clusters (DMN- by 28%, and sensorimotor- by 12%), by increases from the DMN- to itself (22%) and to the sensorimotor- (64%), and of the sensorimotor- to itself (34%) and back to the PH-network (44%).

The auditory feedback chosen for the neurofeedback was a very simple artificial sound composed of three harmonics (see methods). However, to ensure that the neurofeedback mechanisms identified above could not be driven by the sensory stimulation from the auditory feedback alone, we conducted an activation-based analysis on the auditory functional localizer that was performed at the beginning of each training session. Our results show that the sound led to activations in the lingual gyrus, Rolandic operculum, and cerebellum, whereas a parametric modulator for the variation of the feedback detected activations on the postcentral gyrus (Supplementary Table S7). None of these regions overlapped with the PH-network. Additionally, we investigated what information does the interpretation of the auditory feedback carry as compared to the sound alone (contrast: parametric modulator for levels of feedback > auditory feedback sound). This revealed four main clusters: caudate nucleus, thalamus, superior temporal pole, and inferior medial frontal gyrus (Supplementary Fig. S4, Supplementary Table S7). There was again no overlap with the PH-network for any of these maps, indicating that nor the auditory stimuli alone nor its interpretation drove the regulation results. Overall, these results demonstrate that successful participants managed to regulate their succession of brain networks differently than unsuccessful participants, in a manner that benefitted the PH-network and two of its subnetworks (sensorimotor- and DMN-).

Avoidance of the PH-network during baseline

Concerning occurrences of brain networks, we hypothesized that successful participants would increase the occurrences of the PH-network during the feedback condition, specifically when high regulation was achieved. Therefore, we first demonstrated that occurrences changed depending on condition, brain network and high regulation (triple interaction: P = 0.00004, Supplementary Table S8). Post-hoc analyses showed this effect was in fact driven by changes during the baseline condition rather than the feedback one. Specifically, by a significant decrease in occurrences of the PH-network (P = 0.02603; Fig. 5A), posterior network (P = 0.01837, Supplementary Fig. S5), and by a significant increase in occurrences of the sensorimotor+ network (P = 0.04763; Fig. 5B), when comparing the baseline condition between runs with and without high regulation (Supplementary Tables S9, S10). Thus, in the baseline condition we note that recruitment of the PH-network occurred less, and instead the sensorimotor+ network was recruited preferentially, which included more prominent activations in sensory regions, as well as deactivations in regions overlapping with the PH-network. While not initially hypothesized, avoidance and partial deactivation of the PH-network during the baseline condition, does increase the differential between baseline and feedback conditions, thereby augmenting the feedback participants received during the latter. For results regarding the transfer runs see Supplemental Information (Supplemental Tables S11, S12; Supplementary Fig. S6). Collectively, we observed two distinct changes to brain dynamics that improved the received feedback. First, during the feedback condition participants increased auditory feedback by modulating transitions across brain networks, effectively engaging in a re-enforcing cycle between the PH-network and similar networks. This demonstrated that participants learned to up-regulate the PH-network. Second, we observed that the PH-network was recruited less during the baseline condition, in favor of a network that included deactivations over the PH-network, which improved the feedback score received in the next round by lowering the baseline. While we verified that this latter effect was not present during rest (and hence could not be a compensation from the previous feedback condition; Fig. 5), our setup did not include assessment of volitional down-regulation and hence we could not determine if participants in fact learned to do this (explicitly or implicitly).

Fig. 5: Avoidance of the PH-network during the baseline condition of runs with high regulation.
figure 5

A Linear mixed model analysis of networks’ occurrences identified a triple interaction between condition, high regulation, and network (p = 0.00004). Subsequent analysis revealed that the occurrences of the PH-network were significantly lower in the baseline condition of runs with high regulation, as compared to the same condition of runs without high regulation (p = 0.02603). B Subsequent analysis of the triple interaction reported above also revealed that the occurrences of the sensorimotor+ network were significantly higher in the baseline condition of runs with high-regulation as compared to the same condition of runs without high regulation (p = 0.04736).

Effects of NF training on behavioral sensitivity to riPH

We hypothesized that training participants to up-regulate the PH-network would lead to an increase in proneness to riPH after training, and so, we compared riPH before and after fMRI-NF training; i.e., at day 1 versus day 5. On day 1, 10% of participants (2/20) were sensitive to riPH, as measured by a positive difference in ratings between the PH-inducing and control conditions. At the group-level there was no significant difference in induction between both conditions on the first day (as assessed with cumulative linear models, see Methods). This changed after fMRI-NF training (day 5), as 40% of participants became sensitive to the robotic stimulation (8/20, including the two already sensitive at day 1), and a significant difference at the group level was observed for intensity of riPH (P = 0.01126; Fig. 6). Importantly, no changes between day 1 and 5 were observed for other sensations that accompany riPH (passivity experiences: P = 0.9095, Supplementary Fig. S7; loss of agency: P = 0.20910, Supplementary Fig. S8), nor for any of the control questions (Supplementary Fig. S9). In line with our hypothesis, we show that fMRI-NF training targeting the PH-network, in fully blinded participants, increased their proneness to report riPH, but did not alter their sensitivity to accompanying and control sensations.

Fig. 6: riPH ratings before and after neurofeedback training.
figure 6

Raw ratings for the riPH question are shown for both PH-inducing and control conditions, in the sessions before (A) and after neurofeedback training (B). Cumulative linear mixed model analysis showed no statistical difference between the conditions before training. After completing neurofeedback training, there was a significant difference (p = 0.01126) between the two conditions, and more participants were sensitive to riPH. Data was jittered for visualization.

Neural changes after fMRI-NF training in participants that became sensitive to riPH reveal influence of up-regulation mechanisms

The present fMRI-NF protocol targeting the PH-network led to the development of up-regulation mechanisms of the PH-network (i.e., through changes in transition probabilities that re-enforced it) and an observed decrease in recruitment of the PH-network during the baseline condition, as well as a behavioral increase in sensitivity to riPH after training. Next, we investigated whether fMRI-NF training also led to neural changes in the brain dynamics of riPH, by comparing brain data during induction, before training (day 1) and after training (day 5). We used a similar temporal modeling analysis as done for the fMRI-NF sessions, but now modeling the differential networks’ occurrence and transition probabilities between day 5 and day 1, and taking into account participants’ sensitivity to riPH after training, as well as the number of high-regulation runs obtained during training. This revealed that the difference between occurrences on day 5 and day 1, changed in a different manner for each brain network, depending on how many high-regulation runs the participants achieved during training, and depending on their post-training sensitivity to riPH (triple interaction P = 0.00125, Supplementary Table S13). Participants, who became sensitive to riPH following fMRI-NF training, presented changes in the occurrences of the PH-network when comparing the PH-inducing condition to the control condition, and depending on the number of high-regulation runs that were achieved (P = 0.03555; Fig. 7A; Supplementary Tables S14S16). This interaction indicated that the more successful a participant was during fMRI-NF training, the more occurrences of the PH-network were observed in the PH-inducing condition after training, and the less occurrences of the PH-network were observed in the control condition. Critically, this effect was absent for participants who did not become sensitive to riPH after training (Fig. 7B). Further control analyses, repeating the same procedure with other sensations tested during induction such as passivity experiences and loss of agency, did not show any specific changes related to fMRI-NF training (Supplemental Tables S17S21). In sum, participants who became sensitive to riPH displayed neural changes compatible with the up-regulation mechanisms learned during training and with the observed changes at baseline. Specifically, up-regulation was present after training as increased occurrences of the PH-network during the PH-induction condition, and the changes at baseline were present after training as decreased occurrences of the PH-network during the control condition. Importantly, both were more present after training the more the participants had been successful during said training. The same type of analysis for differential transition probabilities analysis did not show any significant differences when comparing pre and post training sessions (see Supplemental Tables S22S27).

Fig. 7: Difference in PH-network occurrences when comparing after vs. before training.
figure 7

The difference in occurrences of the PH-network for after vs. before neurofeedback training, is shown for each condition respectively, in function of the number of high-regulation runs achieved during training. A significant interaction between PH sensitivity, network and number of high-regulation runs was observed (p = 0.00125). A This difference is shown for participants that were sensitive to PH induction at the end of training (N = 8). A significant interaction between number of high-regulation performance runs and condition was observed (p = 0.03555). B Same difference shown for participants that were not sensitive to riPH on the last day (N = 12). No interaction is observed.

Discussion

We combined MR-robotics capable of inducing a clinically-relevant hallucination (i.e., riPH)36,38 with real-time fMRI-NF, to investigate whether volitional control over the dynamics of a prototypical brain network characteristic of PH (i.e., the PH-network40), could change sensitivity to the PH-like state induced by our robotic protocol. During three training days, participants successfully learned to up-regulate the PH-network through continuous closed-loop feedback about its activity. This, ultimately led to an increase in sensitivity to riPH after training. Furthermore, once training was over, participants who became sensitive to riPH showed neural changes related to the PH-network neural regulation mechanisms, with these changes being more prominent the more successful participants were during training.

Real-time fMRI-NF has been used to let participants achieve volitional control over brain activity, with aims such as reducing pain51, PD tremor52, or clinical symptoms related to attention-deficit/hyperactivity disorder53. Targeting functional connectivity through fMRI-NF has shown particular promise in regulating neural markers in several neuropsychiatric conditions (e.g., schizophrenia, autism-spectrum disorder, depression)42,44,45. To date, attempts to modulate hallucinations by regulating activity or static connectivity of brain regions were successful in such neural regulation, but had limited success in linking the targets of NF regulation to decreases in general disease burden or hallucinations54,55. Motivated by recent advances in dynamic connectivity of hallucinations56,57 and the capacity to target a specific and clinically-relevant hallucination (i.e., PH) through its robotic induction in the MRI36, here, we paired the latter robotic system with fMRI-NF, to modulate a whole-brain network whose dynamics underlie PH-induction40 while participants actually experienced PH as induced by the robotic system. By merging these techniques, we demonstrate that participants can learn to up-regulate the PH-network, alter its associated brain dynamics, and thereby increase the proneness to riPH after the training. Importantly, this increase in proneness was specific to riPH, because it was not observed for any control questions or other sensations that typically accompany riPH (i.e., passivity experiences, loss of agency), even though participants were fully blinded to the training goals. This type of closed-loop setup using real-time feedback, provides direct evidence for the neural specificity between dynamics of the PH-network and riPH, and further demonstrates how the impact of fMRI-NF on dynamic network connectivity can significantly alter behavioral outcomes. The present findings link neural modulation to behavioral changes, beyond what was achieved in previous attempts to modulate hallucinations. For example, two different fMRI-NF studies aimed to reduce hallucinations by down-regulating the activity of the superior temporal gyrus (STG) or the STG-IFG connectivity. While both studies achieved successful regulation of their targets, links to reduction in hallucination burden were not found when investigating the targets alone, but were rather found when investigating connectivity changes between the target and untargeted regions, which limited the interpretation of the effect of NF regulation54,55. Another study did achieve a direct reduction in hallucination burden by down-regulating network connectivity between the DMN and an attentional network58, however, this study enforced a meditation technique as neurofeedback regulation strategy, which itself has been shown to reduce hallucination severity59 (the study did not control for this confound). In our study, the direct intra-participant and intra-trial comparison of feedback to baseline condition controlled for several potentially confounding factors, such as differences in motivation, difficulty, and manipulation of the robot across participants, as well as placebo and spatially unspecific effects60. Moreover, we further showed that sensations which typically accompany riPH, but whose neural mechanisms were not targeted by our fMRI-NF, were not affected by the NF training in the way riPH was.

The increase in proneness to riPH that we observed after three days of fMRI-NF training, was accompanied by neural changes of the PH-network on the last day of testing (i.e., after training). Specifically, those participants who became sensitive to riPH and who were successful during training, showed higher occurrences of the PH-network during the PH-inducing condition and less occurrences of the PH-network during our carefully matched control condition. Again, this was specific to the induced hallucination (riPH) and was not linked to any of the other sensations that typically accompany riPH. Crucially, while previous studies showed that fMRI-NF can change activity or average connectivity of hypothesized markers of clinical symptoms such as auditory-verbal hallucinations54,55,58, links to symptomatic change were not found for the targets only, but rather for connectivity between targeted and untargeted regions, or relied on general symptom burden. Our findings demonstrate that targeting a whole-brain network through fMRI-NF, brings about specific changes in riPH and leads to specific neural changes after training is concluded. We argue that the observed change in hallucination proneness, was likely enabled by the participants neural regulation capabilities centered in dynamic connectivity changes, for which increasing evidence points to a central role in hallucinations across multiple diseases (e.g., schizophrenia56,61,62, Parkinson’s disease57,63,64, see ref. 65 for review on auditory-verbal hallucinations).

The neural changes observed in our participants after training, closely match the up-regulation mechanism that was achieved during training. When high regulation was achieved, feedback scores were increased during the feedback condition by rearranging transitions between brain networks in a way that favored transitions from any network to the PH-network (and to two closely related networks that can be considered subnetworks of the PH-network: sensorimotor- and DMN-). This matched the network dynamics we have previously identified to underlie PH-induction40 and implies that feedback solely on the activation of the PH-network, paired with our robotic protocol, was sufficient to help participants modulate the neural mechanism underlying riPH. Moreover, it also demonstrated that fMRI-NF can be used to impact widespread dynamic network connectivity. This is a significant finding, given that dysfunctional network dynamics are increasingly shown to play a major role in hallucinations in several medical conditions56,57,61,62,63,64,65,66. Yet, targeting whole-brain networks with fMRI-NF, as done in our work, is still rare. A recent study on attention found that healthy individuals could regulate average connectivity between the DMN and an attentional network, when presented with intermittent feedback based on the average correlation between these networks over a block; however, these changes did not have a significant impact on attentional performance after training46. Another study found that schizophrenic patients could regulate the ongoing connectivity between the DMN and another attentional network; however, as mentioned previously, it remained unclear whether neurofeedback or meditation were responsible for the changes in symptomatic hallucinations after training, due to lack of appropriate controls58. Moreover, no previous study targeting hallucinations with fMRI-NF54,55, accounted for potentially accompanying (involuntary or voluntary) physiological changes (i.e., breathing) that may be confounded with neural regulation67, when comparing groups or conditions. In our study we corrected for physiological and various other potential confounding factors, in order to ensure that regulation was in fact achieved through effective mental strategies that led to an increase in sensitivity to riPH after training. Additionally, we also verified that the sensory stimulation from the auditory feedback was not associated with these neural regulation mechanisms, as no overlaps were found with the PH-network. Instead, we observed activations in the post-central gyrus, the caudate nucleus and thalamus.

In addition to up-regulation of the PH-network, we also observed that during the baseline condition, the PH-network was recruited less in high-regulation runs as compared to other runs, and instead the sensorimotor+ network was recruited more. Given that the latter contained deactivations of the PH-network, this led to higher feedback scores during the respective block’s feedback condition, as it increased the differential between the two conditions. Such a mechanism cannot be explained by compensatory network deactivation following hyperactivation periods because hyperactivation of the PH-network was not the up-regulation mechanism and because such compensation should have been already observed during rest (i.e., in the condition immediately following the feedback condition). Nonetheless, our setup was not designed to assess volitional down-regulation of the PH-network, and while it is possible that participants learned implicitly that better “clearing their minds” during baseline improved the next feedback (similarly to implicit NF, see for example:68), future studies should directly assess down-regulation of the PH-network. This is also interesting, as compared to up-regulation, down-regulation has been a more common goal of neuropsychiatric fMRI-NF studies54,55, and indeed the clinical application of our NF setup would be to down-regulate network dynamics, for example in Parkinson’s disease where PH are prevalent. In fact, PH is not just frequent in Parkinson’s disease18,26, it also occurs early in the disease course (sometimes even before the onset of motor symptoms)27 and has been associated with a more severe form of the disease marked by the appearance of other hallucinations, cognitive decline, and higher mortality28,29. Hence, by down-regulating neural correlates of PH or related processes, our NF setup may reduce hallucinations and modify related cognitive decline10. This could be achieved by countering temporal processes of network dynamics as described in Dhanis and colleagues (2022)40 and as done in the present work. Such approach could be particularly promising as others have also found that temporal transition processes between network states rather than network configuration, underpin hallucinations in Parkinson’s disease57. Alternatively, processes of other networks could be targeted (instead of the PH-network), as abnormal pairings between the DMN, visuo-attentional, and visual networks, have also been found and hypothesized to underpin hallucinations in Parkinson’s disease64,69.

There is considerable potential in adapting the current setup to regulate dysfunctional network dynamics and decrease hallucination burden in Parkinson’s disease or dementia with Lewy bodies57,63,64, but some limitations of the present study should be considered. First, the number of participants sensitive to riPH was lower than usual when considering previous riPH studies in the MRI38,40. Nonetheless, our protocol included a strict analysis of regulation performance, controlled for a series of potential methodological confounds, and our analyses were further controlled for sensations that accompany riPH, of which none showed a change with fMRI-NF. Second, we could not continuously assess riPH during fMRI-NF training and did not repeat hallucination ratings during training as to not introduce bias. Future studies might benefit from the use of implicit measures70 to assess riPH continuously. Third, future studies should directly investigate down-regulation or bi-directional regulation. Finally, additional research is needed to understand mechanisms of neural regulation that fell outside of the scope of our study. In particular, why did some participants retain neural changes after successful training, while others did not, despite achieving comparable training success, remains to be investigated. Also, the role of specific network dynamics in neurofeedback control needs to be further studied, as it is possible that subnetworks of the PH-network (i.e., sensorimotor- and DMN-) had an active role in regulation, such as driving transitions back to the PH-network, given their major activations over regions tied to cognitive attentional control in NF regulation (IPL and dlPFC41,71,72).

Our goal was to investigate whether volitional control over a specific brain network related to riPH was possible, and whether such neural self-regulation would change proneness to report riPH and lead to neural changes. Our findings confirm that volitional control is achievable, modifies sensitivity to the induced hallucination, and show that fMRI-NF leads to neural changes in successful and sensitive participants. In particular, condition-dependent, increases in transitions probabilities of the target network, show that individuals can exert control not just on singular or static aspects of brain activity, but over large-scale network dynamics. We argue that the relevance of these findings goes beyond the specific case of PH and the neuroscience of hallucinations, as hallucinations are major symptoms in several prevalent diseases6,8 and because hallucinations serve as markers for disease progression and severity in several neurodegenerative diseases8,29. Accordingly, we propose that fMRI-NF paired with our robotic setup (shown to trigger PH in these patients38), could be a non-pharmacological neuromodulation therapy to counter dysfunctional network dynamics. Such treatments may target networks linked to Parkinson’s disease57,63,64 or the network related to PH. As PH is an early non-motor symptom of Parkinson’s disease27, and precedes cognitive decline10, an adaptation of the present fMRI-NF method may potentially down-regulate PH and also slow down the progression of neuropsychiatric and associated neurocognitive dysfunction in Parkinson’s disease patients.

Methods

Participants

28 healthy individuals were recruited to take part in this experiment. Out of these, two participants retracted from the experiment citing time constraints, one participant became unreachable throughout the sessions, another one was excluded due to the use of psychoactive substances between sessions, and four did not pass the anamnestic interview (see below) due to substance use, a neurological atrophy disorder, and potential psychiatric disorders. The remaining 20 participants (13 male) had an average age of 25.25 years (±3.19; range: 21–32), and were all right-handed according to the Edinburgh Handedness Inventory73 (0.68 ± 0,19; range: 0.35–1). All participants gave their informed consent prior to the start of the experiment, following the Declaration of Helsinki, and the study was approved by the local ethics committee of the Canton of Geneva, Switzerland. All ethical regulations relevant to human research participants were followed.

Interview

Prior to the start of the experiment, participants took part in a anamnestic screening session, to evaluate their current medical status, use of medications affecting the nervous and muscular system, history of neurological disorders, substance abuse, and psychiatric disorders. Recruited participants did not have any diagnosed neurological or psychiatric disorder, nor used medications or psychoactive drugs for a period of two weeks preceding the start of the experiment. Recruited participants were routinely asked about their use of medications, and recreational drugs.

MRI data acquisition

Functional image acquisitions were performed at the MRI facility of the Campus Biotech (Geneva, Switzerland), on a Siemens MAGNETOM Prisma 3 T scanner, using a 64-channel head-and-neck coil. For the sensorimotor task and NF task, echo-planar sequences were used (EPI, TR = 1.5 s, TE = 31 ms, with a flip angle of 64°, GRAPPA = 1, SMS = 4), with an in-plane resolution of 2.0 × 2.0 mm2 (108 × 108) and slice thickness of 2.0 mm (no gap, 68 slices, FOV = 216 mm, slices were aligned to the bicommisural plane with the acquisition volume aligned to cover the entire brain hemispheres). Anatomical images were acquired with a T1-weighted MPRAGE sequence (192 slices, FOV = 256 mm, TR = 2.2 s, TE = 2.96 ms).

Robotic procedure for PH-induction

The tactile stimulation presented to the participants during the experiment was administered by a robotic system composed by two main components, a front-robot, and a back-robot concealed below an adapted platform-bed in the MRI36. The front robot included an extended lever that participants used to manipulate the front-robot by performing movements with their right arm. Participants were instructed to pivot movements from the elbow and wrist, but to avoid moving the upper arm and shoulder as to minimize head movements. The back-robot then delivered tactile feedback to the participant’s back (through a gap in the middle of the MRI platform bed) by mimicking the participant’s movements either in real-time (synchronous manipulation: control condition), or with a small delay (500 ms – asynchronous manipulation: PH-inducing condition).

Experimental paradigm

Participants completed five fMRI sessions on five different days, with at least one day of break between each session. Significant efforts were taken to have all participants complete the experiment under two weeks, however, due to the COVID-19 pandemic this was not possible for one participant who completed it in 28 days, and another who attended day 1 twenty days before starting day 2 (first day of training), and then completed the rest in 12 days. The average completion time excluding these two cases was 12.52 days (±3.66; range: 8–19 days).

The first day was dedicated to the assessment of the participants’ baseline sensitivity to riPH before fMRI-NF training. The second, third, and fourth days, were dedicated to fMRI-NF training and transfer runs. In particular, we aimed at 3 runs of NF per day followed by one transfer run. The fifth day was identical to the first, dedicated to the assessment of sensitivity to riPH after training. For all parts of the study, participants had their eyes covered as typically done in previous riPH paradigms36, and wore in-ear earphones (and on-ear sound protection) that were used to both talk with the participants during breaks and to provide auditory feedback during neurofeedback runs.

Days 1 and 5: assessment of riPH

Participants performed two runs of the robot manipulation task, devised to induce riPH36,38 in the fMRI scanner. Each run consisted of 16 blocks of 30 s of robot manipulation interleaved by periods of rest of 15 s. Robot manipulation was performed either in the PH-inducing, or in the control condition. Next, participants performed two isolated blocks of robot manipulation in each condition (random order, counterbalanced across participants), each followed by a questionnaire tailored to assess the intensity of different subjective experiences during each condition (7-point Likert scale, Supplementary Table S28). This included but was not limited to: riPH - “I felt as if someone was behind me”, Passivity Experiences – “I felt as if someone else was touching my body”, and Loss of Agency – “I felt as if I was not in control of my movements or actions”.

Days 2, 3, and 4: fMRI-NF training

During the training sessions, participants performed one run of an auditory functional localizer, three runs of neurofeedback, followed by one transfer run. For the auditory functional localizer participants laid in the MRI without moving and listened passively to all the levels of auditory feedback presented to them in a random order. Each level was presented for 23.5 s and was followed by 6.5 s of rest. The neurofeedback runs were composed by one habituation block, and six neurofeedback blocks. The habituation block was identical to the neurofeedback blocks (further described); however, no feedback was given. This block had the role of stabilizing our real-time algorithms. Neurofeedback blocks were each made up of a rest condition, followed by a baseline condition, and a feedback condition. During the rest condition, participants laid in the scanner without moving the robotic system, and without receiving any feedback, for 14 s. After a single beep, the baseline condition started, and would last for a duration of 21 s. Because this condition would be used as a comparator to the active feedback condition, we made sure to match it as closely as possible in all active parts that were not related to mental strategy: participants moved the front-robot providing tactile stimulation to their back from the back-robot in real-time with their movements, and participants also listened to the minimum level of auditory feedback during this condition, regardless of ongoing brain activity, so that the baseline and feedback conditions were matched as closely as possible. Furthermore, participants were instructed that at this point they should not engage in any regulation strategy, and instead should try to “clear their minds”. After two beeps, the feedback condition started for a duration of 60 s. Here, the robot switched to the asynchronous condition, providing tactile feedback to the participant’s back with a delay of 500 ms with respect to their own movements. The auditory feedback became variable and derived from the participants’ ongoing brain activity. This feedback was continuous and provided as closely to real-time as possible, meaning with a delay of at least the haemodynamic delay (~5 s) and processing time (~1 s). Participants were made aware of the feedback’s intrinsic delay. They were also instructed that during this condition, they should try to achieve the highest level of auditory feedback (see next sub-chapter) by means of a mental strategy of their choice. More details regarding the instructions to participants can be found on “Guidance for participants regarding neurofeedback training” in the supplementary information.

Transfer runs had an identical block design to the neurofeedback runs, however, there were only two blocks during which no feedback was presented. Participants were informed they should still try to deploy the mental strategies learned during neurofeedback runs.

Real-time data processing and neurofeedback

For real-time processing of incoming functional images, and presentation of feedback, we made use of the open-source software OpenNFT74, running on MATLAB 2019b and Python 3.5. In-house adaptations were made to provide a feedback score based on full-brain correlations, to provide auditory rather than visual feedback, and to have OpenNTF communicate with another computer in the network which was controlling the robotic device used for PH induction.

Pre-processing

Throughout the runs, data was directly reconstructed in the scanner’s console and sent to the computer running OpenNFT. The standard OpenNFT pre-processing pipeline was used, which included spatial realignment to the template scan (obtained in a short sequence at the beginning of each day), re-slicing to compensate for head movements, and spatial smoothing (FWHM = 5 mm).

Processing

Linear trends were removed from each voxel in real-time, using an adapted filter, which also functioned as a high-pass filter. Before being used for NF computations, the volumes were also masked with a participant-specific gray matter mask.

Neurofeedback

The processed and detrended data of each full-brain volume was directly correlated with the PH-network brain map as obtained from a previous independent dataset40. This spatial correlation between ongoing brain activity and PH-network (NF-signal) was then used to compute a score that was fed back to the participants in auditory form as one of eight levels.

During the feedback condition, the feedback score at each time point was computed as the NF-signal minus the median NF-signal of the previous baseline condition, normalized by the minimum and maximum NF-signal values obtained in the window with a length of one block (67 scans, i.e., dynamic range: DR) (Eq. (1)). Crucially, this guaranteed that the feedback during the feedback condition had a changing baseline, as assessed by the previous baseline condition, and could also account for amount of movement performed with the robot.

$${Feedback\; Score}=\frac{{NFsignal}{{{{-}}}}{{{{\rm{median}}}}({NFsignal})}_{{baseline}}}{{\max \left({NFsignal}\right)}_{{DR}}-\,{\min \left({NFsignal}\right)}_{{DR}}}$$
(1)

Importantly, the dynamic range was also continuously adapted according to each participant’s performance. At the transition point from the baseline to the feedback condition, and if a participant’s feedback-score was in the upper quartile of the dynamic range for more than 80% of the previous feedback condition, then the upper limit was increased by 10% to make the task slightly more challenging. Conversely, if the participant’s feedback-score was in the upper quartile of the dynamic range for less than 60% of the previous feedback condition, then the upper limit was decreased by 10% to ease the task. No changes were applied if the feedback-score was in the upper quartile for 60 to 80% of the previous up-regulation condition. This process helped keep engagement for both the participants that were very good at the task, and for participants that had more difficulties. The reference values were determined during pilot experiments.

The auditory feedback consisted of an amplitude-modulated sound with main harmonic frequency of 50 Hz (complemented by second and third harmonics), producing a sound comparable to that of a wave approaching and receding. At its minimum level (0), the auditory feedback had an amplitude modulation following a frequency that allowed two waves per TR (~1.33 Hz). The number of waves heard per TR gradually increased by two, over the next six levels (level 1 had four waves per TR, ~2.66 Hz, level 2 had six waves per TR, 4 Hz, and so on). The translation from the feedback score that varied from −1 to +1, to auditory feedback levels, was done by attributing all values between −1 and 0 to level 0, and then dividing 0 to +1 in 7 intervals where for example, 0 to ~0.143 corresponded to level 1, ~0.143 to ~0.286 to level 2, and so on. At the highest level (level 8), no frequency modulation was used, allowing the reproduction of a constant tone. This allowed for participants to always know the highest level of auditory feedback implicitly, which was when the sound became constant (as explained to the participants: when the waves in the sound became “so fast” that they disappear). Simultaneously, participants were also constantly aware of the minimum level of feedback since this was presented during the previous baseline condition. This design guaranteed the typical advantages of a visual feedback (i.e., constant awareness of the minimum and maximum of the feedback on the display), but in an implicit manner. We nonetheless guaranteed that in every training day, before the experiment begun, but already inside the MRI, participants re-heard all auditory feedback levels.

Participants were informed that the auditory feedback was related to their own brain activity, however, they were blinded as to which specific aspect of brain activity the neurofeedback was related to (i.e., they did not know the brain activity of interest was related to the occurrence of a network involved in riPH). Participants were told that they should engage in mental strategies during the feedback condition, but not during the baseline condition. Nonetheless, they were told that their regulation performance could benefit from trying to “clear their minds” during the baseline condition. Further instructions regarding strategies were given: no engagement in physical strategies such as moving parts of their body unrelated to robot manipulation; no explicit change of the pace of the robot manipulation between feedback and baseline conditions; no engagement in breathing strategies; no strategies based on the MR scanner noise. Significant efforts went into the design of the instructions, making sure the participants understood the instructions while simultaneously not giving any information about the overall goal of the experiment. Ultimately this meant guaranteeing that participants remained blinded, but understood they had to find mental strategies, that we could monitor brain activity related to mental strategies, and that the auditory feedback would reflect whether or not they were regulating well (or in other words, in the direction the experiment wanted). Crucially, they did not need to know what our goal was and their instruction was to use mental strategies that would impact the auditory feedback in the desired direction. In that sense, participants remained blinded to the fact that regulation was aimed at the PH-network and attempted to change the perception of riPH. At the end of the last day of the experiment participants were asked what they thought the experiment was related to, in order to assess blindness. No participant assumed the experiment was related to PH-induction. The “Guidance for participants regarding neurofeedback training” in the supplementary information contains more details on the provided instructions.

Data analysis (offline – after all acquisitions)

Neurofeedback training performance

To assess the performance of each participant in neurofeedback regulation, we used non-parametric hypothesis testing. The null hypothesis is operationalized using surrogate data generation. First, for each neurofeedback run we generate 100.000 surrogates of the original NF-signal (Fig. 2A) by amplitude-adjusted phase randomization of the Fourier coefficients of the time series75 (Fig. 2B). Importantly, the surrogate time series obtained in this way preserves the autocorrelation properties of the empirical one. Second, by applying Eq. 1 to the surrogates, we compute the average feedback score for each surrogate, generating a null distribution of session averaged feedback-scores (Fig. 2B). It is important to take note that because the surrogates respect the temporal properties of the original signal, with increasing real modulation done by a participant, the null distribution will start to skew towards both tails (this derives from the feedback score equation and depends on whether the surrogate signal ups-and-downs align with the baseline or feedback conditions). These distributions in fact ensure the stringency of the process. Third, we establish a percentile threshold above which a run is considered as having high-regulation (Fig. 2B). Given the adaptive nature of the feedback, high percentile thresholds selected runs that showed more evidence for modulation of the PH-network between feedback and baseline conditions, while accounting for the NF-signal variability across runs and across participants’ task performance. We then used a Binomial distribution to compute the probability of observing that many successes (at the group level) as compared to chance – with chance here being defined by the threshold (i.e., a threshold of 70% would define random chance as: 70% non high-regulation and 30% high-regulation). The selection threshold was chosen at 84.1%, which corresponds to one standard deviation away from the mean in normal distributions (note that this remains true for two equally skewed in opposite sides normal distributions that are joined together). This decision was made as to prune runs with higher modulation between feedback and baseline conditions, but importantly, we demonstrate that the probability of obtaining the real feedback scores under the null distribution hypothesis of the surrogates is always extremely low for any selection threshold chosen (Fig. 2D).

MRI data preprocessing

Offline data preprocessing was done with standard CONN pipelines76, that in turn make additional use of SPM (Wellcome Department of Cognitive Neurology, Institute of Neurology, UCL, London, UK) and ART toolbox functionalities (Gabrielli lab, MIT). This pipeline included, spatial re-alignment and re-slicing, identification and removal of high-movement frames, followed by tri-linear interpolation of those data-points. Normalization to MNI space (resolution), and spatial smoothing followed (FMWH = 6 mm). Data denoising was further performed with the CONN toolbox to remove potential confounds related to, cardiac activity, respiration, movement, and related to the previous scrubbing-interpolation procedure. Linear trends were also removed.

Auditory functional localizer

We performed an activation-based analysis on the auditory functional localizer data obtained at the beginning of days 2, 3, and 4. Hence we used a generalized linear model approach, considering a predictor for the sound (repeated 8 times per run), another for rest, and a parametric modulator for the auditory feedback. Polynomial expansion was done for the 1st order. Contrasts of interest included main effect of sound, main effect of levels of auditory feedback (parametric modulator), and between both, i.e., levels of auditory feedback > sound. The latter revealed interpretation of the feedback beyond the sound.

Brain network extraction through co-activation pattern analysis

We performed CAP analysis to extract the main brain networks active during fMRI-NF runs (days 2, 3, and 4), transfer runs (days 2, 3, and 4), and pre/post training PH-induction runs (days 1, and 5). This was a two-step procedure.

For the neurofeedback runs, we followed the same procedure used in previous work, that originally identified the PH-network40. In brief, we used the TbCAPs toolbox77 to extract brain networks that co-fluctuated with the activity of two seed regions relevant for the induction of PH38, the pSTS, and the IFG. CAP analysis performs clustering with a k-means algorithm, of selected timepoints of the acquisition, where one or both of the seed regions exceed a z-score of 1 (time-points with a framewise displacement of 0.5 mm were scrubbed). The number of k networks for clustering was set at eight based on consensus clustering78, a measure of clustering stability across clustering’s performed with different number of k networks (Supplementary Figs. S10, S11). Finally, representative networks were obtained by averaging the time-frames that are attributed to the same k network. Timepoints where none of the seeds exceeded a z-score of 1 in activity, were marked as network0.

We used a spatial similarity procedure integrated in the TbCAPs, to attribute the CAPs identified during the neurofeedback runs, to the PH-induction runs and to the transfer runs. In sum, first the correlation between a scan and each of the networks is computed, second, a network is assigned to that scan, if the correlation value is above the 5th percentile of the distribution of correlations for that network.

Network/CAP metrics

We investigated the dynamics of brain networks through two metrics: occurrences, and transition probabilities. Occurrences can be defined as the perceptual occurrence of a network in the entire length of a condition. Hence, if the set DS,C contains all timepoints of a condition C for a participant S, the occurrences of network i are defined as the sum of an indicator function Networki [k] over the timepoints k, divided by the length of the set DS,C (Eq. (2)). Transition probabilities, refer to how different networks transition amongst themselves in time, and gives the probabilities of having a certain next network (j) after an initial network (i) (Eq. (3)). This should be seen as a 1st order Markov chain.

$${Occ}\left(i{{{;}}}{D}_{S,C}\right)=\frac{{\sum}_{k\in {D}_{S,C}}{{State}}_{i}[k]}{{{{|}}}{D}_{S,C}{{{|}}}}.$$
(2)
$${TP}\left(i,j{{{;}}}{D}_{S}\right)=\frac{{\sum}_{k\in {D}_{S}}{{\mbox{Network}}}_{i}[{\mbox{k}}]{{\mbox{Network}}}_{j}[{\mbox{k}}+1]}{{\sum}_{k\in {D}_{S}}{{\mbox{Network}}}_{i}[{\mbox{k}}]}.$$
(3)

Network/CAP metrics modeling – Neurofeedback and Transfer runs

To identify how brain dynamics are modulated by the different tasks, we have modeled the above network metrics using LMM using the package lme479 available for R (version 4.0.0).

For the NF and transfer runs, we modeled the metrics of occurrences of the networks, with predictors for network, condition, number of the run (absolute over days), and a binary predictor defining if the run achieved high-regulation or not (HR). Included in the model were also random-effect predictors for day and participants (example for modeling occurrences: Eq. (4)).

$${Occ} \sim {State}* {HR}* {Condition}* {Run\; Number}+(1{{{|}}}{Day})+(1{{{|}}}{Participant})$$
(4)

Model comparison based on the Akaike Information Criteria80 (AIC) revealed that the predictor for number of the run, gravely increased the complexity of the model, without adding explained variance. This predictor was hence dropped from the model.

Transition probabilities for the CAPs, were modeled in the same manner, however, instead of a predictor for network being used, two other predictors, for initial network and next network were used (Eq. (5)).

Notably our analysis strategy started with the highest hierarchical model (Eq. (4)), and if an interaction was detected we would then seek to further investigate that effect by decomposing the model. However, although our experiment included a considerable amount of data, the number of multiple comparisons for both these occurrences and transition probabilities can grow very fast due to the number of levels of the predictors. To address this we used an analysis strategy where models, for example with 3-way interactions, were decomposed for only one factor (rather than two), and further decomposition would only proceed for intermediate models where the remaining 2-way interaction was maintained significant (Simple Slope Analysis81). For example, if a model showed a triple interaction with network, high-regulation and condition, this model would be first decomposed for condition, which only has 3 levels, rather than immediately decomposed for all levels of conditions and network (30 levels). Analysis would then only proceed with decomposition per networks, if an interaction network:HR was maintained in the intermediate model.

Network/CAP metrics modeling – PH induction task

To identify how brain dynamics changed due to the neurofeedback training targeting the PH-network, we modeled the difference in network metrics between day 5 and day 1. For occurrences, this was done using predictors for network, condition, number of high-regulation performance runs during training, and sensitivity to PH after training (day 5).

For transition probabilities, instead of the single predictor for network, two predictors for initial network and next network were used, as seen in Eq. (5).

$${TP} \sim \,{{State}}_{{init}}* {{State}}_{{next}}* {Number\; of\; HRs}* {Condition}\\ * {PH\; sensitivit}{y}_{{day}5}+\left(1|{Day}\right)+\left(1|{Participant}\right)$$
(5)

Statistics and reproducibility

The regulation performance of 176 NF runs was assessed through a surrogate generation procedure per run, which relies on creating a null hypothesis congruent with the temporal properties of the original correlation NF-signal and analyzing how many runs outperformed the 84.1% quantile of the null for that run (full step-by-step description in the chapter: “Data analysis – offline (after all acquisitions) Neurofeedback Training Performance”). CAP metrics were analyzed for both training and before versus after training using LMM, and significance for each predictor was set at α = 0.05. To reduce multiple comparisons models were first analyzed in full and then decomposed if a significant interaction was present (simple slope analysis; note that further decomposition was then only allowed if contained within the original interaction). Multiple comparison correction was also still performed through False Discovery Rate (specifically using the Benjamini-Hochberg method). Behavioral data from the questionnaires performed before and after training from the participants (N = 20) was analyzed using Cumulative Linear Models also with significance set at α = 0.05.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.