Introduction

The brain interacts in a constantly changing environment and receives input that contains structure on multiple time scales1,2. One of the most characteristic examples of this temporal structure at multiple levels can be found in human narrative processing3. Indeed, to communicate with language the brain needs to integrate information, from short to long time scales, across phonemes, syllables, words and phrases4,5. It has been shown that processing for different time scales is distributed across cortical regions for visual6 and auditory7,8 input, revealing a hierarchical organization. Primary sensory brain areas operate on a fast time scale, reacting to sudden changes in input, while higher-order brain regions, operate on a medium to long time scale, integrating information across longer time periods.

Two recent studies have used intact/scrambled narrative task paradigms to investigate the temporal hierarchy during narrative processing7,8. In their human fMRI experiment8 the subjects were assigned in 2 groups, one listening to the intact narrative, the other one to a scrambled version of it. At certain sections in the narrative both groups would listen to identical content, while previously listening to different content. The study measured the time-constant for alignment of neural activity across subjects in each brain region, after the onset of the identical section of the narrative. The idea being that neural activity is different for different input, but once the same input is received the neural activity across the two groups should progressively become similar. It was shown that the alignment times increased from primary to higher order integrative regions in a temporal processing hierarchy. Interestingly, while the overall hierarchy is present, there are areas (e.g. in the default mode PFC) with alignment times that appear unexpectedly fast, and part of our interest is to understand how long-distance white matter pathways might contribute to this phenomenon.

To explain the empirical observation of temporal hierarchies in the cortex several studies proposed computational models9,10,11,12,13,14. These studies converge on the idea that indeed, network connectivity structure has strong causal influence on the temporal processing hierarchy. Specifically the study from Dominey et al.,12 explained the alignment time hierarchy as a function of local connectivity structure of the connections between brain areas. It exploited the inherent temporal sequencing capability of reservoir computing15 to implement the intact/scrambled narrative task of Chien and Honey8. Reservoir computing builds on networks of recurrent connected neurons, which receive input through an input layer, project the signal into a high-dimensional space through the neural dynamics and from which a readout layer reads a desired prediction16. Reservoir computation has been used previously, together with biologically inspired connectivity or with random connections to explain brain function or to test the efficacy of computation in biologically inspired networks17,18,19,20,21,22,23.

In this study, we used the intact/scrambled narrative task paradigm in a neural reservoir model to assess the effect of brain connectivity on the distribution of processing time scales. We began by extending on a previous study12 to explore the effect of long-range pathways in a reservoir network based on an exponential distance rule (EDR) connectome24. We show that while the EDR connectivity produces a temporal integration gradient, additional pathways can produce changes of time scales in both directions, slowing down faster areas and speeding up slower ones. This provides the theoretical basis for the study of computational connectomics where we examined the impact of different white matter pathways on the human temporal integration landscape. To do this, we used biologically realistic connectomes from human diffusion MRI data to create the reservoir. We then tested the impact on the temporal processing hierarchy of removing anatomically accurate major fibre bundles, and validated the ability of the model to predict the empirically observed hierarchy of Chien and Honey8.

Results

Connectome and fibre bundle reconstruction

An overview of the workflow used in this study to construct the connectomes and performing the intact/scrambled narrative task for this study is given in Fig. 1. We used the T1w MRI to obtain the Schaefer parcellation with 400 Parcels25. The fibre orientation distribution functions (fODFs) were estimated from diffusion MRI data and used to construct the full brain tractogram. The streamlines together with the parcellation resulted in a full brain connectome. The TractSeg26 toolbox was used to estimate bundle specific masks which were used to filter the full brain tractogram. The filtered streamlines together with the parcellation resulted in a bundle specific connectome.

Fig. 1: Image processing and analysis workflow.
figure 1

A We processed diffusion and T1w MR images from 100 subjects of the HCP dataset to construct a full brain connectome. The fibre orientation distribution images were used together with TractSeg to estimate volumetric masks for major fibre bundles and to extract bundle specific connectomes. B We constructed two reservoir networks from full brain connectomes, where the second network was lesioned by removing a specific fibre bundle bilaterally. The reservoir pair was then used for the intact/scrambled narrative task, where a narrative input signal is projected to one of the 7 Networks of the Schaefer parcellation. In the task, each reservoir receives subsequently the input twice, while the order of the first input is intact, the midsection of the second input is scrambled. For each brain region of the reservoir we obtained simulated differences in activity due to these different inputs. The last segment of the input contains the intact order again, thus one can measure the alignment time for each brain region of the network to show similar activity. Finally, we compared the difference in alignment time due to lesions.

Full size image

Next the connectomes were used to construct the connectivity matrix for reservoirs of neurons for the intact/scrambled narrative task. Each reservoir was run twice, once with the intact and once with the scrambled narrative. Thus, the intact and scrambled input timeseries were projected in separate runs to one of the 7 networks of the Schaefer parcellation and for each brain region we measured the alignment time as follows. The intact and scrambled input timeseries are identical for the first and last part of the timeseries. However, the mid-section of the scrambled timeseries is a reordered version of the mid-section of the intact timeseries. Thus, during the mid-section, the two reservoir instances receive different input from each timeseries, leading to different neural activity patterns. During the last part of the timeseries the input to the two reservoirs is identical again, for intact and scrambled conditions, thus the difference in neural activity will slowly decay. From this decay we measure the alignment time per brain region, as the number of timesteps it takes to reach half the difference as it was at the onset of the identical part of the timeseries. We performed the intact/scrambled narrative task twice, first with a full brain connectome as a reservoir and then again after removing connections corresponding to major fibre bundles, to assess the impact of the lesions on the alignment times.

Alignment times in an EDR based connectome model

It was previously shown that echo state networks (ESNs) have computational properties similar to cortical networks11,12 in the context of a continuum of alignment times. Experiments from Dominey et al.12 explored reservoirs with a connectivity matrix that follows an exponential distance rule, with exponentially decreasing connection strength for increasing distance, and narrative input projecting to the first 300 of 1000 reservoir neurons. This topology resulted in alignment times which increase with increasing distance from the input site, as observed in the human brain by Chien and Honey8. The insertion of an additional long-range pathway, outside of the EDR range, connecting the input site with neurons further away resulted in a speed up of alignment in the distant area, demonstrating how the network topology controls hierarchical processing. This provided an explanation of how certain frontal areas might have unexpectedly rapid alignment times due to direct access to input driven activity via long distance white matter pathways.

In this study, we extended this investigation by inserting additional long-range pathways systematically at all positions in the upper and lower triangle of the connectivity matrix at a distance of 300 neurons off the main diagonal (Fig. 2). This corresponds to bi-directional connections between areas in or near the input-driven neurons and areas far from input-driven neurons. This will aid us in the interpretation of alignment times in the full brain model.

Fig. 2: Alignment time change in an EDR based connectome due to long-range pathway insertion.
figure 2

A We built a reservoir using the EDR and projected the intact/scrambled narrative input to the first 300 neurons to measure the alignment time difference caused by the insertion of an additional long-range pathway (black rectangle in the connectome matrix). The pathway connects neurons 200–299 and 800–899 bidirectionally and causes a speed up (blue stars in the box plot), especially for neurons 800–899. Boxes indicate quartiles, with whiskers extending to the farthest datapoint within 1.5 times the inter-quartile range. B Same as (A) but inserting the extra pathway at positions 350–449 and 850–949, which leads to a slowdown of neurons in the range 300–499 (red stars in the box plot). C Systematic exploration of the effect of bidirectional pathway insertion at positions 300 neurons off the diagonal of the connectome matrix. The effect of either significant speed up or slowdown is shown for the neurons grouped at 0–99, 300–399 and 600–699. The positions of the specific connectomes from (A) and (B) are marked by the star and pentagon, respectively.

Full size image

We replicated the scenario of speeding up alignment by inserting a bidirectional pathway that connects neurons 200–299 and 800–899 (Fig. 2A). As in the original study, such a connection in an EDR network provides an explanation or prediction, that fast integration time constants in frontal associative areas could be in part due to their access to input-driven activity via long white matter pathways. In addition to this speeding of integration we can also observe the opposite effect, slowing down of alignment, which is possible by inserting a bidirectional connection between neurons 350–449 and 850–949 (Fig. 2B). In this case we see a slowing of “early” neurons (300–399, 400–499) near the input, due to the influence from distant, slower neurons (800–899). While the first pathway connects input driven neurons to non-input driven neurons, the second pathway connects only non-input driven neurons. We will distinguish them as input driven or associative pathways, respectively.

For 3 groups of neurons (0–99, 300–399 and 600–699) of the EDR reservoir we present systematically the effect of pathway insertion on alignment times (Fig. 2C, See Fig. S1 showing the effect for all groups of neurons of the reservoir). For neurons directly receiving the input timeseries (neurons 0–99), the insertion of an additional pathway either leads to a slowdown in alignment (red) or to no significant effect at all (white). These neurons are in principle the fastest as they are input driven. By influencing them with slower integrating neurons, they can be slowed. The group of neurons which are close to the input site, but do not receive input directly (neurons 300–399), may show different effects, depending on whether the pathway connects them to neurons receiving input (speed up), to neurons further away (slow down) or does not connect them at all (speed up). Finally, for neurons that are far from the input site, the insertion of an additional pathway either leads to a speed up in alignment (blue) or to no significant effect at all (white). Given this functional characterization of pathway effects in the schematized EDR reservoir, in the following we explored how alignment times behave in a more realistic brain network under lesioning of major fibre bundles.

Alignment times in a brain connectome model

The reconstructed brain connectivity matrices form the HCP dataset were used as weight matrices for the reservoirs, as illustrated in Fig. 1. Figure 3A shows alignment times after projecting the input timeseries to the visual network. While the brain regions receiving input are the fastest to align, the areas further away are slower. This distance dependency is visualized in Fig. 3B, as alignment times increase with increasing minimal Euclidean distance of each brain region to the visual input network. Interestingly not all the variance in alignment times can be explained by distance (see the \({R}^{2} \, < \, 0.5\)). Also, alignment times seem to saturate around 14 time steps. Certain regions in the orbitofrontal (OFC) and prefrontal-cortex (PFC) show fast alignment times compared with other regions having a similar distance from the visual network. They are marked by a black “+” and are positioned below the linear regression line.

Fig. 3: Alignment time change in a full brain connectome due to IFO fibre bundle removal.
figure 3

A Spatial distribution of alignment times when projecting the intact/scrambled narrative input to the visual network. B Alignment times (from panel A) increase with increasing Euclidean distance of a brain region to the input visual network. The black line indicates a linear model fit. Observations marked with a black “+” are indicating the regions of the OFC and PFC. C Same as panel B, but after IFO bundle removal. D Depiction of the IFO bundle. E Alignment time changes across the cortex when introducing a bilateral lesion of the IFO. The same regions as in (B) and (C) are marked with a black “+”. F Boxplot of alignment times of marked “+” regions pre- and post-lesion. Boxes indicate quartiles, with whiskers extending to the farthest datapoint within 1.5 times the inter-quartile range. All pre- and post-lesion distribution were tested against each other with a paired t-test and resulted in a p value < 0.05.

Full size image

Given this baseline condition we tested the effects of removal of specific white matter pathways on this pattern of alignment times in the brain connectome. While in the EDR based reservoir we had to insert a pathway, we removed major fibre bundles from the realistic brain connectome reservoir. In other words, if the insertion of a pathway in the EDR reservoir caused a speed up, the corresponding effect in the realistic brain reservoir would be a slow down due to fibre bundle removal, and vice versa. Figure 3D displays the IFO bundle, connecting the occipital cortex with the PFC, which will be removed bilaterally from the connectome. After the removal, globally the alignment times still increase with increasing distance, the marked regions of the OFC and PFC however are now closer to the linear regression line (Fig. 3C). The overall fit of the linear model also improved (\({R}^{2}=0.536\)). Similar observations were made when using streamline length and geodesic distance as metrics (Fig. S3). The effect of the removal is apparent on the cortical display in the OFC and PFC (Fig. 3E, F). The effect is pronounced on the left hemisphere.

In this case the IFO connects input driven areas in the occipital cortex to distant associative areas including PFC and OFC. Removing this connection yields a slowing of integration in these frontal areas. This corresponds to the pathway depicted in the schematized EDR reservoir in Fig. 2A, which connects distant neurons (800–899), analogous to PFC/OFC with input driven neurons, analogous to occipital visual areas. The presence of this pathway yields fast alignment times for these distant frontal areas, and its removal leads to their slowing.

We next examined the impact of different white matter pathways on this temporal processing, while keeping the projection of the intact/scrambled input on the visual network (Fig. 4). We found that, like the explorations with the EDR based reservoir, pathways can have slow-down and speed-up effects. The ILF is an input driven pathway, projecting from the visual network areas to the temporal pole, and its removal leads to a slowdown in anterior temporal lobe regions. This is similar to the IFO and the pathway in Fig. 2A. The UF is an associative pathway, connecting the temporal pole with the orbito-frontal cortex, and removing it leads to a speed up of both cortical areas. In the schematic EDR reservoir this is simulated by the pathway between 400–499 and 850–949 (see Fig. S2). Finally, the SLF_II shows both features, connecting brain areas very close to the input with areas at a distance, causing parts of the frontal lobe to slow down and parietal regions, close to the input network, to speed up, due to its removal. This is similar to pathways between neurons 300–400 and 600–1000 (see Fig. S2). The effect for bilateral lesions of the AF, MLF, SLF_I and SLF_III are shown in Fig. S4.

Fig. 4: Alignment time change in a full brain connectome due to fibre bundle removal.
figure 4

A Change in alignment times across the brain after bilateral removal of the ILF bundle. Within the transparent cortical surface, the geometry of the bundle if demonstrated. B Same as (A) but for the UF. C Same as (A) but for the SLF_II.

Full size image

We thus observe that pathways can have canonical effects as those observed in the EDR reservoir, as well as more complex effects on the temporal processing hierarchy. In order to obtain a more global synthesis, we tested how alignment times are effected by a single bundle lesion when projecting the narrative input changes across all 7 networks. This is exemplified for the AF in Fig. 5. For the control network, the AF functions as an input-driven pathway, connecting the frontal part of the network with the temporal lobe. The lesion of the AF thus creates a slow down in temporal regions. For the visual and limbic network the AF serves as a clean associative pathway with its removal causing widely distributed speed up across the cortex. For more spatially distributed networks, default, dorsal and ventral attention, the effect of the AF lesion is more complex. Sometimes the effect might be asymmetric and more pronounced on one hemisphere. This might be due to the asymmetric pattern of network and fibre bundles across hemispheres.

Fig. 5: Alignment time changes due to AF lesion as a function of the input network.
figure 5

A 3D rendering of the AF in both hemispheres. B Overview of the 7 Networks from the Schaefer parcellation. C Alignment time changes due to AF lesion using each of the 7 networks as input.

Full size image

Global effect of lesions on alignment times

The sum of positive or negative alignment time changes across all brain regions was used as a measure for global effect of fibre bundle removal (Fig. 6). The global effect differs for each fibre bundle and input network. Large lesioned fibre bundles, such as the MLF and AF show a large reduction in alignment times irrespective of the input network, while the alignment time increase is observed in much smaller bundles. Some bundles show specific affinity to an input network. The UF, being a small bundle, has a large impact on the limbic network, which the bundle connects specifically, but only a small impact on the somato-motor network. Similarly the AF increases alignment times when input is projected to the control network. Overall the default mode and somato-motor network appear to be the most robust across lesions, showing the small change in alignment time constant (ATC). When controlling for the size of the bundle (Fig. S5) the effect of the MLF and AF becomes rather small, while the UF still shows a strong impact on the limbic network and the ILF on visual and limbic networks.

Fig. 6: Global effect of fibre bundle removal for each of the 7 input networks.
figure 6

A Each box indicates the distribution of positive alignment time change (ATC) summed across all brain regions for all subjects, as an overall measure of the effect of a specific fibre bundle removal. The network, to which the intact/scrambled narrative was projected to, is indicated by the colour of the box. Boxes indicate quartiles, with whiskers extending to the farthest datapoint within 1.5 times the inter-quartile range. B Same as (A) but for negative ATCs. C The ratio of positive over negative ATCs. If <1 ( >1) indicates that a specific fibre bundle removal has an overall speed up (slow down) effect.

Full size image

Validation of alignment times from the brain connectome model

In order to verify our connectome reservoir model against empirical data from the human brain, we tested whether the cortical distribution of alignment times of the model would resemble the empirically observed pattern from Chien and Honey8. We thus performed the intact/scrambled narrative task, projecting the input timeseries onto the fastest 5 regions per hemisphere of the temporal lobe according to the alignment times presented in the Supplementary Fig. S4 in Chien and Honey8. These 10 regions are part of the auditory cortex and thus represent a plausible entry point for the narrative task. Figure 7A displays the alignment times on the cortical surface. Fastest alignment is observed from regions neighbouring the input areas in the superior-temporal cortex. Alignment times increase towards the inferior-temporal, occipital and parietal cortex. Slowest alignment is shown in the mesial frontal and prefrontal cortex. To validate the predicted alignment times from our model with those from the human brain we computed Spearman’s rank correlation coefficient ρ between the ordering of the 70 regions of interest (ROIs) from fastest to slowest from Chien and Honey8 Supplementary Fig. S4 and the ordering predicted by the model (Fig. 7B). In order to match the ROIs we used the 1000 Parcels 17 Networks Schaefer atlas for this reservoir. With \(\rho =0.685\) and p < 10-10 we obtain a good and significant explanation of the empirical data by our model. We tested the correlation between empirical and predicted alignment time ordering with several null-model connectomes generated using the brain connectivity toolbox27. We found that the original connectome performs significantly better than all the null models (p < 10-52) (Fig. S6). Furthermore, we tested how shortest and longest connections contribute to the spatial distribution of alignment times in our model (Fig. S7). We found that overall, the alignment times are dominated by shortest connections, their removal leads to a strong decline in the correlation between simulated and empirical alignment times. However certain long-range bundles, such as the AF, can still act as shortcuts, speeding up the alignment of distant brain regions.

Fig. 7: Comparison of the alignment time hierarchy between reservoir network and Chien and Honey, 2020.
figure 7

A Mean alignment times across subjects when projecting the intact/scrambled narrative to the 5 fastest regions (of Chien and Honey, 2020) in the temporal lobe of the full brain connectome. B Ordering the ROIs of the connectome according to their alignment time from fastest to slowest and comparing the ordering with the 70 regions of Chien and Honey, 2020. Each dot represents a brain region. Spearman’s rank correlation coefficient ρ = 0.685.

Full size image

Impact on the language network

Additionally, we wanted to investigate the impact of fibre bundle lesions on the alignment times within the language network when narrative input is projected to the same brain areas as in Fig. 7. We used the surface map of the language network from Lipkin, B. et al.28, which identified the network using fMRI in 804 subjects. Because of the importance of interhemispheric connections, via the corpus callosum, for auditory processing29 we added this special lesion to the analysis. Overall alignment times within the language network were most affected by the MLF and AF (Fig. 8). The removal of interhemispheric connections led to strong speed up of alignment. When controlling for the bundle size the ILF and UF are the most impacting bundles (Fig. S8). When simulating unilateral lesions, we observe a stronger impact on the left hemisphere compared to the right one (Fig. S9).

Fig. 8: Impact of bilateral lesions on alignment times within the language network (black and white inflated cortex) when the input timeseries is projected onto the auditory regions from Fig. 7(A).
figure 8

A The boxplot shows positive alignment time change (ATC) summed across all brain regions for all subjects, as an overall measure of the effect of a specific fibre bundle removal. Boxes indicate quartiles, with whiskers extending to the farthest datapoint within 1.5 times the inter-quartile range. B Same as (A) but for negative ATCs.

Full size image

Discussion

The observation of temporal processing hierarchies, with rapid temporal processing in input driven areas, and progressively slower temporal processing in progressively associative areas is a prevalent physiological property of the primate brain6,8. In the current research, the impact of connectivity structure upon the shape of such hierarchies has been demonstrated.

The first set of experiments investigated the effects of a highly schematized EDR connectivity on the temporal processing hierarchy. This local connectivity produces a form of flow of information from input driven to more associative areas via traversal of local connections. This flow is expressed in the temporal processing hierarchy as a continuous gradient of alignment times. We demonstrated how the insertion of long-distance connections could modulate these alignment times, speeding up distant areas by providing them with input driven activations, and slowing down areas proximal to the input by providing them with slower associative activations.

Based on these schematized observations, we then performed an exhaustive exploration of the effects of removal of specific white matter pathways on the temporal processing profile in reservoir networks with connectivity derived from human connectomes. As in the more abstract simulations, we observed that removal of specific pathways could produce increases or decreases in alignment times, depending on the input-driven status of the projection sites of the pathway. We also observed that the effects of pathway removal could be observed not only in the target regions of the pathway, but well outside of these regions as well. While these explorations can be easily compared to empirical experiments involving auditory8, visual6, or sensory input, comparisons with other functional networks may be less straightforward. Nonetheless, we utilized our computational approach to make predictions about the processing in these different networks and the role of fibre bundles in their functioning.

We started the systematic exploration of fibre bundle removal by projecting input to the visual network, with the removal of ILF and IFO producing increased time constants in the anterior temporal and orbitofrontal cortex, respectively. This means that the presence of these pathways can produce integration time constants in frontal areas that are smaller, faster, than would otherwise be expected from their anatomical location. Both pathways are involved in a multitude of cognitive processes. The integrity of the ILF correlates with functions such as object and face recognition, reading, semantic and emotional processing30. The IFO also is involved in semantic speech processing, visual recognition and reading and writing tasks31.

We found a strong slowing down effect when removing the uncinate fasciculus while projecting input to the limbic network. The uncinate fasciculus, being an integral part of the limbic system, connects the orbitofrontal cortex with the anterior temporal lobe32. It is involved in reversal learning, emotional regulation, reward processing and long-term memory retrieval32,33. The integrity of the white matter bundle, measured by fractional anisotropy, is related with performance in these domains. Also a clear correlation between functional connectivity of temporal lobe and prefrontal cortex with FA of the uncinate fasciculus has been demonstrated34. The MLF showed widespread effects on alignment times across the brain. However, its functional importance is not yet well understood. A few studies suggest it to play a role in language and auditory processing35,36. While performance in the different cognitive domains and functional connectivity has been correlated with integrity of the fibre bundles, the change in time scales during tasks has not yet been related to these specific white matter fibres. Future studies would be valuable to further test the prediction of our model.

We found that the SLF III and the AF had a strong effect when the input was projected on the frontoparietal control network. This is in line with recent studies linking functional resting state networks with the underlying white matter connectivity37, in which the AF and SLF were involved in multiple of the fronto-parietal-temporal networks.

The time constants seemed least effected by bundle removal when input was projected to the somatomotor network. This may be because of the selection of fibre bundles, of which none directly connects to these regions. Different corticofugal pathways, connecting the cortex with the thalamus, might show a stronger impact in these scenarios, but were beyond the scope of the current study38.

We observed widespread effects of bundle removal, not only at the direct connection sites. Similar observations have been made in functional connectivity due to stroke lesions. Disrupted functional connectivity was not only observed at directly structurally connected region, but even across indirectly connected functional networks39,40.

In addition to characterizing the effects of removal of specific pathways, we compared the distribution of alignment times in the intact human connectome based model with those from empirical human data8 and found a good and significant correlation. The model with a human connectome could replicate the increase in alignment times from superior temporal towards inferior-temporal, occipital and parietal cortex with slowest alignment predicted in the mesial frontal and prefrontal cortex. This underlines the fact that functional dynamics follow along structural scaffolds. A limitation remains that we could only compare to 70 empirical regions, while the model predicts alignments times for the full brain. Next, we tested how the alignment times in the language network would be influenced by specific bundles. We observed that overall, the MLF and AF cause the largest perturbations. The AF has long been associated with the dorsal stream of language processing, associating sounds with motor representation41. However, the role of the MLF remains rather unclear, it has been associated with the ventral stream for decoding of the meaning of speech in some studies42. Interestingly the removal of interhemispheric connections caused mainly a speed up of alignment, showcasing a reduction of integration of neural activity across the hemispheres. This is in line with observations in split-brain patients, whose corpus callosum has been severed, which show impaired performance in dichotic listening tasks29. When controlling for bundles size, both the ILF and UF emerge as bundles having a large impact. Both pathways have been previously associated with the ventral stream of language processing41. The other bundles, IFO and SLF_I/II/III, are associated with ventral and dorsal stream, respectively.

While we tested how the hierarchy of time scales emerges from the connectome in a task paradigm, we did not add heterogeneous intrinsic times scales across neurons of the reservoir. In previous studies it was shown that brain regions work at different time scales during rest and which are correlated with the cortical myelin content and gene expression maps1,43,44,45. These intrinsic timescales were added in another computational study using a macaque connectome, by introducing heterogeneous recurrent excitation across brain regions, based on anatomical observation14. It was shown that in absence of the connectome, the time scales of the simulated brain regions followed a clear hierarchical gradient. Introducing long range connections however was able to break this hierarchy apart and created surprisingly long-time scales in areas lower in the hierarchy and vice versa. We found a similar effect of long-range white matter fibre bundles in the human connectome. The introduction of heterogeneous intrinsic time scales in model with human connectomes should be investigated in a future study and might be able to increase the fit of empirical and simulated task paradigms even further.

While we investigated the effect of large, long-range fibre bundles on time scales, we did not specifically test for the effect of short-range U-fibres46, connecting adjacent gyri, or intra-cortical connections as simulated with neural field models47. Both could be integrated in future studies investigating neural time scales, most suitably in combination with high-resolution, atlas independent connectomes which estimate connections between vertices of the cortical grey matter mesh48.

Another limitation of our study is the use of anatomical landmarks only, to register the template brain atlas to the subjects. The HCP offers a multimodal parcellation with better personalized functional area delineation49. However we used the Schaefer parcellation25 to compare our predictions with the ones from Chien and Honey8. We expect future studies with more refined and personalized parcellation in task and rest fMRI to give even better model predictions.

Most computational studies for brain dynamics use neural mass models, which are systems of differential equations, often derived from detailed spiking neural networks using mean-field approaches50. Those models can display intrinsic activities, while our model is purely input drive. The choice of our model was motivated by previous studies showing that reservoir networks can perform narrative event processing11. However, distribution of time-scales across cortex also arise in models using more biologically inspired neural masses14.

We assume the structural connectivity to be static for the time scale of interest of this study. This agrees with empirical observations of white matter plasticity happening on longer timescales of days, weeks, month and years. However local synaptic plasticity effects are acting on a faster time scale51. They have been shown to lead to global activation changes, even when LTP is induced only locally52. Thus, it cannot be excluded that plasticity affects processing on the time scale of interest in this study. Future modelling and empirical studies are needed to disentangle this effect.

Part of the initial theoretical motivation for this research was to understand how discontinuities in the temporal processing hierarchy could be accounted for12. Chien and Honey elegantly demonstrated a cortical hierarchy of narrative integration time constants, but they also showed that within this hierarchy there were frontal cortical areas whose time constants were faster than would be expected from their position in the anatomical hierarchy8. We hypothesized and demonstrated that such discontinuities could result from specific long distance white matter connections that linked input driven areas to these frontal areas. This is consistent with recent observations from Betzel et al. 2018 who “ claim that the primary function of long-distance connections is to deliver unique inputs to brain areas and to serve as novel targets for brain areas’ outputs, thereby enhancing those areas’ functional diversity”53. Our demonstration that specific long-distance connections modulate the temporal integration of their target areas is an example of this function.

Here we began to characterize how long-distance connections between potentially distant brain regions can have profound impacts on a temporal processing hierarchy. It is of significant interest that such long distance connections have been demonstrated to likewise play a crucial role in state of the art deep learning architectures54,55. The review from Oyedotun et al. 202156 discusses how excessive depth of layers in deep neural networks (DNN) can result in optimization problems where the training set cannot be successfully learned. They further explain how these difficulties are alleviated by using skip connections of identity mappings from shallow to deep layers where the proposed DNN with skip connections is referred to as a residual network (ResNet)57. Huang generalized this notion of connectivity such that each layer obtains additional inputs from all preceding layers and passes on its own feature-maps to all subsequent layers, yielding improved performance with less computational cost58. Such skip connections which play a crucial role in the stabilization of activity in DNNs can be considered as functionally analogous to white matter pathways in the primate brain. In both cases, when propagation of information via local connections over excessive distances becomes problematic, long-distance shortcut connections become crucial components of the system architecture. This provides a potentially new outlook on the functional significance of human white matter pathways, and a provident venue of exploration between the functional consequence of connectivity in brains and deep learning models.

Methods

Image processing and connectome construction

We used minimally preprocessed neuroimaging data of 100 subjects (first 100 subjects in numerical ascending order, age range 22–35, 56 female) from the human connectome project S1200 data set59,60. For each subject we downloaded the minimally preprocessed T1w and diffusion MRI, as well as the FreeSurfer61 output directory. The diffusion imaging data was processed with the MRtrix3 toolbox62. Single fibre response functions were estimated using the algorithm from63. The response functions were averaged across subjects to enable cross subject comparison of the connectomes. Fibre orientation distribution functions (fODFs) were computed using multi-shell multi-tissue constrained spherical deconvolution64 and were normalized to correct for residual intensity inhomogeneities65. Probabilistic tractography with anatomical constraints66 was performed to generate 15 million streamlines per subject. Streamline weights were estimated using the SIFT2 algorithm67. Using the cortical surfaces the 7 Networks 400 Parcels Schaefer parcellation was projected into subject space25. Grouping weighted streamlines according to the parcellation resulted in a full brain connectome.

Next we used the TractSeg toolbox26 to estimate single major tract bundles. This toolbox uses a trained neural network to estimate volumetric masks of bundles from the fODF peaks. We used the estimated masks to filter the full tractogram. Masks for beginning and end of the bundle, i.e. where streamlines of the bundle intersect the grey matter, and a mask for the full path of the bundle were used as inclusion criteria. The rest of the brain, i.e. any voxel not inside the 3 inclusion masks, was used as an exclusion mask. Only streamlines complying with inclusion and exclusion masks were kept and together with the Schaefer parcellation a bundle specific connectome was generated. The following major tract bundles were used for this study: Inferior fronto-occipital fascicle (IFO), Inferior longitudinal fascicle (ILF), Middle longitudinal fascicle (MLF), Superior longitudinal fascicle I, II and III (SLF_I, SLF_II and SLF_III, respectively), Uncinate fascicle (UF) and Arcuate fascicle (AF).

Simulation and analysis of reservoir network simulations

The reservoir of neurons was simulated using echo state networks (ESN). From68 we describe ESNs the following way. Given a discrete input timeseries \(u\left(t\right)\in {{\mathbb{R}}}^{{N}_{u}}\), the ESN tries to predict a target timeseries \(y\left(t\right)\in {{\mathbb{R}}}^{{N}_{y}}\), minimizing some error function. \(t\) represents the discrete time steps, with \(t=1,\,...,T\) and \(T\) being length of the input timeseries. The input \(u(t)\) is fed onto, and a prediction \(\hat{y}\left(t\right)\in {{\mathbb{R}}}^{{N}_{y}}\) is read from, the reservoir neurons using the weights matrices \({W}^{{in}}\in {{\mathbb{R}}}^{{N}_{x}\times \left(1+{N}_{u}\right)}\) and \({W}^{{out}}\in {{\mathbb{R}}}^{{N}_{y}\times \left(1+{N}_{u}+{N}_{x}\right)}\), respectively. The readout matrix \({W}^{{out}}\) is optimized using linear regression. The temporal dynamics \(x\left(t\right)\in {{\mathbb{R}}}^{{N}_{x}}\) of the reservoir neurons are given by Eqs. (1) and (2):

$$\widetilde{{{{\rm{x}}}}}\left(t\right)=\tanh \left({W}^{{in}}\left[1;u\left(t\right)\right]+{Wx}\left(t-1\right)\right)$$
(1)
$$x\left(t\right)=\left(1-{{{\rm{\alpha }}}}\right)x\left(t-1\right)+{{{\rm{\alpha }}}}\widetilde{x}\left(t\right)$$
(2)

With \(\alpha =0.5\) being the leaking rate which controls the speed with which the neural state is updated. We omitted training of the reservoir and fitting \({W}^{{out}}\), because we were only interested in the time course of neural activity, but not the accuracy of any prediction.

Subject individual connectomes were log transformed and used as connectivity \(W\) of the ESN. The connectome was scaled to a spectral radius of 0.8, as it was previously shown that this leads to best performance in computational tasks17 and tunes the network dynamics close to criticality, a common characteristic of brain dynamics69. The full brain connectome was lesioned by removing connections that belong to a fibre bundle connectome, to investigate the effect on alignment times in the intact/scrambled narrative task (see next subsection).

The input layer connectivity \({W}^{{in}}\) is initialized with random samples of a uniform distribution in the range [–0.5, 0.5] and then constrained to only map onto one of the 7 networks of the Schaefer parcellation as will be indicated in the results section. To account for different initializations, we perform the intact/scrambled narrative task for 10 different random seeds.

For simulation of reservoir networks we used the implementation in the EasyESN python library70.

Intact/scrambled narrative input task

We simulate the intact/scrambled narrative input task by using two different input timeseries \({u}_{{intact}}\left(t\right)\) and \({u}_{{scrambled}}\left(t\right)\) for the reservoir, as done in Dominey et al.12. Both timeseries contain the same 4 segments [A,B,C,D]. However, the order is different, [A > B > C > D] and [A > C > B > D] for \({u}_{{intact}}\left(t\right)\) and \({u}_{{scrambled}}\left(t\right),\) respectively. Thus, the first and the last segment are equal, with swapped segments in between. These input timeseries represent the intact and scrambled auditory input from Chien and Honey8. In the empirical study the intact and the scrambled input are used to measure activity alignment times of cortical areas. The activity of cortical areas in subjects that heard a different sentence previously but now hear the same aligns over time. This alignment is faster in hierarchically lower areas compared to areas higher in hierarchy. For our in-silico experiment we define the alignment time as the time required for difference of each reservoir neurons activity to decrease to half of its absolute value at the start of segment D. More precisely, let \({t}_{D}\) be the timepoint of the start of segment D of the input timeseries \(u\left(t\right)\), then the activity of a brain region at this moment is given by \(x\left({t}_{D}\right)\). For the simulation with intact and scrambled input, we obtain \({x}_{{intact}}\left({t}_{D}\right)\) and \({x}_{{scrambled}}\left({t}_{D}\right)\), respectively. We define the alignment time \({t}_{a}\) as the time point when \({{|x}}_{{intact}}\left({t}_{D}\right)-{x}_{{scrambled}}\left({t}_{D}\right)|=2* \left|{x}_{{intact}}\left({t}_{a}\right)-{x}_{{scrambled}}\left({t}_{a}\right)\right|\), with |v| indicating the absolute value of v.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.