Introduction

Many problems of interest in a wide range of fields find their most natural representation as dynamics unfolding on complex networks. This ever-expanding list includes the spreading of viruses1,2, synchronization of neurons3 or people4, and the stability and robustness of the electricity grid5. A variety of methods have been introduced to characterize the multi-scale nature of such dynamics, including computational approaches like spectral coarse graining6 or renormalization7 and analytical equation-based methods via the derivation of mean-field models3 or other low-dimensional representations8.

Although not always leveraged as reduced order models, spectral representations of network dynamics4, which often arise from a linearization of the underlying problem, can lead to fruitful analysis. Observing the independent modes of a system near a dynamical fixed point can be used to relate dynamic scales (e.g., the synchronization time of oscillators) and topological ones (e.g., the eigenvalues of underlying networks)9 or to assess the stability of various low-dimensional fixed points of a high-dimensional system10.

But beyond solutions of linear or linearized problems, an adroit application of Fourier tools offers a way to precisely resolve the consequences of nonlinearity in the dynamics. In the network context, it has been increasingly recognized that a one-dimensional or lower-dimensional model of network dynamics suffers from generic inaccuracies owing to the nonlinearities of the governing equations. These inaccuracies can be quantified analytically and measured11, but the precise ways in which neglected scales influence the modeled dynamics is difficult to isolate. In fluid mechanics, where a turbulent flow state exists far from any fixed points or equilibria, Large-Eddy simulation (LES)12 and related filter-based techniques13,14,15,16,17,18,19,20,21 have long grappled with the consequences of such nonlinearities, and are now a cornerstone of the simulation of large-scale flows like those found in the atmosphere and ocean22.

In the filter-based framework, through a filtering of the equations of motion, direct scale-dependent effective forcings can be derived for the individual scale-local modes of a system. This allows one to view phenomena like the scale-to-scale transfer of energy as the consequence of physical work being done by some scales of a system on other scales20. This interpretation leads to the introduction of a fictitious Reynolds stress, an effective stress representing the transfer of momentum from large-scale mean flow to smaller-scale turbulent fluctuations.

In this paper, we investigate a graph filtering methodology that allows us to introduce corresponding notions in the context of network dynamics. Applying this technique to a network of nonlinear oscillators, we show how the synchronization of the network is modified via spectral energy transfer between network modes facilitated by an effective force akin to the Reynolds stress. Further, we are able to localize such a higher order interaction to individual nodes in the network and comment on their topological dependence.

Spectral view of network dynamics

The Laplacian matrix is the fundamental building block of spectral graph theory, and is given by \(\textbf{L}=\textbf{D}-\textbf{A}\) where \(\textbf{A}\) is the graph adjacency matrix and \(\textbf{D}\) is the node degree matrix defined as \(\textbf{D}=\textrm{diag}(\sum _jA_{ij})\), with eigendecomposition \(\textbf{L}=\textbf{V}\varvec{\Lambda }\textbf{V}^T\) assuming the network is real and symmetric. For simplicity, we will focus on undirected networks. Just as the eigenfunctions of the discrete Laplace operator form the basis functions of the Fourier transform, the eigenvectors of the graph Laplacian can be used as a complete set to represent any graph signal—the naming convention of the matrix is not a coincidence.

While there is no strictly corresponding sense of frequency that the eigenvectors of the graph Laplacian inherit from the discrete Laplacian, there is an approximate one. Consider the metric of graph signal smoothness known as total variation23

$$\begin{aligned} \textrm{TV}(f) = \frac{1}{2}\sum _i\sum _jA_{ij}(f_i-f_j)^2=f^TLf, \end{aligned}$$
(1)

a natural measure which will take small values when the signal varies smoothly across connected nodes. Treating the kth eigenvector of the graph Laplacian, \(v^{(k)}\), as a graph signal, we can see that \(\textrm{TV}(v^{(k)})=(v^{(k)})^TLv^{(k)}=\lambda ^{(k)}\), where \(\lambda ^{(k)}\) is its corresponding eigenvalue. Thus in ordering the graph eigenvectors by increasing eigenvalue as is typically done, they are also ordered by decreasing total variation. A projection of a signal f onto the eigenvectors of the graph Laplacian therefore captures its smoothness with respect to the underlying graph24.

Spectral oscillator dynamics

While this terminology was developed in the field of graph signal processing25,26,27, it can carry dynamical significance as well. To demonstrate this significance, we introduce a form of coupled, unforced Duffing-type oscillators governed by

$$\begin{aligned} \ddot{x}_i = -\alpha x_i - \beta x_i^3 + \sum _jL_{ij}\dot{x}_j. \end{aligned}$$
(2)

While there are many other systems that have dynamically significant spectral representations28,29, we choose this model for its expository simplicity (c.f.30). The energy of a single oscillator, given by the quantity \(E_i = \frac{1}{2}\dot{x}_i^2+\frac{1}{2}\alpha x_i^2 + \frac{1}{4}\beta x_i^4\), evolves according to

$$\begin{aligned} \dot{E}_i = -\dot{x}_i\sum _jL_{ij}\dot{x}_j, \end{aligned}$$
(3)

so the total system energy \(E=\sum _iE_i\) evolves as

$$\begin{aligned} \dot{E} = -\dot{x}^TL\dot{x}, \end{aligned}$$
(4)

exactly the total variation of the oscillator velocity signal. This means that that energy is dissipated inversely proportionally to the smoothness of the velocities, which motivates us to write out Eq. (2) in its spectral representation, capturing precisely this important notion of smoothness.

If we define the kth spectral coefficient as \(\hat{x}_k=\sum _jv^{(k)}_jx_j\) and inversely \(x_i=\sum _kv^{(k)}_j\hat{x}_k\), we can manipulate Eq. 2 to become

$$\begin{aligned} \ddot{\hat{x}}_k ={} & {} -\lambda ^{(k)} \dot{\hat{x}}_k - \alpha \hat{x}_k \nonumber \\{} & {} -\beta \sum _iv^{(k)}_i \sum _{l,m,n}v^{(l)}_iv^{(m)}_iv^{(n)}_i\hat{x}_l\hat{x}_m\hat{x}_n \end{aligned}$$
(5)
$$\begin{aligned} \ddot{\hat{x}}_k ={} & {} -\lambda ^{(k)} \dot{\hat{x}}_k - \alpha \hat{x}_k -\beta \sum _{l,m,n}\zeta _{klmn} \hat{x}_l\hat{x}_m\hat{x}_n, \end{aligned}$$
(6)

where we have rewritten the final term in Eq. (5) by defining \(\zeta _{klmn}=\sum _iv^{(k)}_i v^{(l)}_iv^{(m)}_iv^{(n)}_i\). Pulling out the entirely scale-local contribution from the final term, we arrive at

$$\begin{aligned} \ddot{\hat{x}}_k ={} & {} -\lambda ^{(k)} \dot{\hat{x}}_k - \alpha \hat{x}_k - \beta \zeta _{kkkk}\hat{x}_k^3 \\{} & {} -\beta \sum _{l,m,n}\zeta _{klmn} \hat{x}_l\hat{x}_m\hat{x}_n. \nonumber \end{aligned}$$
(7)

A consequence of the cubic nonlinearity, the hypergraph \(\zeta _{klmn}\) weights the higher-order interactions between the spectral coefficients, an interesting example of emergent higher-order interactions described recently by Thibeault et al.31. They showed that, generically, many forms of dimension reduction will lead to higher-order interactions between the observables if there exists a non-commutativity between the equations of motion and the reduction technique itself (often a nonlinearity, but also for some linear operators). In our case, such higher-order interactions are interpretable as the transfer of kinetic energy between scales, with the hypergraph \(\zeta _{klmn}\) determining purely structural constraints on such transfer. Thus, viewing the system as independently evolving scales28 will necessarily abstract away such energy transfer and may obfuscate the dynamics of the system.

Such transfer of energy is a well studied problem in fluid mechanics, with one of the most illustrative examples being the study of decaying turbulence32,33. In two-dimensional turbulence where energy transfer is typically towards larger-scale modes, limited damping at the largest scales leads to the phenomenon of spectral condensation34. Here too, as energy is dissipated more efficiently at scales with larger associated eigenvalues, the transfer of energy holds dynamic relevance. At the largest scale (\(k=1\)), there is no dissipation as \(\lambda ^{(k)}=0\) and energy trapped at this mode will not dissipate.

Consider then the following example: taking an Erdos-Renyi random graph with \(N=100\) (number of nodes) and \(p=0.1\) (probability of an edge) and keeping \(\alpha =1\), \(\beta =1\), we initialize the network with spectral weight of \(\hat{x}\) and \(\dot{\hat{x}}\) in the lower third, middle third, and top third of wavenumbers k and allow the network to relax towards a synchronized state. These three forcings therefore correspond to injecting energy from the large scales, medium scales, or small scales; we run all three since various dynamical systems often exhibit energy transfer from either the large scales to the small scales (on average) or vice versa.

Fig. 1
figure 1

Time evolution of the spectral velocity coefficients for initialization in the bottom third of wavenumbers (top), middle third (middle), and top third (bottom). Color indicates associated wavenumber of the spectral coefficient.

Full size image

We observe that although only the prescribed modes are imparted with kinetic energy, other modes of the system are energized due to the flow of energy between scales. In Fig. 1 we show individual spectral weights evolving in logarithmic time (the greatest energy transfer occurs in the first few moments of the simulation). Note that although only a restricted band of wavenumbers is initially energized, additional modes appear active as the original modes decay, indicating significant energy transfer occurring across the modes of the network. Focusing then on just the bottom third of wavenumbers for all three simulations, we visualize the spectral velocity weight in Fig. 2. Interestingly, the modes are not uniformly energized or energized in a monotonic fashion; rather, specific modes activate depending on the forcing range in a nonuniform manner. Such a result is expected given the complexity of the spectral hypergraph \(\zeta _{klmn}\) which determines the allowable interscale transfer, but it is not straightforward to further analyze such a high rank tensor. Although all of the relevant information is contained therein, further techniques will make the study of this phenomenon more tractable.

Fig. 2
figure 2

From top to bottom, the same three cases as in Fig. 1. Note the different color ranges for the three panels.

Full size image

Filtered view of network dynamics

Graph filtering is particularly suited to the study of such spectral phenomena as it has been defined largely through a spectral decomposition27. Filtering by a filter G can be written as

$$\begin{aligned} \overline{f}=Gf = Vg(\Lambda )V^Tf, \end{aligned}$$
(8)

or indexed at each node as

$$\begin{aligned} \overline{f_i} = \sum _k v_i^{(k)}g(\lambda ^{(k)})\hat{f}_k, \end{aligned}$$
(9)

where the filter g acts to attenuate or enhance spectral coefficients of a graph signal. We can define the unfiltered component of the signal to be \(\widetilde{f}=f-\overline{f}\) such that any signal can be written as the sum of its filtered component and an unfiltered residual.

An obvious candidate for filter choice is the smooth spectral filter given by \(g(\lambda ^{(k)},\kappa ,\gamma )=1-\frac{1}{1+\exp {(-\gamma (k-\kappa ))}}\), where larger modes with \(k<\kappa\) are preserved and smaller modes with \(k>\kappa\) are attenuated. The sharpness of the filter can be enhanced by increasing \(\gamma\), but for simplicity we take \(\gamma =1\). Although in equation (9) the filter is allowed to be a function of the eigenvalues of the network, this filter does not depend on the spectrum. A result is that the filtered signal can be easily interpreted as allowing all modes \(k<\kappa\). Figure 3 shows a graph signal before and after filtering with two different values of \(\kappa\) with their corresponding spectral representations.

Fig. 3
figure 3

(Top) Graph signal spectral representation before and after attenuation by a smooth spectral filter with \(\kappa =20\) (left) and \(\kappa =5\) (right). (Bottom) Corresponding graph signals in node space. Observe the enhanced smoothness in the filtered signal for lower filtering wavenumber.

Full size image

One advantage of a filtering methodology is that imperfect information is not always a hindrance. To compute the spectral coefficients \(\hat{f}_k=\sum _iv^{(k)}_if_i\), every \(f_i\) must be known. Although the definition of Eq. (9) employs the spectral coefficients and so requires complete information, it is a general definition and many filtering operations can be computed at a local scale.

Consider, for example, the heat equation \(\partial _tx_i =\epsilon \sum _j L_{ij}x_j\). As the heat equation evolves over a graph, the individual heats \(x_i\) tend to a uniform state, so its advancement can be interpreted as interpolating between an entirely disordered and perfectly uniform state of a network. Such an equation also arises in the linearized form of Kuramoto oscillators, and has been studied extensively in the context of synchronization28. To advance the heat equation at a single node, all that must be known is the heat value of the node itself and the heat value of its neighbors.

A discrete update of the heat equation is equivalent to the filtering operation \(\overline{f}=Gf\) if we set \(G=I+\epsilon L\), so the filtering function looks like \(g(\lambda ^{(k)})=1-\epsilon \lambda ^{(k)}\). Such an operation therefore attenuates high frequency spectral coefficients preferentially over low frequency spectral coefficients. So although the heat equation can be viewed in a spectral form where all information is needed, it can also be advanced at a single node with only self and neighboring information.

Taking the limit of the discrete heat filter while fixing the total time T, one finds \(g(\lambda ^{(k)})=\exp (-\kappa \lambda ^{(k)})\) where \(\kappa =\epsilon T\). Although strictly interpreting the cutoff scale \(\kappa\) is now more challenging, one can directly observe its effect on the node space of the graph, and it can be computed for individual nodes without full information. An example of the application of the heat filter is shown in Fig. 4.

Fig. 4
figure 4

(Top) Graph signal spectral representation before and after attenuation by a heat filter with \(\kappa =10\) (left) and \(\kappa =100\) (right). (Bottom) Corresponding graph signals in node space. Observe the enhanced smoothness in the filtered signal for lower filtering wavenumber.

Full size image

Filtered oscillator dynamics

Returning to our analysis of Eq. (2), instead of transforming it to a spectral representation, consider its form under the filtering operation. Filtering the entire equation and expanding as \(x_i=\overline{x_i}+\widetilde{x}_i\) when necessary, we can derive

$$\begin{aligned} \ddot{\overline{x}}_i = - \alpha \overline{x}_i - \beta \overline{x}_i^3 - \sum _j L_{ij} \dot{\overline{x}}_j - \beta F_i^{R}, \end{aligned}$$
(10)

where the emergent \(F_i^{R}\) we define as the Reynolds forcing, which can be written as

$$\begin{aligned} F^R_i{} & {} = \underbrace{\overline{\overline{x}_i\overline{x}_i\overline{x}_i}-\overline{x}_i\overline{x}_i\overline{x}_i}_{\text {Leonard force}} \\{} & {} + \underbrace{3\overline{\overline{x}_i\overline{x}_i\widetilde{x}_i}+3\overline{\overline{x}_i\widetilde{x}_i\widetilde{x}_i}}_{\text {Cross force}} \nonumber \\{} & {} + \underbrace{\overline{\widetilde{x}_i\widetilde{x}_i\widetilde{x}_i}}_{\text {Subfilter force}}. \nonumber \end{aligned}$$
(11)

See supplementary materials for more details. In this way, we can view the filtered dynamics of the network as the same as the dynamics of individual oscillators with three additional and distinct forcing terms, a combination of the self-interactions of the large-scale degrees of freedom, cross-interactions between the large and small scales, and the self-interactions of the small scales. In the turbulence literature, the analogous effects are known as Leonard, cross, and subgrid stresses, respectively13; we therefore adopt similar nomenclature. We can also examine the energy of each filtered level by multiplying Eq. (10) with the filtered oscillator velocity to obtain

$$\begin{aligned} \frac{1}{2}\partial _t\dot{\overline{x}}_i^2 = - \alpha \dot{\overline{x}}_i \overline{x}_i - \beta \dot{\overline{x}}_i \overline{x}_i^3 - \sum _j L_{ij} \dot{\overline{x}}_i\dot{\overline{x}}_j - \beta \dot{\overline{x}}_iF_i^{R}. \end{aligned}$$
(12)

Looking back to the relaxation dynamics of the three cases examined in the earlier section, we can filter the oscillator at increasingly coarse scales and sum the energy transfer given by the last term of Eq. (12). The results are shown in Fig. 5.

Fig. 5
figure 5

From top to bottom, summed energy transfer for the same three cases as in Fig. 1. Values are summed over the full duraction of the simulation (\(T=10\)). The forcing range is shaded and corresponds to the most active transfer of energy.

Full size image

As expected, the energy transfer is largest in the most energized wavenumbers, and net transfer is upwards towards larger scales. Interestingly, our triple decomposition reveals that although cross-scale energy transfer is almost exclusively active in the initialized regime, the self-interactions of the unfiltered scales peak at the bottom of this range and is active at larger scales as well. Additionally, it appears that the self-interactions of the filtered modes are unable to transfer energy across scales, a phenomenon also observed in the context of turbulent flows19.

Next, we perform a similar analysis on an ensemble of random graphs with initial conditions for each oscillator chosen uniformly at random in the interval \([-0.5,0.5]\). We generate ten random graphs and perform ten simulations for each graph. The average scale-dependent energy transfer summed over the entire simulation (\(T=10\)) is shown in Fig. 6. As before, we observe that the peak cross and subfilter energy transfer appear at different wavenumbers, but the net direction remains positive for almost all wavenumbers, excluding the bottom few.

Fig. 6
figure 6

Transfer of energy for random graphs. Uniformly random initial conditions correspond to a wide spectral forcing range and activates energy transfer at all scales.

Full size image

Filtering methodology localizes higher order interactions

Having observed that the emergent higher order interactions give rise to a richness of behavior, our next step is to investigate to what extent these interactions can be analyzed at a node-local level, one of the advantages of a filter-based methodology. For the previously shown random initializations, we compute the average absolute energy transfer at nodes of a particular node degree. The results are shown in Fig. 7.

Fig. 7
figure 7

Node-local energy transfer as a function of node degree for five different filter scales. Values are averaged across 10 realizations of a random graph with 10 runs each.

Full size image

We observe that at the largest scales (e.g., \(k=10\)), nodes with lower degrees participate the most in scale energy transfer while the reverse occurs at smaller scales. Intermediate-scale filtering (e.g., \(k=50\)) smoothly interpolates between these two extremes, but the overall energy transfer is the same order of magnitude for all displayed scales (shown by Fig. 6). Such an observation is not possible from a perspective looking at only reduced order variables, and can be studied in the context of other reduced order models if similar filters can be produced.

Finally, we investigate to what extent this degree-dependent energy transfer can be dependent on the larger neighborhood of nodes. For this, we construct a \(10 \times 10\) grid where we connect the top half of nodes with weights equal to 1, and the bottom half are connected with weights equal to 10. This example also tests the effects of weighted graphs on our filtering methodology, as the Laplacian spectrum of weighted graphs has been shown in some cases to be qualitatively different from its unweighted counterpart35. We run 1000 simulations initializing the positions and velocities as before, and show the summed absolute energy transfer at individual nodes after filtering at two different scales in Fig. 8.

Fig. 8
figure 8

Total absolute energy transfer summed over 1000 random initial conditions for system filtered at \(k=10\) (left) and \(k=60\) (right).

Full size image

As before, nodes with lower degrees have in general larger associated Reynolds forcing when filtered at a large scale and smaller associated Reynolds forcing when filtered at a smaller scale. But there is also a longer-range structural component: the nodes in the center of the higher degree region have the smallest associated energy transfer (Fig. 8 left), while for the second case nodes at the boundary of the two regimes are more active than those at the center despite having a smaller degree.

Conclusion

To summarize, we have introduced and discussed a method for filtering possibly nonlinear dynamics on a complex network. Just as in the more familiar case of dynamics in continuous spaces, nonlinear dynamics on networks can lead to interactions between different scales. Using the filtering operations we describe, we are able to compute an effective Reynolds force that encodes these scale interactions and demonstrate its relevance for the synchronization of the system—and in particular not just the asymptotic outcome of the system (that is, a synchronized network) but also the dynamical approach to that outcome. As systems far from a synchronized or steady state are ubiquitous in real world application, we anticipate that tools to quantify deviation from linearity (e.g. the ability to quantify local scale interactions) will find use in engineering applications. This is particularly relevant when the time-varying aspects of the approach to synchrony are relevant, for example when transients can trip nonlinear instabilities.

Along these lines, one productive application of our work is towards predicting cascading failures in power networks36. While these networks tend to exhibit homophily in the steady state (large scale), failure often occurs at a single edge (small scale). Our results suggest that such a network may fail endogenously due to the transfer of energy from large scale synchronized structures towards the small scales: a single link in the extreme form. Understanding how exactly the structure of the network contributes to its spectral energy properties may prove essential in mitigating such failures. Our approach bridges the gap between these kinds of localized failures and the reduced order modeling of their parent systems.

Finally, even though we invoked fluid turbulence here only as the origin of the filtering techniques we adapted for use on networks, our results may also have bearing on our understanding and simulation of turbulence itself. Although the importance of scale-to-scale energy transfer in turbulence has been recognized for a century37, the detailed mechanisms that drive the turbulent energy cascade remain poorly understood38,39, largely due to the vast complexity of the turbulent state. Indeed, on algebraic multigrids reminiscent of our final example, the geometry of the grid can influence the scale transfer of simulated turbulence40. Our ability to quantify spectral energy transfer—using machinery already developed and well known in the context of fluid mechanics—in other, simpler systems may thus lead to the discovery of tractable test cases for fully understanding such structure-based effects.