Cyclic quantum annealing: searching for deep low-energy states in 5000-qubit spin glass

Abstract

Quantum computers promise a qualitative speedup in solving a broad spectrum of practical optimization problems. The latter can be mapped onto the task of finding low-energy states of spin glasses, which is known to be exceedingly difficult. Using D-Wave’s 5000-qubit quantum processor, we demonstrate that a recently proposed iterative cyclic quantum annealing algorithm can find deep low-energy states in record time. We also find intricate structures in a low-energy landscape of spin glasses, such as a power-law distribution of connected clusters with a small surface energy. These observations offer guidance for further improvement of the optimization algorithms.

Introduction

The concept of a quantum computer, as originally envisioned by Feynman¹, is a system consisting of a large number of spins. With the advent of quantum devices encompassing more than 5000 qubits in spin-like system^{2,3,4,5,6,7,8}, quantum computing has gained even more substantial momentum, showing promise across various fields^{9,10,11,12,13}, including the exploration of dynamics of quantum phase transition⁸.

One of the most promising applications is the approximate solving of a broad spectrum of discrete optimization problems. Many such problems can be formulated in terms of finding low-energy states of spin systems^14,15,16,17. Quantum annealing uses quantum fluctuations to navigate the system between local minima towards its low-energy states. It has been designed, developed, and validated as an effective method for an approximate solution of these optimization tasks^{2,15,18,19,20,21,22,23,24,25,26,27,28,29,30,31}. Ideally, the ground state is attainable if the adiabatic condition is strictly met^22,23,25,32. However, the spin-glass phase^{10,21,33,34,35,36,37} presents significant challenges, characterized by a vast number of local minima and exponentially small energy gaps, which require exponential time to satisfy adiabaticity. Any deviations from adiabaticity lead to a cascade of Landau-Zener transitions^{21,38,39,40,41,42,43} to higher energy states, making deep low-energy states improbable.

This raises a question: how to minimize Landau-Zener transitions and go deeper into the energy landscape of the spin glass. Recently, an iterative cyclic quantum annealing algorithm⁴³ has been proposed. In this paper we demonstrate that the key elements of this algorithm—the reference Hamiltonian^43,44,45,46 and cycling through the many-body localization transition^47,48,49—are indeed useful steps towards improved quantum optimization routines. These elements, used together, allow for the manipulation of the spin-glass energy landscape, effectively reducing the number of Landau-Zener transitions and facilitating a deeper exploration of low-energy states.

Figure 1

(a) Schematic phase diagram. Presented here is a schematic representation of the phase diagram for the system under study, as defined in Eq. (1). The green line denotes the many-body localization transition, while the black dashed line indicates the first-order transition. The path of the cyclic annealing algorithm is represented by the orange triangle contrasting with the blue path, which illustrates the path taken by standard forward annealing. A trapezoid cycle for cyclic annealing is also shown here by the purple line. (b) Contour of hamming distance. Cycles with different $(b_z,b_x)$ are sampled to produce an approximate phase diagram. The average Hamming distance between the final and initial states of a single cycle is calculated and depicted as a contour plot. The contour lines in Hamming distance may indicate the boundary of the many-body localization transition. The disparate magnitudes of $b_x$ and $b_z$ arise from the normalization of the problem Hamiltonian.

Using D-Wave’s 5000-qubit quantum processor, the Advantage^™ system, to simulate large spin glasses⁸, we demonstrate that cyclic quantum annealing can indeed reach deep low-energy states in the spin-glass landscape and can surprisingly save 85% of the annealing time resources as compared to traditional forward annealing. Considering the limited availability of annealing time, cyclic quantum annealing demonstrates significant practical utility.

Analyzing the local energy minima found by the algorithm, we observed interesting structures in the low-energy landscape of the spin glasses. Spins flipped between relatively close local minima tend to cluster with a single large cluster contributing most to the energy difference. The distribution of cluster sizes follows a power-law distribution with an exponent close to − 2.1, which shows that rare events (i.e., atypical clusters) predominantly contribute to energy balance. These findings provide useful clues into further improvements to optimization search.

Cyclic quantum annealing can provide experimental data on the statistics of deep minima in large 3D spin glasses. This advancement has the potential to shed light on longstanding conjectures in the field, see, e.g.^50,51,52.

Cyclic quantum annealing

The traditional forward quantum annealing³¹ involves evolution along an open path that connects a Hamiltonian with ground state close to a product state with a Hamiltonian with unknown ground state. The idea is illustrated by the blue path in Fig. 1a. Cyclic annealing⁴³ takes a closed path in the parameter space, passing through the Hamiltonian of the problem, depicted as an orange path in Fig. 1a. Such a cyclic process allows the energy to progressively decrease, cycle by cycle, until a satisfactorily deep low-energy state is achieved.

The algorithm operates based on the following Hamiltonian:

$$\begin{aligned} H(s) = H_p+B_z(s)H_\text {ref}+B_x(s)H_q, \end{aligned}$$

(1)

where s parameterizes the cycle. It comprises three components, each serving a distinct function and corresponding to different steps of the algorithm:

Step 1—Prepare the problem Hamiltonian:

$$\begin{aligned} H_p=\sum _{ij}^N J_{ij}\, \sigma _i^z\sigma _j^z. \end{aligned}$$

(2)

The problem Hamiltonian $H_p$ is an Ising-like spin Hamiltonian with $\sigma _i^z$ representing the z-Pauli matrices, realized by qubits. Coupling parameters, $J_{ij}$, encode information about the optimization problem at hand. All couplers $J_{ij}$ available in the quantum processor are used in the simulations described below. The topology of $J_{ij}$ is essentially a 3D lattice, more details can be found in⁵³. The initial qubit states for the first cycle can be chosen as an arbitrary z polarized product state. We use the trivial all-up-spin state for this purpose. In the last step of each cycle, the initial qubit state for the next cycle is updated. This state is called the reference state: it is a z polarized product state with components denoted as $s_i^r$.

Step 2—Turn on the reference Hamiltonian:

$$\begin{aligned} H_\text {ref}=-\sum _i^N s_i^r\sigma _i^z. \end{aligned}$$

(3)

This term biases the problem’s Hamiltonian towards the reference state $\{s_i^r\}$. This step is realized by increasing $B_z(s)$ from 0 to $b_z$, passing through the first-order transition. This transition marks the $B_z$ field where the reference state becomes a true ground state of the Hamiltonian $H_p+B_z(s)H_\text {ref}$ .

Step 3—Turn on the quantum Hamiltonian:

$$\begin{aligned} H_q = -\sum _i^N\sigma _i^x. \end{aligned}$$

(4)

This term, which is non-commuting with respect to $H_p$ and $H_\text {ref}$, makes the states to be superpositions of bit-string states by providing matrix elements between them. For sufficiently high $B_x$, it drives the system through the many-body localization transition. This step is achieved by increasing $B_x(s)$ from 0 to $b_x$. Due to the bias induced in Step 2, the mixing is mainly with the states whose $H_p$ energy is less than that of the reference state.

Step 4—Complete the cycle and perform measurement: The final step is to turn off $H_\text {ref}$ and $H_q$, completing a full cycle by returning to the original problem Hamiltonian. Measurements taken in the end cause the state to collapse into a classical bit-string states. The turn-off process involves numerous Landau-Zener transitions, thus the final state is not the ground state. However, due to the bias $H_\text {ref}$, the final state has a considerable probability of being lower in energy than the initial reference state. Each cycle is repeatedly sampled multiple times in the quantum processor to obtain a distribution of states. The lowest-energy state of this distribution is then selected. If it is lower than the initial state of the current cycle, it becomes the new reference state for the next cycle; otherwise, the reference state does not change.

Simulations

Multiple cycles of the cyclic quantum annealing algorithm are executed, to progressively find deeper low-energy states, as illustrated by the decreasing energy curve (orange line) in Fig.2a. More details can be found in the “Methods” section.

Figure 2

Performance of cyclic annealing. (a) Decreasing energy curve. The orange dots represent the progressively lower energy state found by cyclic annealing for a selected problem from our comparison set. Green stars and blue dashed lines mark the answers from cyclic annealing and forward annealing respectively. Notably, by the third cycle, cyclic annealing outperforms forward annealing. (b) Position in the Gaussian distribution. The histogram depicts the distribution of states obtained through forward annealing, fitting a Gaussian distribution with $\mu =-15{,}685.47, \sigma =19.30$; see the orange line. The relative positions of cyclic (green dashed line) and forward (blue dashed line) annealing solutions in the distribution highlight the deeper reach of the former; see inset. (c) Results of comparison experiment. In an equal-time comparison of cyclic and forward annealing across 50 problems, the energy differences $\Delta E$ (normalized to the forward annealing’s standard deviation $\sigma$) are marked by crosses. Cyclic annealing’s superiority is evident, with lower-energy solutions in $86\%$ of cases (green crosses).

Evaluation of cyclic annealing performance involves direct comparison with forward annealing with the same amount of annealing time T, that is, cyclic annealing takes T/N for each of the N cycles. The latter typically produces results that fit a Gaussian distribution; see the histogram and orange line in Fig. 2b. For a specific problem, we run it 1000 times and take the lowest energy as solution. This is marked by a blue dashed line in Fig. 2a and b. In contrast, cyclic annealing is executed for 10 cycles and takes the answer of last cycle as solution (illustrated by the green star in Fig. 2a and the green dashed line in Fig. 2b). The cycle is the triangle with parameter $b_z=0.03, b_x=8.04$; see Fig. 1a.

The comparative study encompassed a set of problems chosen to represent typical spin-glass-type hard optimization problems in the form of Eq. 2, see the “Methods” section. The energy differences $\Delta E$ between the solutions obtained from cyclic annealing $E_\text {cyclic}^f$ and forward annealing $E_\text {forward}^f$, normalized by the standard deviation of the latter $\sigma$, are indicated by crosses. Notably, in $86\%$ of these comparisons, cyclic annealing achieved lower energy states (green crosses), affirming its enhanced performance, see Fig. 2c. Regarding the instances where the cyclic approach did not succeed, this can be attributed to fluctuations inherent in forward annealing, which occasionally explores deeper low-energy states due to the tail behavior of its distribution. The average improvement in energy achieved by cyclic annealing is quantified as:

$$\begin{aligned} \overline{\left( \frac{\Delta E}{\sigma }\right) } = -0.56\pm 0.49. \end{aligned}$$

(5)

Consequently, solutions derived from cyclic annealing are positioned, averaging over the problem set, at a significantly lower energy level relative to the mean energy level of forward annealing (see green vertical line and blue vertical line in inset of Fig. 2b for a typical problem), as given by:

$$\begin{aligned} \begin{aligned} \overline{\left( \frac{E_\text {cyclic}^f - \mu }{\sigma }\right) }&= - 3.42 \pm 0.57 \\ \overline{\left( \frac{E_\text {forward}^f - \mu }{\sigma }\right) }&= - 2.86 \pm 0.40 \end{aligned} \end{aligned}$$

(6)

with $\mu$ representing the mean and $\sigma$ the standard deviation of the forward annealing distribution. From Eq. (5) one can show that to achieve the same performance as cyclic annealing, forward annealing should be performed about 7000 times instead of 1000 times. In other words, cyclic annealing saves about 85% of the annealing time to achieve same performance as forward annealing.

Figure 3

Distribution of final states for a single cycle. A wide range of the path parameters $(b_z,b_x)$ of trapezoidal cycle were sampled, see main text. The problem Hamiltonian, initial qubit states, and other parameters were kept fixed. For each data point, a single cycle is executed 1000 times to produce a distribution of final states. They are shown as dots in the plane of energy and Hamming distance, which is measured from the reference state. Red dashed lines represent the energy of the reference state. (a) Overview. Sub-figures are arranged according to $(b_z,b_x)$. (b) Enlarged view. Distributions of varying $b_x$ but identical $b_z$ are shown in the same sub-figure. Six distinct $b_z$ values are selected.

Phase diagram

To better understand how cyclic annealing operates and to find the optimal parameters, we analyzed the distribution of final states reached by a single cycle. A trapezoidal cycle defined by the corners (0, 0), $(b_z',0)$, $(b_z',b_x)$, $(b_z, b_x)$ was used; see the purple cycle in Fig. 1a. To ensure the occurrence of the first-order transition in step 2, $b_z'$ is set to a specific value. The choice of a trapezoidal path is intended to not miss the first-order transition. For larger values of $b_z$, this trapezoid can be simplified to a triangle. A broad range of path parameters $(b_z,b_x)$ were sampled with each fixed problem Hamiltonian. A problem Hamiltonian is created by setting the $J_{ij}$ values for all available couplers to be uniformly distributed between $-1$ and 1. For each data point, a single cycle is executed. It is repeated 1000 times to generate a distribution of final states. The outcomes are shown as dots in the plane of energy and Hamming distance, which is measured from the reference state; see Fig. 3a,b.

These distributions show that cyclic annealing can explore a wide range of the Hamming distances for a 5000-qubit spin glass, from several hundreds to several thousands away from the reference state. For a fixed $b_x$, an increase in $b_z$ augments the magnitude of energy-spectrum distortion, thereby reducing the number of Landau-Zener transitions (as indicated by the shrinking distribution range). This facilitates a deeper exploration in the energy spectrum (evident from the lowering of the energy distribution). For a fixed $b_z$, a higher $b_x$ increases the magnitude of the many-body delocalization effects (visible from the increased Hamming distance distribution), albeit at the cost of elevating the energy of distribution. Taking these two competing effects into account, optimal parameter combinations can be chosen in practice; see enlarged view in Fig. 3b.

Furthermore, we calculate the average Hamming distance and present it as a contour plot in Fig. 1b. The lines of equal hamming distance potentially hint at the boundary of the many-body localization transition. It is also noteworthy that an advanced variant of the cyclic annealing algorithm can be designed to, for example, dynamically adjust the $(b_z, b_x)$ parameters for each cycle.

Discussion

Cyclic quantum annealing can explore the structure of low-energy states in spin-glass systems. It follows the same approach as above for data acquisition. Cyclic annealing of a single cycle is performed with a variety of annealing parameters $b_x$, while maintaining $b_z$ at a constant value of 0.03. The reference state is chosen as a fixed low-energy state. In the analysis, each final state is compared to the reference state to identify flipped spins. These flipped spins are classified into clusters based on their connectivity via non-zero $J_{ij}$ couplings. A flipped spin i is considered part of a cluster if it is connected to another flipped spin j in the cluster by a nonzero coefficient $J_{ij}$.

Figure 4

Clusters of flipped spins in a low-energy state. This figure illustrates the clusters formed by flipped spins when a low-energy state is compared to the reference state. Flipped spins are identified, and those connected by nonzero $J_{ij}$ values are grouped into the same cluster. The clusters thus identified are depicted here, with each cluster’s size and contribution to the total energy difference (relative to the reference state) detailed alongside its percentage. Notably, a single, large cluster—Cluster 27—emerges as the most significant contributor, accounting for 40.31% of the total energy difference. The color coding represents the degree of each flipped spin in the original graph, which is constructed with nodes representing all spins and links denoting nonzero $J_{ij}$ connections.

This analysis reveals a tendency for flipped spins to aggregate into clusters. Figure 4 illustrates the clusters for a typical low-energy state, showing the distribution and structure of all the flipped clusters. The size of a cluster is determined by the number of flipped spins it contains. The energy difference $\Delta E$ represents the contribution of a particular cluster C of a state $\{s_i\}$ to the overall energy difference relative to the reference state $\{s_i^r\}$,

$$\begin{aligned} \Delta E= \sum _{i\in \text {C}}\sum _{j\in N_i} J_{ij}(s_is_j-s_i^rs_j^r), \end{aligned}$$

(7)

where $N_i$ is the set of all nearest neighbor spins of $s_i$.

Analyzing multiple low-energy states, we consistently observe the presence of a dominant cluster that plays a substantial role in the overall energy difference. For instance, in the case illustrated in Fig. 4, the largest identified cluster, Cluster 27, contributes a significant $40.31\%$ to the total energy difference. This highlights the critical impact of the largest clusters within the spin system, underscoring their importance in the overall energy landscape of the system.

Figure 5a shows the frequency of finding a connected cluster of flipped spins of a given size for data from different $b_x$, which has same number of states and roughly same number of clusters. It is plotted against the size of the cluster, normalized to the total Hamming distance, d, between the two low-energy states. For example, the normalized distance 1/2 means that the given cluster contains half of the spins flipped between the two states. The probability of finding a cluster of size s follows a power-law distribution $P(s,d) \propto \left( \frac{s}{d}\right) ^{-\alpha }$. Since all data with different d collapse with same exponent, after summation over d, one get

$$\begin{aligned} P(s) \propto s^{-\alpha }, \end{aligned}$$

(8)

with the exponent $\alpha \approx 2.12$ from fitting. This pattern emerges consistently across clusters identified in final states obtained from cycles with varying annealing parameters $b_x$. The data collapse observed for clusters from different $b_x$ values underscores this characteristic as a universal feature of the system. The fact the exponent $\alpha <3$ shows that the variance is not defined. This implies a prevalence of large clusters within the patterns of flipped spins, as mentioned above.

Figure 5b shows cluster’s flip energy change, $\Delta E$, versus its size, s. It indicates a relationship between the magnitude of the energy reduction and the minimum size of the cluster required to achieve such a decrease. This shows that substantial energy decreases are possible upon the formation of larger clusters.

Figure 5

Distribution of clusters. Clusters are identified for final states from cycles with different annealing parameters $b_x$. The parameter $b_z$ is fixed as 0.03. (a) Power law distribution of cluster sizes. Data collapse for clusters of final states from cycles with different $b_x$. The red line is fitted with power-law equation: $\log (y) = -2.12 \log (x) -0.03$ with r-value − 0.95. The exponent 2.12 shows the variance is not finite, implying the possibility of rare super clusters. (b) Energy contribution $\Delta E$ vs. Cluster Sizes. For each cluster, we calculated its energy contribution $\Delta E$ to the energy difference with respect to the reference state. Clusters from cycles with different annealing parameters $b_x$ are shown here in different color. The legend is the same as that in (a). Data of negative $\Delta$E are shown here. A dashed line shows the relation between energy decrease and smallest possible cluster size to achieve that. To find a large energy decrease, one must find a large cluster.

In summary, transitions between local minima of the spin-glass landscape involve flipping a certain number of connected clusters. Cluster sizes follow an algebraic distribution with the exponent $\alpha \approx 2.12$. This makes large clusters rather prevalent. Moreover, there is a direct proportionality between the achieved energy gain and the cluster size. Targeting the larger clusters with negative energy may be the key to achieving significant energy reductions within the low energy part of the spin-glass landscape, as it may help to escape the local energy basin. These observations have practical implications for the development of optimization algorithms, which should be geared towards identifying and flipping large clusters.

It is of considerable interest to investigate the performance of cyclic quantum annealing in comparison with the classical algorithms. Furthermore, it would be pertinent to explore the potential integration of the idea of cyclic quantum annealing into classical algorithms to improve their performance⁵⁴.

Methods

This section outlines the details of the cyclic quantum annealing algorithm and simulations. The algorithm is implemented on D-Wave’s Advantage 4.1 quantum processor, which features 5,627 functional qubits and 40,279 couplers. We utilized all available qubits and couplers in our studies.

The problem set in the “Simulations” section consists of 50 random realizations of spin glass configurations: for Problem 0 to problem 9, $J_{ij}$ is randomly chosen as ±1; for problem 10 to problem 19, $J_{ij}$ is randomly chosen as ±1, ±2; for problem 20 to problem 29, $J_{ij}$ is randomly chosen as ±1, ±2, ±3, ±4; for problem 30 to problem 39, $J_{ij}$ is randomly chosen as ± 1, ± 2, ± 3, ± 4 ...± 8; for problem 40 to problem 49, $J_{ij}$ is randomly chosen as ±1, ±2, ±3, ±4 ...±16. Given that the underlying topology closely resembles a 3D lattice, these problems represent 3D spin glass systems, which are known to be NP-hard. We normalize the problem to have average $|J_{ij}|$=1 before the simulations.

For the cyclic annealing, the initial qubit state for the first cycle can be arbitrary z polarized product state. Interestingly, starting with a high-energy state does not hurt the algorithm’s efficiency, as it rapidly converges to a low-energy state after the first cycle. An important aspect of the algorithm involves specifying the duration of each segment of the path. Table 1 presents a typical path with corresponding time allocations, as used in the performance study described in the “Simulations” Section.

Table 1 Cyclic annealing path with time.

During the first step of the algorithm, all eigenstates are bitstring states, as the non-commuting driving Hamiltonian (Eq. 4) is not yet turned on. The evolution in this part can be fast. The second step, characterized by a finite energy gap above the ground state, can also be executed rapidly without consuming substantial annealing time. However, the third step, which goes into the spin-glass phase and encounters numerous Landau–Zener transitions, necessitates a slower pace. It is recommended to set the annealing time within the range of 100–1000 $\upmu$s. In our experiments, we used 300 $\upmu$s in the “Simulations” section and 500 $\upmu$s in the subsequent sections.

It is noteworthy that the cycle’s shape is not confined to a triangular or trapezoidal shape. An intriguing and open question remains on how different cycle shapes influence the annealing performance and the quest for an optimal one.