Introduction

The advancement of various significant technology fields relies on discovery of innovative materials with desired chemical or physical properties1, and obtaining correct structures of materials, arrangement of atoms in the unit cell, is the prerequisite. To achieve the goal, crystal structure prediction (CSP)2,3, in which the most stable crystal structure is inferred only from its chemical composition, has been widely adopted. The vast configuration space and the richness of local minima on potential energy surfaces (PESs) renders CSP a challenging task4. Optimization algorithms, such as genetic algorithms5,6,7,8,9, particle-swarm optimization (PSO)10,11, Bayesian optimization (BO)12,13,14, are proposed and successfully applied in practice. Typically, they create roughly-shaped initial structures and the final optimization is done by a geometric relaxation software either based on first-principles calculation or pretrained universal neural network potentials (NNPs). Nevertheless, these methods generally require a great number of iterations. In recent years, deep learning-based crystal generative models15,16,17,18,19,20,21 are developing fast, but they might find problems in extrapolation outside their training datasets. Therefore, as an example, due to the scarcity of corresponding data, CSP on 2D materials22,23 and nanoclusters24,25,26 generally relies on optimization methods. Besides, in both categories, most of the methods work well for crystals containing less than 60 atoms18 in a unit cell, but are still not ideal for larger crystals. For example, the training data of CDVAE15 and MatterGen21 does not include large crystals with more than 20 atoms in a unit cell. GNoME16 successfully explores larger ones approximating 100 atoms, but considerable computational cost is required.

Ising machines27,28 are hardware-assisted discrete optimizers that solve a quadratic unconstrained binary optimization (QUBO) problem,

$${{{{\bf{x}}}}}{\ast }={{{{\rm{argmin}}}}}_{{{{\bf{x}}}}\in {\{0,1\}}^{M}}\left({\sum }_{i=1}^{M}{h}_{i}{x}_{i}+{\sum }_{i=1}^{M}{\sum }_{j=i+1}^{M}{J}_{ij}{x}_{i}{x}_{j}\right),$$
(1)

where x is an M-dimensional bit vector and hi and Jij are real-valued parameters. CSP can be represented as a QUBO problem, either by simplifying the energy function29,30,31 or the use of a surrogate machine learning model32,33. Among the existing studies, Gusev et al.29, Ichikawa et al.30, and Xu et al.32 provided QUBO formulations of CSP under a fixed space group. Couzinié et al.31 and Couzinié et al.33 disregarded the space group and employed a grid-based representation of atomic coordinates in their QUBO formulation. Notably, they lack a feature of dynamically adjusting the space group, which is common in state-of-the-art CSP algorithms such as CALYPSO10,11, USPEX7,9,34, and CRYSPY13,14. Furthermore, these algorithms have not been tested on complex problems, mainly due to the scale restriction of D-Wave35 quantum annealer.

In this work, we develop a method named CRYSIM (CRYstal structure prediction with Symmetry-encoded Ising Machine). Our bit vector represents the lattice parameters, the symmetry information including the crystal system (CS), the space group (SG) and the Wyckoff positions combination (WPC), and coordinates of independent sites. This bit vector is translated to a crystal structure, and M3GNet36 provides its potential energy. Our goal is to find the optimal bit vector that gives the lowest potential energy. To enable the search with an Ising machine, a 2-order factorization machine (FM)37,38,39,40 is trained with available pairs of bit vectors and corresponding energies with an active learning workflow41. Since the prediction function is quadratic, the optimal bit vector that minimizes the FM-approximated potential energy can be found with an Ising machine. It does not always coincide with the real optimal solution, but one can expect that the error decreases as the amount of training data increases during the search process. Our information-rich bit vector inevitably inhibits the use of D-Wave quantum annealers. Instead, we employ a GPU-based Ising machine, Fixstars Amplify42, to solve a problem with over several thousand bits. It is based on simulated annealing and uses multi-level parallel processing on multiple GPUs to find optimal solutions. Fixstars Amplify relies on conventional semiconductor technologies, but can handle large-scale problems up to 130,000 bits with full connectivity. It has been employed in molecular generation43, materials design44,45,46, and various engineering fields47,48.

Our method outperforms BO12 and CALYPSO10,11 on three small crystals as well as large ones containing more than 150 atoms in unit cells. Specifically, CRYSIM successfully generates the ground truth Ca24Al16(SiO4)24 structure, containing 160 atoms in the unit cell, within 300 relaxations in 4 out of the total 5 trials. In this work, GPUs are adopted, but CRYSIM can leverage any Ising machines, including rapidly developing quantum devices.

Results

Bit vector encoding

The binary representation in CRYSIM consists of the following three parts: lattice parameters, symmetry information, and 3D coordinates of independent sites. In the first part, the six-dimensional lattice parameters are individually discretized and summarized into a bit vector with one-hot encoding. The second part includes a crystal system (CS), a space group (SG), a group of Wyckoff position combinations (WPCs). The sizes of each vector segment depend on the set of all possible space groups compatible with the given chemical composition, which is determined by whether there exists at least one WPC for achieving symmetry of the SG. Similarly, only compatible CSs are included in the embeddings. Accordingly, if m crystal systems are involved, each of which has s1, …, sm compatible space groups, the CS part has m bits to represent the CS and the SG part has \({\max }_{i=1,\ldots ,m}{s}_{i}\) bits to represent the SG. If the crystal structure has the i-th CS and j-th SG, the corresponding bits are set as 1, and the remaining are 0s.

Given an SG, we engineered the WPC generator in GN-OA package49 to compute the list of plausible WPCs according to the input chemical composition50,51. In the process, WPCs for each element species are calculated recursively based on its frequency in the unit cell. They are sorted in a descending order according to the maximum multiplicity of involved WPs, so that more plausible combinations are prioritized, considering the fact that most crystals in nature tend to occupy WPCs with general Wyckoff positions34,52. Then, the Cartesian product among the WPCs of elements is conducted to derive WPCs for the whole system, which are appended in the list until the maximum number, 30,000, is reached. Finally, they are divided into 300 groups of size 100, which is encoded in the WPC segment with a 300-dimensional one-hot vector for specifying the group.

The third part consists of k copies of a g3-dimensional bit vector in order to represent a crystal containing k element species, in which g denotes lattice discretization resolution (LDR). A 3D g × g × g grid is assumed within the unit cell. If an independent site of the atom species exists near a grid point, the corresponding bit is set to 1. In decoding, 100 structures are generated corresponding to all WPCs in the specified WPC group. Among them, the one with the largest minimum interatomic distance (MID) is selected to increase the possibility of deriving stable states4. Details of the encoding and decoding procedures are provided in the Method. Besides, Supplementary Note 1 and 2 present a detailed explanation about WPCs generation and application.

CRYSIM Workflow

The workflow of CRYSIM is depicted in Fig. 1. First, 1000 initial structures are obtained by random generation (RG) developed in this work (see Method for details) with the given chemical composition, and converted to bit vectors xl. Their potential energies yl are estimated using M3GNet without structure relaxation, leading to a training dataset described as the pairs of bit vectors and energies, i.e., D = {(xl, yl)∣l = 1, 2, …, 1000}. The dataset is then used to train an FM model 37,38, simulating the M3GNet-estimated potential energy surface of the materials family. The functional form of FM is described as

$$y=b+{\sum }_{i=1}^{M}{h}_{i}{x}_{i}+\mathop{\sum }_{i=1}^{M}{\sum }_{j=i+1}^{M}{\sum }_{k = 1}^{K}{w}_{ki}{w}_{kj}{x}_{i}{x}_{j},$$
(2)

where b, hi and wki are real-valued parameters, and xi is the i-th element of a vector x. It is similar to QUBO, but the weight matrix of quadratic terms is a low-rank matrix parameterized by wki and wkj. Then, Fixstars Amplify42 is used to optimize the bit vector to minimize the energy approximated by FM. PyTorch53 is leveraged to implement FM and the training workflow, with detailed training settings and hyperparameters of Amplify provided in Supplementary Note 3.

Fig. 1: The workflow of CRYSIM that contains T iterations, using Si4O8 as an illustration.
Fig. 1: The workflow of CRYSIM that contains T iterations, using Si4O8 as an illustration.
Full size image

Thin arrows denote the workflow at the t-th iteration, and thick arrows denote entering and exiting iterations. a Given the considered material system, a dataset is obtained by random generation (RG) to provide training samples and determine the upper bound of lattice parameters for binary representation. Potential energy of each material is also estimated by pretrained neural network potential (NNP) without structure relaxation. b Structures in the dataset {S1, S2, …, S1000} are encoded into binary vectors {x1, x2, …, x1000} using symmetry-informed integer encoding. c Factorization Machine (FM) is used to perform regression from the binary vectors to their corresponding estimated energies, obtaining the objective function to be optimized. d An Ising solver is employed to solve the learned objective function to minimize y in t-th iteration, resulting in x*,t. Amplify is used in this work. e The solved binary embeddings x*,t is decoded into crystal structures. Since one bit in the Wyckoff position combination (WPC) segment represents a group of 100 WPCs, 100 structures are derived. The one with the largest minimum interatomic distance (MID) is selected as S*,t. We note that the Si4O8 structures drawn in the figure (e) are indicative, which have different space groups (SGs). f The solved structure S*,t is relaxed by NNP, leading to a structure-energy pair \(({S}_{r}^{* ,t},{E}_{\min ,r}^{t})\). If iterations have not finished, frames in the relaxation trajectory are sampled. g Among the sampled structures, if one contains an MID smaller than 0.5 Å but still is estimated to have a negative energy, the energy is reassigned with a high positive one before adding the points into the training dataset for the next iteration. After finishing all iterations, the final structure \({S}_{r}^{* }\), the one with the lowest relaxed energy among all crystals in all T iterations, will be regarded as the discovered stable structure of this system.

Based on the solution, 100 structures with the same SG but different WPCs from the solved WPCs group are translated back to crystal structures, and the one with the largest MID is selected and relaxed using M3GNet. After relaxation, we sample 30 structure frames from the relaxation trajectory. Among the samples, if a structure has an MID lower than 0.5 Å but is still assigned a negative energy, the energy is adjusted to a high positive value to mitigate negative impact due to inaccuracy of NNP. Then the data points are added to D and FM is retrained. The above procedure is repeated T = 300 times and the most stable structure is recorded as the final result.

Recovering benchmark materials

We begin with relatively simple benchmark crystal tasks to demonstrate CRYSIM’s ability to address CSP. Wei et al.54 developed a series of quantity measurements for evaluating CSP algorithms, and selected five crystals, including ScBe5, Ca4S4, Ba3Na3Bi3, Li4Zr4O8 and Li3Ti3Se6O3, as examples to conduct tests. Ground states of all compounds are determined based on the Materials Project (MP) database55, i.e., mp-11277, mp-1672, mp-31235, mp-755253 and mp-1211008, respectively. Classical algorithms considered for comparison include CALYPSO10,11 and simple BO that directly optimizes lattice parameters, fractional coordinates, the SG number and the WPC index of crystals, implemented based on the hyperopt package56 following the scheme of GN-OA49, denoted as “Crystal param. + Hyperopt BO”. All methods are limited to perform 300 times of structure relaxation during one run to make a fair comparison. Accordingly, CALYPSO is leveraged for 30 generations, with the population size per iteration set as 10. Since the number of atoms are not optimized in CRYSIM, we treat the structure indicated by the input chemical formula as conventional cells, i.e., symmetry is calculated within the given lattice, instead of a combination of several cells. Accordingly, the NumberOfFormula parameter of CALYPSO, representing the smallest and largest times of the input chemical formula of considered structures in one optimization process, is set as “1 1” in all tests. Besides, in all experiments in this article, structures containing interatomic distances smaller than 1.0 Å are excluded from the statistics, to ensure that all crystals remain physically valid. Tests of each method repeat three times with different seeds. Hyperparameters of classical algorithms are reported in Supplementary Note 4.

We use StructureMatcher function from the pymatgen package57 to determine structural similarity between predicted and known ground truth materials. The function can compute minimum average pair-wise displacement between two corresponding atoms in two configurations among all permutations. The predicted structure successfully matches the ground truth as long as the displacement is computable, which suggests that StructureMatcher is able to distinguish corresponding atoms between them. Details of criterion of matching is provided in Method section.

When assessing results, we only include structures reaching the lowest energy (\({E}_{\min }\)) among all obtained ones in 300 iterations, i.e., \(\{E| E={E}_{\min }\}\), and compare them against the ground truth. Several major metrics are defined for this task: (1) IM,0 denotes the first iteration at which the ground truth is identified; (2) NE denotes the number of iterations reaching the lowest energy, i.e., \({N}_{E}=| \{E| E={E}_{\min }\}|\); (3) NM denotes the number of successfully matched ones among all considered structures. We provide a further illustration on evaluation of CSP algorithms in Discussion section. Figure 2 summarizes results of two representative crystals, and comprehensive information is presented in Supplementary Tables 1–5. Other metrics, such as displacement calculated by StructureMatcher of the structure in iteration IM,0, denoted as DM,0, as well as the minimum displacement \({D}_{M,\min }\) and corresponding iteration \({I}_{M,\min }\), are also reported. Predicted configurations with the lowest estimated relaxed energies in the three trials are shown in Fig. 3.

Fig. 2: Optimization performance comparison between CRYSIM and two classical crystal structure prediction (CSP) algorithms, CALYPSO and Bayesian optimization (BO), on two benchmark materials.
Fig. 2: Optimization performance comparison between CRYSIM and two classical crystal structure prediction (CSP) algorithms, CALYPSO and Bayesian optimization (BO), on two benchmark materials.
Full size image

The first iteration when the generated structure matches the ground truth (IM,0), and the number of successfully matched structures among the generated ones with the lowest energy (NM) are recorded for (a, b) Ca4S4 and (c, d) Ba3Na3Bi3 for the three optimization methods in 300 iterations with 3 seeds. Shadowed bars in c indicate that the corresponding methods fail to find the ground truth structure with these seeds.

Fig. 3: Comparison between the most stable benchmark configurations discovered by three crystal structure prediction (CSP) algorithms, CALYPSO, Bayesian optimization (BO) and CRYSIM, and the ground truth structure recorded in Materials Project (MP).
Fig. 3: Comparison between the most stable benchmark configurations discovered by three crystal structure prediction (CSP) algorithms, CALYPSO, Bayesian optimization (BO) and CRYSIM, and the ground truth structure recorded in Materials Project (MP).
Full size image

The side views (left column for each method) and top views (right column) of materials are visualized by VESTA software84, including mp-11277 (Sc: purple, Be: green), mp-1672 (Ca: blue, S: yellow), mp-31235 (Ba: green, Na: yellow, Bi: pink), mp-755253 (Li: green, Zr: blue, O: red), and mp-1211008 (Li: green, Ti: blue, Se: orange, O: red), respectively, with M3GNet-estimated relaxed energies labeled above. We note that the optimization algorithms are guided by the total energies of the systems, which are normalized afterwards in this figure by system sizes. Most configurations are expanded into superlattices to display the patterns. Crystals with the lowest energies are selected. If there are more than one crystals having the same energy, the one obtained in the earliest iteration is shown.

Ground states of ScBe5 and Ca4S4 can be readily discovered by all three methods, but CRYSIM generates significantly more stable states than the two classical methods. Besides, smaller IM,0 of CRYSIM optimizers indicate that FM can quickly and effectively characterize the PES by learning from initial datasets. For the more complicated Ba3Na3Bi3 system, CRYSIM is the only method that successfully discovers the stable state with correct estimated energies.

All methods fail on Li4Zr4O8 and Li3Ti3Se6O3 structure prediction if only the crystals of \({E}_{\min }\) are counted, which, however, reach an even lower estimated relaxed energy than the ground states. This may be attributed to two main reasons. First, the selected benchmark structures may not represent the ground states of corresponding chemical compositions, suggested by positive energies above hull. Especially, Li3Ti3Se6O3 (mp-1211008) exhibits a substantial energy above hull, 0.617 eV/atom according to MP, indicating a potential to transform into alternative phases predicted by the optimization methods. Second, the pretrained NNP leveraged in this study is not sufficiently accurate, and a 0.01 eV variation in potential energy can affect the behavior of optimization algorithms. As an instance, we further introduce predicting results on Li8Zr4O12 (mp-4156), a stable structure of Li-Zr-O family that has been observed in experiments, in Supplementary Fig. 1 and Supplementary Table 6. In all configurations with the lowest energies, hexa-atomic rings absent in mp-4156 can be found, which may suggest a systematic energy underestimation on near-equilibrium structures, as is discussed in other related works58. On the other hand, CRYSIM is a flexible tool, which can be incorporated with any energy estimators. For small systems, first-principles calculation software59,60 is available. We also expect that the continuous improvement of NNP technology can mitigate the problem.

Large crystal structures prediction

This section introduces experiment results on three large material systems, including Y6Co51, Ca24Al16(SiO4)24 and (SiO2)96, to demonstrate capability of CRYSIM. The crystal Y2Co17 has been a classical benchmark for assessing CSP algorithms12,61,62, but the two stable structures in this material family recorded in MP can only be achieved with unit cells Y4Co34 (mp-570718) and Y6Co51 (mp-1106140). Since the number of atoms in unit cells are not optimized in CRYSIM, we start directly with Y6Co51. Ca24Al16(SiO4)24 (mp-6008)29,63 and (SiO2)3263 have also been discussed in previous works as examples of CSP on large crystals. Here, (SiO2)96 (mp-1200292) is chosen since it is the largest SiO2 crystal in MP that has been observed in experiments.

Apart from CALYPSO and BO introduced earlier, simple RG, which is employed to generate initial training set for CRYSIM, and PyXtal52-based RG in CRYSPY13, denoted as “CRYSPY RG”, are additionally included as baseline CSP methods. 300 times of structure relaxation, i.e., 300 iterations, are conducted in one run, and tests of each method repeat five times with different seeds. Fig. 4a-c presents averaged accumulated lowest energies of crystals during generation in the 300 cycles. Optimal materials found by each algorithm are visualized in Fig. 4d. Corresponding data is summarized in Supplementary Table 7, as well as Supplementary Tables 8–10 for metrics defined in the last section.

Fig. 4: Optimization performance comparison between CRYSIM and classical crystal structure prediction (CSP) algorithms on large crystals.
Fig. 4: Optimization performance comparison between CRYSIM and classical crystal structure prediction (CSP) algorithms on large crystals.
Full size image

Normalized accumulated lowest M3GNet-estimated relaxed energies in the course of iterations of Y6Co51, Ca24Al16(SiO4)24 and (SiO2)96 structures derived from various CSP algorithms are shown in (a–c), respectively. Each curve is averaged on five tests with different random seeds, and colored shaded areas cover the maximum and minimum in the five trials. Dash lines are relaxed energies of ground truth materials in Materials Project (MP). Representative predicted configurations after relaxation, as well as corresponding ground states in MP, are displayed in (d), including side view (left column for each method) and top view (right column) of mp-1106140 (Y: grey, Co: blue), mp-6008 (Ca: grey, Al: light blue, Si: deep blue, O: red), and mp-1200292 (Si: blue, O: red), respectively, visualized by VESTA software84, with M3GNet-estimated relaxed energies labeled above. We note that the optimization algorithms are guided by the total energies of the systems, which are normalized afterwards in this figure by system sizes.

On Y6Co51, BO is the only method that discovers the ground state with a −7.127 eV/atom energy, the same as corresponding relaxed energy of mp-1106140. However, computational complexity of BO scales with the total number of atoms64,65,66, leading to a significantly reduced performance on Ca24Al16(SiO4)24 and (SiO2)96, even falling below CRYSPY RG. CALYPSO exhibits a higher stability than BO for large systems, and the implementation of pair-wise distance consideration in input renders it unaffected when screening out configurations with small MIDs, as is shown in Supplementary Table 11. However, this feature accelerates the optimization of energies only in the first tens of iterations in Fig. 4b-c, and then the algorithm is surpassed by CRYSIM methods.

On the other hand, length of CRYSIM embeddings is determined solely by the number of elements given the LDR, making it advantageous over other algorithms especially on large crystals. On the Ca24Al16(SiO4)24 system, which contains 160 atoms in the unit cell, CRYSIM successfully finds the ground truth structure in four out of five trials. For (SiO2)96, CRYSIM identifies a configuration with a relaxed energy (−7.890 eV/atom) close to the stable one (−7.891 eV/atom), significantly lower than the ones found by other methods. CSP for large crystals has been a long-standing challenging task. Energy distribution of the configuration space tends to concentrate on unstable states as the system size grows, which means that the difficulty of finding the ground state via RG exponentially increases63,67,68. Though CRYSIM does not outperform in all systems, the superiority on large crystals establishes it as a promising approach for CSP.

Effects of Processing Techniques

Lattice Discretization Resolution

When converting 3D structures into binary vectors, a higher LDR can reduce information loss, nevertheless, leading to exponentially increasing solving difficulty. We investigate the influence of LDR on CRYSIM optimization performance by testing g ∈ {5, 7, 9, 12, 15} across the three large crystals considered in this study, namely Y6Co51, Ca24Al16(SiO4)24 and (SiO2)96. Table 1 summarizes lowest energies with corresponding average accumulated energy curves recorded in Supplementary Fig. 2, where CRYSIM with different LDRs are denoted as CRYSIM-g, such as “CRYSIM-5” for g = 5. Each value is the mean of three trials with different random seeds.

Table 1 Influence of lattice discretization resolutions (LDRs) and minimum interatomic distance-related (MID-related) procedures on optimization performance
Full size table

Y6Co51 and Ca24Al16(SiO4)24 achieve the best CSP results at LDR 12, while (SiO2)96 performs the best at 15. This phenomenon stems from the implementation of CRYSIM, where each atom species is encoded using a g × g × g grid of bits, with 0 or 1 indicating the presence of an atom at each discretized unit. Accordingly, a higher LDR is advantageous when the number of atoms per element increases, rather than the total number of atoms. An LDR of 12 appears to strike a balance between representability and optimization difficulty for Y6Co51 and Ca24Al16(SiO4)24, where the maximum number of atoms of a specific element is 51 and 96, respectively. In contrast, (SiO2)96, which contains 192 oxygen atoms, may require a higher LDR to better capture interatomic spatial relationship in CRYSIM. According to the results, as a guidance on selecting LDRs for other tests, it is recommended to use a 15 LDR when the maximum frequency of an element species in the unit cell apparently exceeds 100, otherwise 12. Numbers of bits for representing each parameter for the three systems are further reported in Supplementary Table 12. We also provide time costs in the Ising solving process by Amplify and spatial resolutions of each block for different LDR in Supplementary Table 13 and Supplementary Table 14, respectively. We refer the readers to the Method section for details of calculation.

Factorization machine

In this work, FM is employed as the regressor in CRYSIM to build Ising objective functions, and here the fitting accuracy is investigated. Supplementary Fig. 3a–e shows distributions of predicted versus calculated energies of one of the initial Ca24Al16(SiO4)24 datasets derived by RG, the system requiring the largest number of bits to represent due to its chemical composition. Learnable parameters of FM are decided upon metrics on the validation set, comprised of 10% of the dataset (see Supplementary Note 3 for details). Changes of Pearson correlation coefficients (PCCs) and mean absolute errors (MAEs) during training are further provided in Supplementary Fig. 3f. The consistently high PCC values indicate effective optimization toward the global optimum, despite of fluctuations due to out-of-distribution energies. Similar trends are observed across all other systems and random seeds.

Additionally, a comparison between FM and full-rank quadratic regression (QR), in which quadratic terms in regression functions are independently learned instead of multiplications between linear terms, is exhibited in Supplementary Fig. 4. These experiments are conducted with CRYSIM-5 representation of Y6Co51 system, containing 801 bits in the embeddings. Under these conditions, QR involves more than 600,000 trainable parameters, whereas FM requires only 13,617 ones. For reference, optimization results of BO and CALYPSO, previously shown in Supplementary Table 7 are also included as baselines, with three trials performed for each method. On this system, CRYSIM-QR achieves a lower average accumulated energy than CRYSIM-FM, indicating superior optimization performance. However, for systems represented with more than 1,000 bits, QR requires millions of trainable parameters, making FM a more practical option for these tasks in terms of computational efficiency.

Processing with minimum interatomic distance

Inaccuracy of NNPs on configurations with extremely small MIDs renders negative impact on regression models. To mitigate the effect, procedures related to MIDs are designed and integrated in the workflow, including selecting the structure with the largest MID from one solution vector in Fig. 1f, and adjusting unphysical energy estimations in Fig. 1g. Effectiveness of the procedures is demonstrated in Table 1 by comparing CRYSIM of all considered LDRs with and without including these steps during optimization. Tests of each method repeat three times with different seeds. The corresponding accumulated energy curves are provided in Supplementary Fig. 2. Optimizers equipped with the modules achieve a widespread performance enhancement, particularly for larger systems with higher LDRs. Besides, inclusion of MID processing enables structures exploration with high LDRs. For Y6Co51 and Ca24Al16(SiO4)24, CRYSIM-12 performs the best with MID-related procedures, but it cannot realize the full potential and is outperformed by CRYSIM-9 without them. The advantage is attributed to improved efficiency in obtaining valid crystals, indicated by a reduction of filtered-out configurations reported in Supplementary Table 15.

Discussion

In this section, we provide a detailed interpretation covering: (i) symmetry representation, (ii) atomic position representation, (iii) MID-related procedures in the workflow, (iv) evaluation on end-to-end CSP algorithms, and (v) the use of Ising solvers.

First of all, incorporating symmetry representations is a fundamental requirement in developing algorithms for crystal generation. Most stable crystal structures in the nature have symmetry69,70, making it necessary to consider symmetry constraints in the CSP task. Classical optimization methods, including evolutionary algorithms34,71, PSO10,11, and BO12,13,14,49, rely on RG algorithms50,51,52 to sample symmetric configurations as starting points. Following discussions proposed in RandSpg51, two strategies of organizing and utilizing WPCs lists are summarized in Supplementary Note 1. CRYSIM differs from the above strategies since the search is conducted within a space of symmetric structures, in which the SG number is optimized directly. Other methods, including USPEX34 and CALYPSO10,11, only guarantee the symmetry requirement in initialization.

Recently, symmetry of crystals is also emphasized in generative models to increase the possibility of deriving reasonable configurations. As one of the earliest trials in this field, DiffCSP++17 does not represent symmetry of crystals explicitly, in which networks generate configurations given a specific pre-set WPC constraints. WyCryst20, SymmCD72 and WyckoffDiff73 successfully explore symmetry representations with one-hot labelling, operation matrices and graphs, respectively. However, current generative model-based solutions are still not perfect from the perspective of end-to-end CSP. The related works, including the ones trained with an auto-regression strategy, such as CrystalFormer74 and WyFormer75, and diffusion models, such as SymmCD and WyckoffDiff, rely on a prior distribution of SG from the training dataset for conditional generation. WyCryst is among the methods that predict both SG and WPC for constructing structures, but it requires PyXtal as an external tool to provide atomic positions and fill in WPCs, similar to WyFormer and WyckoffDiff. In comparison, CRYSIM treats the CSP problem in an exact end-to-end manner, in which SG, WPC index and atomic positions are all optimized and do not condition on any given information, except for the input chemical formula.

Second, an accurate description on the CSP problem relies on an effective representation of atomic coordinates in the lattices. There are two main strategies for representing fractional coordinates as variables to be optimized. The first is to treat each coordinate (xi, yi and zi for the i-th atom) as independent variables11,49, and the second is to split the whole crystal lattice and use the derived discrete blocks in the 3D space to encode positions29,30,31,32. Most optimization methods based on Ising models adopt the second approach, as it aligns with the goal of achieving guaranteed optimal solutions through quantum annealing by fitting the system’s PES. By representing atomic positions via lattice splitting, an Ising model can encode physical interactions: first-order terms capture the energy contribution of a single atom due to external fields, while second-order terms describe pairwise atomic interactions. This allows the Ising model to approximate the interatomic potential in a quadratic form.

Nevertheless, in practical implementations that account for symmetry, such as CRYSIM and other works29, the solved atomic positions are not external coordinates for building structures, but internal or independent sites to insert in WPs. As a result, the learned Ising model does not fully reflect an actual interatomic potential. One possible solution is to first estimate or sample an SG and WPC, derive corresponding constraints on lattice parameters and coordinates, and then optimize the two parts. Accordingly, by adding penalty terms, Ising solver can optimize directly on external coordinates and preserving the symmetry simultaneously. However, the order of constraints on coordinates would be the same as the multiplicity of corresponding WPs, making it challenging to implement for current solvers.

From a practical perspective, another advantage of the second strategy over the first one is that it requires less bits to encode coordinates in large systems, especially those with few atomic species. This is because the second strategy scales linearly with the number of atomic species and remains constant with respect to the number of atoms, whereas the first strategy scales linearly with the number of atoms within the cell. The scaling with the number of atoms is generally more computationally intensive in large systems. Taking the (SiO2)96 system in Results as an example, suppose the lattice is split into 15 * 15 * 15 blocks. Following the first strategy, each coordinates require 15 bits to represent, leading to 15 * 3 * (96 + 192) = 12960 bits in total, but only 15 * 15 * 15 * 2 = 6750 are needed based on the second one.

Third, the MID-related procedures employed in this study are necessary for preserving effectiveness of the algorithm and enhancing optimization efficiency practically. Ideally, a pretrained NNP should assign high energies to unstable structures, allowing the objective functions to reflect an accurate structure-energy relationship through active learning for diversion structural configurations. Accordingly, CSP optimizers, designed to identify low-energy solutions, can correctly discover the ground states. However, training sets for state-of-the-art NNPs generally lack out-of-distribution structures in the configuration space, especially for crystals containing extremely close atom pairs, rendering their estimated energies unreasonably low. As presented in Supplementary Table 15, many CRYSIM optimizers without MID processing are encouraged to generate abnormal structures, due to their low estimated relaxed energies, which are, however, ineffective from a practical perspective. Directly replacing pretrained NNP with first-principles calculation software59,60 can circumvent the problem, but it is still unrealistic for large crystals considering current computational power.

According to an observation that most abnormal relaxed structures originate from abnormal decoded unrelaxed ones, we design MID processing techniques on generated configurations (Fig. 1f, g) to improve the efficiency of CRYSIM optimizers. Besides, previous works4 also suggest that atoms in stable structures tend to uniformly distribute, instead of clustering in a small space. However, for some material systems, strategies aimed at controlling MIDs of generated materials (e.g., CRYSPY RG and CALYPSO) or attempting to obtain materials with larger MIDs (e.g., CRYSIM) may hinder discovering of ground states, as evidenced by the Y6Co51 system in Supplementary Table 7. Although many structures derived by RG and BO implemented in this work are screened out due to very small MIDs, as reported in Supplementary Table 11, CSP of Y6Co51 is finally accomplished after structure relaxation on crystals that may not have high MIDs. We expect that when a more accurate pretrained NNP is proposed, the MID-related procedures can be discarded.

Forth, this work adopts a simple but rigorous criterion to evaluate end-to-end CSP optimizers. The primary objective of our optimization-based CSP algorithm, CRYSIM, is precisely to locate the global minimum on the PES, representing the most thermodynamically stable structure according to the chosen energy model. To rigorously assess this specific capability during benchmarking, we consider the number of “successful match” (NM) only for structures with the lowest relaxed energies out of the 300 total structures instead of all of them. This prevents crediting success to fortuitous sampling into higher-energy local minima, thereby isolating the evaluation of optimization performance.

More critically, this methodology also reflects the practicability in CSP tasks. When the target structure is unknown, researchers inevitably rely on the calculated energy ranking, treating the lowest-energy prediction as the most likely candidate for experimental synthesis or validation. Higher-energy predictions, even if potentially correct, cannot be identified as such without prior knowledge of the ground truth. Thus, by focusing our evaluation on the lowest-energy structure, our metric is not only relevant to CRYSIM’s optimization objective but also aligned with the practical interpretation and utility of CSP results.

Finally, as a solver for addressing combinatorial optimization problems, quantum annealing (QA)76,77,78,79,80,81 has gained attention due to its theoretical ability to escape from local minima, and D-Wave system35 is among the most widely used implementations of QA29,31,33. However, the maximum number of variables for the D-Wave system is limited to 124 bits43, severely restricting its application. In this work, we integrate Amplify42, a GPU-based Ising solver, into CRYSIM as a substitute of quantum annealers. Nevertheless, we claim that the present implementation can be applied directly on quantum annealers without adjustment as quantum computers continue to develop.

Conclusion

In conclusion, CRYSIM, a CSP optimizer based on symmetry-encoded Ising models, is proposed and tested across various CSP tasks. To the best of our knowledge, it is the first Ising machine-based optimizer for CSP that dynamically optimizes symmetry. CRYSIM outperforms CRYSPY RG, BO, and CALYPSO on most systems, showcasing its strong optimization capabilities not only for small benchmark crystals but also for larger ones, including Ca24Al16(SiO4)24 and (SiO2)96. The predicting accuracy of FM in CRYSIM is also discussed, highlighting its expressivity in CSP tasks. CRYSIM offers a promising Ising machine-based optimization tool for CSP that could potentially be applied to quantum annealers in the future.

Methods

Random generation of crystal structures

Random generation (RG) constitutes the foundation of CSP11,13,68, in which atomic positions are randomly sampled according to the given chemical composition. In this work, a simple crystal RG tool is implemented as the CSP baseline, as well as preparing training data for Ising models.

In RG, six lattice parameters (lattice lengths a, b, c, and lattice angles α, β, γ) and fractional coordinates are sampled independently from uniform distributions. To determine the default lower and upper bounds of the distributions for lattice lengths, we perform statistical analysis on materials in MP. Let M be the set of all materials in the MP database. Define a function \(| m| :M\to {\mathbb{N}}\) that maps each material m to the number of atoms in its unit cell. We then partition M into five categories, {M1, M2, M3, M4, M5}, such that for each material m ∈ M, it belongs to category Mi if it satisfies

$$\left\{\begin{array}{l}{M}_{1}=\{m| | m| \le 20\},\hfill\\ {M}_{2}=\{m| 20 < | m| \le 50\},\\ {M}_{3}=\{m| 50 < | m| \le 80\},\\ {M}_{4}=\{m| 80 < | m| \le 100\},\\ {M}_{5}=\{m| | m| > 100\}.\hfill\end{array}\right.$$
(3)

Next, the averages of a, b, and c for materials in each category Mi are computed, denoted as \({a}_{{M}_{i}}\), \({b}_{{M}_{i}}\), and \({c}_{{M}_{i}}\), respectively. For a specific material system m0 to be generated, if the number of atoms ∣m0∣ falls within one of the ranges, the lower and upper bounds are determined as follows:

$$\left\{\begin{array}{l}{l}_{{m}_{0}}=0.8\times ({a}_{{M}_{i}}+{b}_{{M}_{i}}+{c}_{{M}_{i}}),\\ {u}_{{m}_{0}}=2\times ({a}_{{M}_{i}}+{b}_{{M}_{i}}+{c}_{{M}_{i}}).\end{array}\right.$$
(4)

These bounds are the same for the three lattice lengths. The lower and upper bounds for the lattice angles are set to 50∘ and 130∘, respectively.

Then, space group (SG) and corresponding Wyckoff positions combination (WPC) are derived for building symmetry. Given the input chemical composition, let \({{\mathbb{S}}}_{+}\) be the set of SGs that are compatible with the stoichiometry. For each \(S\in {{\mathbb{S}}}_{+}\), let \({{\mathbb{W}}}_{S}\) be the set of corresponding WPCs (see Supplementary Note 1). Let \(| {{\mathbb{W}}}_{S}|\) denote the number of distinct compatible WPCs for SG S. The process involves sampling an SG and then a WPC. An SG number is sampled from all compatible ones (\({{\mathbb{S}}}_{+}\)) uniformly, i.e., an SG Sl is selected by sampling its identifier \({i}_{{S}_{l}}\) uniformly from the set of identifiers for SGs in \({{\mathbb{S}}}_{+}\):

$${i}_{{S}_{l}} \sim {{{\mathscr{U}}}}(\{{{{\rm{id}}}}(S)| S\in {{\mathbb{S}}}_{+}\}),\quad {S}_{l}={{{\rm{SG}}}}({i}_{{S}_{l}}).$$
(5)

Based on Sl, a WPC is subsequently sampled. Define the maximum WPC count \({W}_{\max }={\max }_{S\in {{\mathbb{S}}}_{+}}| {{\mathbb{W}}}_{S}|\). Sample an integer \({i}_{{W}_{l}}\) uniformly from \(\{0,1,\ldots ,{W}_{\max }-1\}\):

$${i}_{{W}_{l}} \sim {{{\mathscr{U}}}}(\{0,1,\ldots ,{W}_{\max }-1\}).$$
(6)

The WPC Wl is derived from the WPCs set corresponding to the chosen SG Sl based on the sampled identifier:

$${W}_{l}={{{{\rm{WPC}}}}}_{{S}_{l}}\left(\left\lfloor {i}_{{W}_{l}}\cdot \frac{| {{\mathbb{W}}}_{{S}_{l}}| }{{W}_{\max }}\right\rfloor \right),$$
(7)

where ⌊x⌋ is the floor function, which returns the greatest integer less than or equal to x.

Finally, fractional coordinates are uniformly sampled from the interval [0, 1). These coordinates are treated as independent sites and placed into the Wyckoff positions Wl, where they are transformed into external coordinates to satisfy symmetry constraints. Similarly, the generated lattice parameters are assigned to variables defined by the crystal system (CS) Cl associated with the sampled SG Sl. Since WPCs impose dependencies among coordinates, reducing the degrees of freedom, only the earliest generated coordinates are used. The same approach applies to lattice parameters constrained by a CS.

When generating datasets for training, structures containing atom pairs with distances smaller than 1.5 Å are removed to ensure that most generated structures have a reasonable estimated energy, which is essential for training an accurate objective function. Distance filtering is not involved when evaluating the performance of the RG baseline. A much refined RG process is implemented by PyXtal52, which has been tested as the CRYSPY RG baseline in this work.

Details of symmetry-informed integer encoding in CRYSIM

Integer encoding can be interpreted as follows. Suppose a binary vector segment containing Nv bits is leveraged for representing parameter v. For a continuous parameter \(v\in [{v}_{\min },{v}_{\max })\), the iv-th bit of the vector segment will be assigned as 1 and other elements are 0s with

$$\begin{array}{r}{i}_{v}=\left\lfloor \frac{v-{v}_{\min }}{{u}_{v}}\right\rfloor ,\end{array}$$
(8)

where \({u}_{v}=({v}_{\max }-{v}_{\min })/{N}_{v}\) is the unit or interval of the representation. For a discrete parameter, iv = 1 if the parameter v of the system corresponds to the i-th category.

Lattice parameters encoding

In the workflow of CRYSIM, integer encoding is initially performed on training sets generated by RG. The upper and lower bound of lattice length encoding, simultaneously the highest and lowest lattice length of decoded materials, are calculated based on data points in the sets. Let \(l{l}_{0,\max }\) and \(l{l}_{0,\min }\) represent the maximum and minimum lattice lengths (for a, b, and c) among all structures in the initial training set. Then, the upper and lower bounds for lattice length encoding are defined as

$$\left\{\begin{array}{l}l{l}_{\max }=1.1* l{l}_{0,\max },\\ l{l}_{\min }=l{l}_{0,\min }.\hfill\end{array}\right.$$
(9)

The number of bits for representing lattice lengths Na = Nb = Nc = Nll is dependent on divisions of the lattice when encoding atomic coordinates, which is calculated by

$${N}_{ll}={C}_{ll}* g* \frac{l{l}_{\max }-l{l}_{\min }}{l{l}_{\max }},$$
(10)

in which g denotes the current LDR, and Cll = 10 by default. For lattice angles (α, β and γ), the lower and upper bound are 50∘ and 130∘, the same as RG. The unit for encoding angles is 2°, so that one lattice angle is encoded using

$${N}_{lg}=(130-50)/2=40$$
(11)

bits. The number of bits for encoding lattice parameters would be 3*Nll + 3*Nlg.

Fractional coordinates encoding

Atomic configurations are represented by discretizing the unit cell into a g × g × g voxel grid, where g is the LDR. A 3D binary matrix X ∈ {0, 1}g×g×g is constructed, where element Xl,m,n corresponds to the voxel region Rl,m,n defined by fractional coordinates y = (y1, y2, y3):

$${R}_{l,m,n}=\left\{{{{\bf{y}}}}| \frac{l}{g} < {y}_{1} < \frac{l+1}{g},\ \frac{m}{g} < {y}_{2} < \frac{m+1}{g},\ \frac{n}{g} < {y}_{3} < \frac{n+1}{g}\right\}$$
(12)

for l, m, n ∈ {0, 1, …, g − 1}. Xl,m,n is set to 1 if an atom’s fractional coordinates fall within Rl,m,n, and 0 otherwise. For crystals containing multiple element species, a separate flattened matrix is constructed for each of them. After concatenation, encoded information of each element is stored in separate regions of the final embedding.

We note that the optimized coordinates are internal ones, which will be placed into WPCs to satisfy symmetry constraints. Besides, similar to RG, the derived bits might be redundant. In implementation, 1-bits in the leftmost positions in the vector segment for each element are used, until all variables in the solved WPC are decided. Special positions are not involved in optimization, therefore not encoded in the discretized space. Once a specific WPC is selected/decoded, the special positions participate in constructing structures automatically. For instance, when filling the optimized position (0.2, 0.4, 0.8) into a WP:

$$\left\{\begin{array}{l}{A}_{1}:(1/2,1/2,1/2),\\ {A}_{2}:(x,1/2,1/2),\hfill\\ {A}_{3}:(-x,1/2,1/2),\\ {A}_{4}:(x,y,z),\hfill\\ {A}_{5}:(-x,-y,-z),\hfill\end{array}\right.$$
(13)

the result is:

$$\left\{\begin{array}{l}{A}_{1}:(1/2,1/2,1/2),\hfill\\ {A}_{2}:(0.2,1/2,1/2),\hfill\\ {A}_{3}:(-0.2,1/2,1/2),\hfill\\ {A}_{4}:(0.2,0.4,0.8),\hfill\\ {A}_{5}:(-0.2,-0.4,-0.8).\end{array}\right.$$
(14)

For experiments on benchmark crystals, the Y6Co51 and Ca24Al16(SiO4)24 system in this study, LDRs of CRYSIM are set to 12, with (SiO2)96 being 15.

Symmetry information representation

Symmetry information involves the CS, SG and WPC, which define the crystal’s symmetry. Numbers of bits for the three parts, i.e., NC, NS and NW, are dependent on the WPCs list calculated based on stoichiometry of the system. To be specific, only compatible SGs and CSs are encoded, and whether an SG and CS is compatible or not is determined by the existence of WPCs that can be used to build the corresponding symmetry, as is illustrated in Supplementary Note 1.

Details of calculating NC, NS and NW are presented as follows. Let \({{\mathbb{C}}}_{+}\) and \({{\mathbb{S}}}_{+}\) denote all compatible CSs and SGs, respectively, and \({{\mathbb{S}}}_{C}\) denotes the set of SGs associated with a CS C. We then define the set of compatible SGs for C as \({{\mathbb{S}}}_{C,+}={{\mathbb{S}}}_{C}\cap {{\mathbb{S}}}_{+}\). The numbers of bits for representing CSs and SGs can be decided as

$$\left\{\begin{array}{l}{N}_{C}=| {{\mathbb{C}}}_{+}| ,\\ {N}_{S}=\mathop{\max }_{C\in {{\mathbb{C}}}_{+}}| {{\mathbb{S}}}_{C,+}| .\end{array}\right.$$
(15)

NW can be independently set. In many cases, the number of distinct compatible WPCs for an SG S, denoted as \(| {{\mathbb{W}}}_{S}|\), is so large that encoding all WPCs within a binary segment becomes impractical. To address this, only WPCs with indices within the set \(\{\lfloor \frac{i\cdot | {{\mathbb{W}}}_{S}| }{{N}_{W}}\rfloor | i=0,1,\ldots ,{N}_{W}-1\}\) are included. In this work, NW is set to 300 by default. However, during decoding, each bit i corresponds to a group of 100 WPCs with indices \(\{\lfloor \frac{i\cdot | {{\mathbb{W}}}_{S}| }{{N}_{W}}\rfloor +j| j=0,1,\ldots ,99\}\).

When encoding symmetry information of a crystal having CS Cl, SG Sl and WPC Wl, first of all, the corresponding bit representing the CS is assigned as 1. The bit for Sl is derived by

$${i}_{{S}_{l}}=\lfloor {{{{\rm{id}}}}}_{{C}_{l},+}({S}_{l})* \frac{{N}_{S}}{| {{\mathbb{S}}}_{{C}_{l},+}| }\rfloor ,$$
(16)

in which \({{{{\rm{id}}}}}_{{C}_{l},+}({S}_{l})\) is the identifier of Sl in the ascending ordered sequence with respect to the set \({{\mathbb{S}}}_{{C}_{l},+}\). For instance, in the SG set of the cubic CS, i.e., \(\{P23,F23,...,Ia\overline{3}d\}\), the identifier of SG P23 is 0. Similarly, the bit for Wl is calculated by

$${i}_{{W}_{l}}=\lfloor {{{{\rm{id}}}}}_{{S}_{l}}({W}_{l})* \frac{{N}_{W}}{| {{\mathbb{W}}}_{{S}_{l}}| }\rfloor .$$
(17)

where \({{{{\rm{id}}}}}_{{S}_{l}}({W}_{l})\) represents the identifier of Wl in \({{\mathbb{W}}}_{{S}_{l}}\).

During decoding processes, the following relationships are adopted

$$\left\{\begin{array}{l}{{{{\rm{id}}}}}_{{C}_{l},+}({S}_{l})=\lfloor {i}_{{S}_{l}}* \frac{| {{\mathbb{S}}}_{C,+}| }{{N}_{S}}+0.5\rfloor ,\quad {S}_{l}={{{{\rm{SG}}}}}_{{C}_{l},+}({{{{\rm{id}}}}}_{{C}_{l},+}({S}_{l})),\\ {{{{\rm{id}}}}}_{{S}_{l}}({W}_{l})=\lfloor {i}_{{W}_{l}}* \frac{| {{\mathbb{W}}}_{S}| }{{N}_{W}}+0.5\rfloor ,\quad {W}_{l}={{{{\rm{WPC}}}}}_{{S}_{l}}({{{{\rm{id}}}}}_{{S}_{l}}({W}_{l})),\end{array}\right.$$
(18)

to make the encoding-decoding procedures stable and reversible.

We note that according to the integer encoding strategy implemented in CRYSIM, only one bit should be assigned as 1 in a vector segment for one parameter, so that decoding from the segment to real values can be performed directly. Amplify provides options to add constraints as penalty terms to the objective function to encourage generation of solutions fulfilling specific requirements. In summary, the number of bits for symmetry encoding would be NC + NS + NW.

An example on symmetry information representation

We further provide an illustrative example on encoding and decoding symmetry information. For the A4B4 system, there are 2 SGs available for the triclinic CS, 12 for monoclinic, 56 for orthorhombic, 66 for tetragonal, 12 for trigonal, 19 for hexagonal, and 17 for cubic. Since all CSs are compatible with the system, NC = 7. The maximum number of compatible SGs across all CSs is 66, and therefore NS = 66. NW can be independently set as 300 by default. Accordingly, the total number of bits for encoding symmetry information is 7 + 66 + 300. When encoding SGs, as an example, P41, the No.76 SG, is the second compatible SG belonging to tetragonal CS, thus the 4th bit for CS and the 2nd bit for SG are set to 1. For P23 (SG No.195) and F23 (SG No. 196), the first and second compatible SG in the cubic category, the corresponding bit for SG is calculated following

$${i}_{P23}=\left\lfloor 0* \frac{66}{17}\right\rfloor =0,$$
(19)

and

$${i}_{F23}=\left\lfloor 1* \frac{66}{17}\right\rfloor =3,$$
(20)

respectively. On the other hand, in the decoding process, the bit 2, 3 and 4 will correspond to

$$\left\{\begin{array}{l}\lfloor 2* \frac{17}{66}+0.5\rfloor =0\to P23,\\ \lfloor 3* \frac{17}{66}+0.5\rfloor =1\to F23,\\ \lfloor 4* \frac{17}{66}+0.5\rfloor =1\to F23.\end{array}\right.$$
(21)

In actual implementation, since it is impossible to determine WPC index from an already generated structure, we only curate training sets obtained from our RG algorithm, in which crystals are constructed based on an already chosen WPC.

Criterion for matching structures

StructureMatcher function in pymatgen package57 is used to compare configurations, in which parameters are set as stol=0.5, ltol=0.3, angle_tol=10.0, consistent with other related works82,83. This function will calculate the minimum normalized average root mean square pair-wise displacement between two input structures among all atom permutations. But if corresponding atoms in the two structures are not detected, which means that the function cannot identify any similarity between them, the calculation will not be proceeded. Accordingly, a structure is recognized to be accordant with the ground truth if having a computable displacement with it, and a model successfully finds the ground truth in one run if there is at least one such structure being generated.

Factorization machine for quadratic regression

Factorization machine (FM) is a type of regression model proposed as a substitute of Support Vector Machine to address its failure on sparse data37. FM creates a mapping between a vector \({{{\bf{x}}}}\in {{\mathbb{R}}}^{M}\) and real value y by equation (2), where b, hi, wk,i are coefficients for bias, linear and quadratic interactions, respectively. In principle, FM can be extended to model n-interaction terms, but we restrict it to quadratic terms since current combinatorial optimizers are efficient only for solving quadratic objective functions. In that case, FM can be reformulated as

$$y=b+\mathop{\sum }_{i=1}^{M}{h}_{i}{x}_{i}+\frac{1}{2}{\sum }_{k=1}^{K}\left({\left({\sum }_{i = 1}^{M}{w}_{ki}{x}_{i}\right)}^{2}-{\sum }_{i=1}^{M}{w}_{ki}^{2}{x}_{i}^{2}\right),$$
(22)

reducing computational complexity from O(KM) to O(2K)37. One of advantages of FM in this work is that it requires less fitting parameters, enabling a quadratic regression on binary vectors containing thousands of bits. Taking a vector of 2000 bits as an example, a full-rank quadratic regressor requires 2000 × 1999 terms for interactions, while FM only needs 2000 × K terms, in which K is usually smaller than 30. In this work, we implement FM with PyTorch53 based on equation (22) to the accelerate learning process.