Introduction

The computational cost of solving electronic Hamiltonians increases rapidly with the size of the electronic Hilbert space, i.e., with the number of orbitals and electrons. This presents a substantial constraint on electronic structure calculations for molecules and an essential problem for the study of correlated solids. To overcome this dilemma, it is desirable to construct models with fewer electronic degrees of freedom in a systematic and controllable way. For this purpose, one needs to select a smaller target Hilbert space and determine the structure of the target Hamiltonian and its matrix elements. In practice, typically, no more than a handful of orbitals per unit cell are kept in the target space. Altogether, this procedure is known as downfolding1 and serves as a bridge connecting ab initio methods such as density functional or GW theory with higher-level many-body treatments including (extended) dynamical mean field2 or dual theories3,4, ab initio DΓA5, self-energy embedding6 or Eliashberg theory7, which cannot be performed on the full Hilbert space. There have been various suggestions for these downfolding schemes, ranging from constrained density-functional perturbation theory for phononic degrees of freedom8,9,10 to the constrained random-phase approximation (cRPA)11, constrained density-functional theory12, constrained GW13 and constrained functional renormalization group14 approaches for purely electronic Hamiltonians, next to embedding schemes, e.g., starting from coupled-cluster references15,16 or via projection schemes17. Our focus here is on interacting target space Hamiltonians of the form \(H={\hat{t}}_{\alpha \beta }{c}_{\alpha }^{{\dagger} }{c}_{\beta }+{\hat{U}}_{\alpha \beta \gamma \delta }{c}_{\alpha }^{{\dagger} }{c}_{\beta }^{{\dagger} }{c}_{\gamma }{c}_{\delta }\), where \(\hat{U}\) is a static Coulomb interaction in the target space, \(\hat{t}\) defines the single-particle energies and the Greek indices run over the electronic target space. Generalized Hubbard model Hamiltonians of this form are the standard in “DFT++”18 (or density-functional theory based embedding) approaches to describe correlation effects in solids for which various solvers have been developed and implemented.

Although the idea of downfolding is conceptually simple, and reductions of the Hilbert space are ubiquitous in quantum physics, the question has remained of how one should do this in a systematic and practical way. By definition, the original Hilbert space is too large to do exact calculations, so one needs to use approximate methods whose errors are usually not controllable. At the same time, one needs to avoid double counting of correlation effects. This arises because of the interplay between the two-body term \(\hat{U}\) and the single-particle term \(\hat{t}\) in the Hamiltonian. The latter is frequently extracted from a density-functional theory (DFT) calculation that already contains some interaction and correlation effects. Several single-particle double-counting corrections to \(\hat{t}\) exist in the literature19,20,21,22, with some physical arguments supporting their use, but there is no general consensus on how to tackle this problem. In fact, the uncertainty surrounding these single-particle double-counting corrections can be a limiting factor for the accuracy of DFT++ approaches, see, e.g., refs. 23,24 for further discussions.

In terms of the interaction in the target space, downfolding a priori generates many-fermion interactions of arbitrary order14 and not just one- and two-particle terms. Furthermore, these interactions might be non-instantaneous (frequency-dependent)11, requiring an action rather than a Hamiltonian formalism. Even if one accepts the restriction to a static two-particle Coulomb interaction tensor, its strength should be different from the corresponding Coulomb interaction in the full space to account for the screening by electrons outside of the target space, while screening by the other electrons inside the target space should not be included. The latter is to avoid double counting of screening processes to the Coulomb interaction \(\hat{U}\) within the target space. A variety of constrained methods8,9,10,11,12,13,25,26 has been developed to avoid this two-particle double-counting issue, e.g., via partial (constrained) screening approaches.

Evaluating the performance of downfolding techniques has proven challenging in practice, as the matrix elements of the downfolded model are not measurable quantities themselves and as the models are often solved using approximate solvers. This multi-step procedure makes it hard to establish how accurate the downfolding by itself is since deviations from full first-principles calculations or experimental data might also originate in approximations made while solving the low-energy model.

Given the widespread use of cRPA calculations based on DFT input to derive interacting model Hamiltonians and their promise of accurately describing correlation effects at low numerical costs, it is critical to quantify their accuracy. To this end, we analyze here the sensitivity of this method at the hand of the vanadocene molecule. The latter is a good proxy for correlated bulk materials or embedded correlated defects with well-defined interacting sub-spaces within gapped semiconducting backgrounds (referred to as rest spaces). Because vanadocene is a relatively small system, we are able to use state-of-the-art first-principles methods to derive reliable reference data, and the downfolded models are solved exactly, which allows us to focus on errors in the downfolding procedure. To avoid obtaining correct spectra with incorrect states, we assess both energies and the character of the states. We systematically assess how the model Hamiltonian form, the target space basis, the impact of double-counting corrections, and the (constrained) screening models affect the ground state and excited state energies and wave functions.

In the following, we first explain our benchmarking strategy and introduce the vanadocene test system. Afterward, we discuss the first-principles reference data before we present our step-wise benchmark, showing how each downfolding step affects the comparison to the reference system. The impact of our findings is discussed at the end.

Results

Benchmarking strategy

To obtain accurate reference data, we use a combination of several state-of-the-art first-principles methods which each provides systematically improvable many-body wave functions to treat the electronic correlations at an attainable cost. Specifically, we apply equation of motion coupled cluster (EOM-CC)27,28, auxiliary-field quantum Monte Carlo (AFQMC)29, and fixed-node diffusion Monte Carlo (DMC)30, which have been shown to accurately describe the excited states for transition metal molecules31,32,33,34.

As the test system, we have chosen the unperturbed eclipsed vanadocene molecule VCp235, which consists of a V atom situated between two planar and parallel C5H5 rings as illustrated in Fig. 1. The unperturbed molecule has D5h symmetry with a five-fold rotation axis, five two-fold rotational axes orthogonal to the five-fold rotational axis, and a mirror plane in the xy-plane. The vanadocene molecule may also exist in a staggered geometry with D5d symmetry. Previous DFT calculations predict a difference of only 13 meV in the ground state energy between the D5h and D5d structures with the D5h structure as the global ground state, consistent with experiment36,37. On the level of DFT, there are 5 V 3d dominated Kohn-Sham states forming the highest occupied and the lowest unoccupied molecular orbitals, which host 3 electrons in total. The D5h symmetry of the crystal field generated by the carbon rings leads to two doubly degenerate (e1, e2) and one non-degenerate (a1) single-particle energies for these d-dominated molecular orbitals as depicted in Fig. 1. This way, VCp2 forms an optimal test bed for cRPA-based ab initio downfolding approaches: it hosts a well-defined partially filled correlated target space spanned by the V d orbitals, which is well disentangled from the carbon ring “background” electronic structure, which has a significant single-particle gap of more than 6 eV. The latter is important to guarantee that the RPA treatment of the background screening is adequate38.

Fig. 1: Single-particle properties. Left: DFT Kohn-Sham energies.
Fig. 1: Single-particle properties. Left: DFT Kohn-Sham energies.
Full size image

Purple, green, and red colors indicate the projected orbital weight of each Kohn-Sham state, being mostly H s, C p, or V d, respectively. Red lines mark single-particle energies resulting from diagonalization of the downfolded single-particle Hamiltonian, which perfectly overlaps with the Kohn-Sham ones. Right: Maximally localized Wannier functions used within the downfolding procedure.

Our goal is to benchmark the effects of electronic correlations in this molecule. For this purpose, we use the same frozen positions of the nuclei in all methods and do not consider any electron-phonon coupling. Furthermore, all methods start from the same ccECP pseudopotentials39 and ignore spin-orbit coupling. In this sense, our benchmark should be understood as a comparison of several computational methods for the interacting electron Hamiltonian defined by this pseudopotential and these atomic positions rather than as a benchmark to experimental data.

For the downfolding, we start from conventional DFT calculations and project onto a set of localized orbitals. These are subsequently used to perform constrained random-phase approximation1,11,40 (cRPA) calculations, which are widely used for deriving Coulomb interaction matrix elements for downfolded models41,42,43. The resulting downfolded models are evaluated using exact diagonalization to avoid any artifacts from approximate model solutions.

Using the reference data from accurate quantum chemistry calculations, we assess the effects of changing the downfolding procedure as a sort of sensitivity analysis, in which an independent variable (the choice in downfolding procedure) is varied to determine the sensitivity of the output (the eigenstates of the embedded problem). For each step in the DFT+cRPA procedure, we consider several reasonable and common choices and analyze the sensitivity of the results.

First-principles reference

Before we delve into the benchmark of the downfolded results, we first establish the first-principles references for the vanadocene molecule. In this study, we focus on the lowest four spin-flip and the lowest two crystal-field excitations, using three first-principles methods: EOM-CCSD, AFQMC, and DMC. They are among the most accurate quantum chemical methods for the investigation of excited states in correlated systems. Calculations have been systematically converged, and details of the methods can be found in the section “Methods”. Calculations have been performed without imposing specific point group symmetry properties on many-body wave functions.

We found with all reference methods that the ground state is 4A2 with S = 3/2, consistent with previous experimental and theoretical studies36,37,44,45,46,47. Specifically, the DMC ansatzes were chosen to be of Sz = 1/2 on the S = 3/2 quartet. The configuration of the DMC ground state is determined to be \({({e}_{2})}^{2}{({a}_{1})}^{1}{({e}_{1})}^{0}\), characterized by the one-body reduced density matrix of a natural orbital basis (see section “Methods” for details). The symmetry of the many-body state computed using AFQMC is inherited from the trial wave function used. For the ground state, AFQMC calculations were performed using a CASSCF trial wave function with \({({e}_{2})}^{2}{({a}_{1})}^{1}{({e}_{1})}^{0}\,\)3d-orbital occupancy.

Figure 2 shows a summary of the charge-neutral excitation spectra obtained using these first-principles methods. The ground state energies are aligned at zero, and the dashed (solid) lines represent spin-flip (crystal-field) excitations, identified by states’ zero (finite) occupation on the high-energy e1 manifold. All three first-principles methods are in qualitative agreement regarding the order and degeneracy of the spin-flip and crystal-field excitation energies. Quantitatively, all methods agree to within about 200 meV.

Fig. 2: First-principles excitation energies obtained using EOM-CCSD, AFQMC, and DMC.
Fig. 2: First-principles excitation energies obtained using EOM-CCSD, AFQMC, and DMC.
Full size image

The ground state energies obtained from all the methods have been shifted to align at zero. Dashed and solid lines represent spin-flip (SF) and crystal-field (CF) excitations, respectively, and the shaded regions around the energy levels show statistical uncertainties.

The energy difference between excitation energies of the same type (i.e., within either the set of spin-flip excitations or the set of crystal-field excitations) is remarkably similar across all methods. According to Fig. 2, the lowest three spin-flip excitation energies are found to be within windows of < 250 meV, while the fourth spin-flip excitation energy is higher than the lowest three spin-flip states by 350 meV (DMC). The crystal-field excitations are identified to be higher than the lowest four spin-flip excitations. The two crystal-field excitations are found to be separated by about 200 meV for each first-principles method. Overall, the first-principles methods obtain the same ordering and character of the many-body eigenstates of interest but differ in energy by approximately 60–200 meV.

In the following sections, we proceed with the discussion of the six lowest charge-neutral many-body excitations with Sz = 1/2, which take place within the V d shell and are thus describable within the minimal subspace of the downfolding calculations.

Downfolding results

Now that we are familiar with the test system and the first-principles reference data, we turn to the results obtained using our downfolding procedure. We aim to systematically test and scrutinize all relevant steps in the DFT+cRPA procedure, especially those for which multiple strategies exist in the literature such that choices need to be made. Specifically, we will discuss the chosen form of the model Hamiltonian, the target space basis, the single-particle double-counting procedure, and finally, the Coulomb screening model. The downfolding procedure is qualitatively assessed based on the predicted ground state properties, the ordering of the excitation energies, the many-body characteristics of the excited states, and the shape of their charge densities. To quantify the agreement of the (charge-neutral) excitation energies of the downfolded model with the first-principles reference data, we further calculate

$${\chi }^{2}(\{{E}_{n}\})=\frac{1}{2{\sigma }^{2}}\sum\limits_{i,n}{\left({E}_{n}^{{{{\rm{ref}}}},i}-{E}_{n}\right)}^{2},$$
(1)

where En are the charge-neutral excitation energies in the downfolded model and \({E}_{n}^{{{{\rm{ref}}}},i}\) are the corresponding excitation energies according to the ith first-principles method with i ∈ {EOM-CCSD,AFQMC,DMC}. χ2 is the log-likelihood from a Bayesian inference of the embedding model probability, based on the reference values. It naturally accounts for the fact that some of the states are in close agreement among the reference sets, and some are not. We determine σ as the average over the variances of the n excitations within the reference set (EOM-CCSD, AFQMC, DMC), which results in σ ≈ 0.1 eV. χ2 accounts for the fact that the reference data itself has errors, and so it represents our uncertainty about the exact result.

The purpose of this quantifier is to summarize how consistent a given downfolding procedure is with the excitation spectrum of the reference data in a single number, given that there is uncertainty within the reference data set itself. The smaller χ2, the better the match.

Note that this procedure requires us to identify which excitations belong together (the label n, the colors of the energy levels in the figures). We achieve this based on the quantum numbers, the spatial structure of the spin-resolved charge densities \({\rho }_{n}^{\sigma }\), and the many-body characteristics of the corresponding excited wave functions for each state n as explained in the section “Wannier basis sets”.

We consider several variations of the DFT+cRPA procedure. The choice of localized orbitals for the embedded space was either constructed using maximally localized Wannier functions48 (MLWF) or first-guess Wannier functions (FGWF). The single-particle energies were constructed either using double-counting corrections, no double-counting corrections, and by explicitly modifying crystal-field energies, which emulates using different DFT functionals to create the one-particle energies. We then move to considering the construction of the interacting part of the downfolded Hamiltonian by considering mono- and miltipole screening channels and the frequency dependence.

Reference DFT+cRPA procedure

To limit the number of calculations, we start with a set of common choices that yield reasonable agreement with the reference data. We use MLWF localized orbitals, single-particle energies with no double-counting corrections, and the full four index cRPA-screened Coulomb matrix elements in the ω → 0 limit. As shown for the ground state in Fig. 3 and for all excited states in section “Wannier basis sets” this model is in qualitative agreement with the reference data, with a loss function χ2 of 27.3. We will consider modifications to this strategy, and examine the changes in χ2; if χ2 varies upon a change in the choice, then the DFT+cRPA results are sensitive to the choice, and if it does not, then the DFT+cRPA procedure is not sensitive to the choice. In this way, we can identify what approximations deserve further analysis.

Fig. 3: DMC and downfolded ground states charge densities.
Fig. 3: DMC and downfolded ground states charge densities.
Full size image

ρ↑/↓(r) from each method is normalized by the corresponding ρ↓(r) from the same method.

Target space basis set

We proceed with the discussion of the chosen basis sets. For comparison to the conventional MLWF, which is constrained here to reproduce the DFT single-particle energies, we also use a projector-like49,50,51,52 first-guess Wannier function (FGWF) basis set to calculate all model Hamiltonian matrix elements. The FGWF Wannier function basis set differs in its construction by not applying any inner (or “frozen”) energy window, such that the DFT single-particle energies are not reproduced, and by using a much larger outer energy window, such that the resulting wave functions are more localized. All details can be found in the section “Methods”. In this case, both basis sets are capable of reproducing the correct ground state, so we proceed with the analysis of the charge-neutral excitation spectrum. Figure 4 shows all spin-flip and crystal-field excitations between 0.6 and 2.6 eV. For the FGWF basis, various double and four-fold degenerate spin-flip excitations appear in the energy window of interest, with significantly different characters, such that a 1:1 mapping to the spin-flip excitations from the other methods cannot be established and χ2 cannot be defined. Correspondingly, the FGWF excitation energies are colored gray in Fig. 4.

Fig. 4: Basis set influence on downfolded excitation energies.
Fig. 4: Basis set influence on downfolded excitation energies.
Full size image

Comparison between maximally localized and first-guess Wannier function basis sets using static cRPA Coulomb interactions. Dashed and solid lines represent spin-flip (SF) and crystal-field (CF) excitations, respectively. (Light) Gray excitation energies could not be (have not been) characterized.

This comparison strikingly shows that projector-like first-guess Wannier functions, which are not constrained to reproduce the DFT single-particle energies (see section “Methods” for details), can result in quantitatively and even qualitatively wrong many-body excitation properties.

As shown in Table 1, the single-particle energies in the FGWF basis are not as different from MLWF basis functions as other modifications; however, the interactions as shown in Tables 2 and 3 differ by up to 27%. Thus, at least in this case, the basis functions are important mainly because of their influence on the interactions rather than the single-particle energies.

Table 1 Single-particle energies (in eV) from DFT output (KS), after wannierization (MLWF and FGWF), and including double-counting corrections (MLWF+H)
Full size table
Table 2 Bare and cRPA-screened density-density (viijj) and Hund’s exchange (vijij) Coulomb matrix elements in the Wannier basis of the MLWF in eV
Full size table
Table 3 Bare density-density (viijj) and Hund’s exchange (vijij) Coulomb matrix elements in the Wannier basis of the FGWF in eV
Full size table

Single-particle energies

Although it is convenient to extract \(\hat{t}\) from the DFT energies, there are several uncontrolled approximations in doing so. First of all, the Kohn-Sham (KS) DFT energies are known to be auxiliary quantities that are not guaranteed to accurately represent the excitations of the system53,54, even if the exact density-functional were available. Secondly, the DFT calculation already incorporates some electronic interaction effects, while the (ED) solution of the downfolded Hamiltonian will add further interaction effects. Correcting for this double counting could be either achieved by starting from constrained GW calculations13 or by using a so-called double-counting correction term in the model Hamiltonian, which aims to remove any Coulomb interaction effects from the single-particle starting point. We consider two modifications to the single-particle energies: a Hartree double-counting correction20,55,56, which decreases the energy difference between a1 and e1 by more than 1 eV such that the crystal-field excitations shift significantly up in energy (see Table 1), and rigid shifts of the energy difference between a1 and e1 levels by approx. ± 150 meV. We use the rigid shifts rather than varying the functional because varying the DFT functional also changes orbitals, which could result in conflated effects. For discussions on how to possibly vary the single-particle correction scheme for different DFT functionals, we refer the interested reader to refs. 20,55.

Figure 5 shows how modifications to the single-particle terms affect the excitation energies in the downfolded model. The spin-flip excitations are barely affected by single-particle energy changes, as they are mainly defined by exchange Coulomb matrix elements, which are unaffected by single-particle double-counting corrections. The crystal-field excitations can, however, change significantly: Increasing the energy difference between a1 and e1 with \({H}_{{{{\rm{CFC}}}}}^{{a}_{1}}\) by about 150 meV lowers the crystal-field excitation energies while decreasing the difference between a1 and e1 by about 150 meV using \({H}_{{{{\rm{CFC}}}}}^{{e}_{1}}\) shifts the crystal-field excitations up in energy. This trend also holds using the Hartree double-counting correction, which decreases the energy difference between a1 and e1 by more than 1 eV such that the crystal-field excitations shift significantly up in energy (see Table 1 for further details).

Fig. 5: Single-particle double-counting correction influence on downfolded excitation energies using the MLWF basis and static cRPA Coulomb interactions.
Fig. 5: Single-particle double-counting correction influence on downfolded excitation energies using the MLWF basis and static cRPA Coulomb interactions.
Full size image

Dashed and solid lines represent spin-flip (SF) and crystal-field (CF) excitations, respectively. Light gray excitation energies have not been characterized.

In all of these cases, we can exactly map the excited many-body states to the reference ones, allowing us to calculate χ2. We see that \({H}_{{{{\rm{CFC}}}}}^{{a}_{1}}\) and \({H}_{{{{\rm{CFC}}}}}^{{e}_{1}}\) only mildly affect χ2. The Hartree single-particle double-counting correction, however, significantly increases χ2, yielding a poor agreement with the reference data. From this, we can conclude that the Hartree double counting is not a favorable correction scheme for charge-neutral local excitations in our vanadocene test system. A similar effect from Hartree double-counting corrections has already been observed for Fe impurities in AlN20. In fact, we find the best agreement with the first-principles reference without applying a double-counting correction at all. This is in line with conventional double-counting corrections, as regularly applied in DFT++ schemes57,58,59,60, which do not have an orbital dependence within the correlated space.

The effects of the single-particle energies on the accuracy of the model are relatively straightforward compared to the other terms. As long as the energies are not too much in error, the ground state and spin excitations are robust to their precise value. The crystal-field excitations are shifted roughly linearly with the single-particle energies.

Static screening model

We now investigate how different screening models affect the charge-neutral excitation spectrum. To this end, we show in Fig. 6 the original data as obtained from using the conventional screened cRPA Uijkl tensor (referred to as “full”), together with results obtained from a cRPA tensor which is predominantly screened in the monopole channel (defined here as the leading eigenvalue of the screening, as explained in section “Methods”), and results from using the bare (unscreened) Coulomb interaction tensor. The “monopole” cRPA model is inspired by Scott and Booth61, and further details can be found in the section “Methods”.

Fig. 6: Static screening model influence on downfolded excitation energies using the MLWF basis and no double-counting correction.
Fig. 6: Static screening model influence on downfolded excitation energies using the MLWF basis and no double-counting correction.
Full size image

Dashed and solid lines represent spin-flip (SF) and crystal-field (CF) excitations, respectively. Light gray excitation energies have not been characterized.

The full cRPA results that have already been discussed yield a χ2 = 27.3. Restricting the cRPA screening predominantly to the monopole channel, such that predominately density-density Coulomb matrix elements are screened, yields a better agreement with χ2 = 16.5. This additional constraint on the cRPA screening shifts the crystal-field excitation energies down by about 200 meV, and clusters the spin-flip excitations into 3:1 pairs, with the highest spin-flip excitations being closer to the lowest crystal-field excitations (in Fig. 6 they actually now overlap). This results in a better agreement with the reference data as quantified by the reduced χ2 value. When screening is completely neglected, i.e., when we use the bare vijkl Coulomb tensor, the agreement with the reference data as measured by χ2 becomes, however, unsatisfactory compared to the (monopole) cRPA cases. In the bare case, the crystal-field excitation energies are strongly underestimated, which mixes the overall ordering of the spin-flip and crystal-field excitations, resulting in a large error, χ2 = 86.5. The spin-flip excitations, however, also show the 3:1 clustering similar to the monopole screening case.

From this study, we learn that the cRPA screening model is indeed better than bare Coulomb interactions. However, the screening model is not perfectly accurate; a modification of only screening the monopole channel, motivated by the underlying RPA screening by long-range charge fluctuations61, results in better agreement of the model eigenstates with the quantum chemistry reference results, at least for vanadocene.

Finite frequency screening

The Coulomb tensor in the cRPA approximation is frequency-dependent11, i.e., \({\hat{U}}_{{{{\rm{cRPA}}}}}(\omega )\). To obtain a static Hamiltonian which is easier to solve, it is conventional to evaluate \({\hat{U}}_{{{{\rm{cRPA}}}}}(\omega =0)\), based on the idea that the low-energy target space physics is slow compared to the screening processes. To assess this approximation, we study how the results change when the cRPA Coulomb matrix elements are evaluated at finite frequency. Here, we perform this analysis based on a simple model of the frequency dependence, the plasmon pole model (see Methods for details).

Figure 7 shows the excitation energies using a Coulomb tensor \(\hat{U}(\omega )\) evaluated at various ω, which are all below the estimated background screening plasmon pole frequency of approximately 16.5 eV. Note that these are all calculated using the monopole screening model of Fig. 6, which corresponds to ω = 0 here. For ω = 4 eV and ω = 8 eV, we find an increase in the crystal-field excitation energies and a concomitant small improvement in the match with the reference data as quantified by χ2. The spin excitation energies barely change for these ω values. ω = 12 eV is close to the plasmon pole frequency of 16.5 eV, so the Coulomb matrix elements change more rapidly, and the excitation energies change more as a result. At this point, the agreement with the reference data deteriorates substantially. We should note that the theoretical justification of cRPA relies on the screening frequency ω being smaller than the gap in the rest space38, which is roughly 6 eV in vanadocene (Fig. 1). I.e., for ω > 6 eV, the dynamics of particle-hole excitations start to become relevant, which we cannot capture in the static Hamiltonian formalism.

Fig. 7: cRPA screening frequency influence on downfolded excitation energies within the monopole channel using the MLWF basis and no double-counting correction.
Fig. 7: cRPA screening frequency influence on downfolded excitation energies within the monopole channel using the MLWF basis and no double-counting correction.
Full size image

Dashed and solid lines represent spin-flip (SF) and crystal-field (CF) excitations, respectively. Light gray excitation energies have not been characterized.

With regard to the frequency choice in the cRPA background screening, we thus find slightly better agreement with the reference data upon using ω > 0 in comparison to ω = 0. Among all benchmarked quantities, however, the background screening frequency has the smallest effect.

Discussion

By comparing the ground state and charge-neutral excited states obtained directly from first-principles descriptions and from minimal downfolded Hamiltonians, we have provided a systematic and quantitative sensitivity analysis of the DFT+cRPA scheme for the ab initio derivation of interacting model Hamiltonians for the description of correlation effects in the vanadocene molecule. We expect that the results in this study will be most applicable to similar systems, such as defects in gapped semiconductors or correlated bulk materials with well-separated correlated sub-spaces. These systems are ideal for cRPA, as noted by some of us38, and even so, we were able to quantify errors in every step of the DFT+cRPA process.

To summarize, we find the following about the DFT+cRPA process: Orbital shapes are crucial. The single-particle energies for FGWF are not that different from MLWF but the model is extremely inaccurate because the two-body terms highly depend on the orbital shapes. Naïve application of the orbital-dependent double-counting corrections results in very poor estimations of crystal-field excitations, because it fails to describe the one-particle levels. Small corrections do not change much within our uncertainties. The cRPA-screened interactions are much better than the bare ones. Monopole-only screening is clearly better than traditional. Frequency dependence, in this case, is not that big, unless one goes to very high energies.

From this data, we can make a few recommendations. One should be very careful about the basis orbitals, especially in cases where disentanglement of bands is necessary since orbital shapes can completely change the results. Similarly, the function used to choose localized orbitals might have significant effects on the interactions computed. Double-counting corrections do not necessarily improve the accuracy of the Hamiltonian. These “corrections” can also be conflated with the tendency of DFT to incorrectly compute the one-particle energies. Even in the case of vanadocene, which is a best-case scenario for cRPA, a naïve application of cRPA screening obtains interactions that are better than bare, but appear to be limiting accuracy. In this case, they can be improved by monopole-only screening in particular, although it remains to be seen if such a method is generally more accurate.

Looking forward, we have demonstrated that it is possible to generate reference quantum mechanical data of sufficient quality and quantity to systematically analyze heuristic (but inexpensive) methods like DFT+cRPA, at least for model systems such as vanadocene. The application of multiple first-principles methods allows us to evaluate the uncertainty in the reference data, partially mitigating the problem of overfitting. The data from this work can serve as a reference for other methods, such as GW-based approaches, and could be applied to a variety of materials and systems and has been made publicly available62.

Methods

DFT calculations

We use Quantum Espresso (QE)63,64,65 and embed the vanadocene molecule in a 15 × 15 × 15 Å3 supercell to minimize spurious wave function overlap and undesired screening between repeated cells. The plane-wave cut-off was set to 440 Ryd, and we applied the spin-restricted (spin-unpolarized) generalized gradient approximation within a PBE functional66. In these spin-restricted calculations, we find an ordering of e2 < a1 < e1, which is different from spin-unrestricted results for both C2h and D5h VCp2 where the e2 < a1 ordering is reversed between the spin channels45,67. The occupations of the three lowest d orbitals are constrained to exactly 1 to avoid partial state occupations due to the conventional smearing methods in plane-wave DFT codes. This constraint does, however, not significantly affect the Kohn-Sham energies compared to calculations with a large smearing, c.f. Table 1.

Wannier basis sets

We use the three highest (partially) occupied and the two lowest unoccupied molecular orbitals with predominant V d-orbital character to construct five maximally localized Wannier functions (MLWF) \(\left\vert {\phi }_{i}\right\rangle\) utilizing RESPACK68. For the construction of these MLWFs, we use an inner (frozen) window to constrain the energies of the five Kohn-Sham states of interest. The resulting Wannier orbitals resemble the expected V d orbital symmetries, however, with small lobes at the carbon rings as visualized in Fig. 1, which is in good agreement with ref. 67. These indicate a finite hybridization between the V d states and the carbon rings.

To investigate this specific choice of Wannier functions and to qualitatively compare to projector-based methods, we construct a second Wannier basis without using the frozen (inner) window, utilizing a larger Wannierization (outer) window, and without performing maximal localizations. The resulting first-guess Wannier function (FGWF) single-particle energies do not perfectly reproduce the KS ones, as indicated in Fig. 8 and listed in Table 1, and cannot be generated via some uniform rotation of the MLWFs. These FGWF orbitals are nevertheless similar to the MLWFs, but do have a slightly different shape, which is reflected in the overlap matrix elements between Wannier orbitals of the same symmetry:

$$\begin{array}{rcl}&&\left\langle {\phi }_{{e}_{2}}^{{{{\rm{MLWF}}}}}| {\phi }_{{e}_{2}}^{{{{\rm{FGWF}}}}}\right\rangle \,\approx \,0.936,\\ &&\left\langle {\phi }_{{a}_{1}}^{{{{\rm{MLWF}}}}}| {\phi }_{{a}_{1}}^{{{{\rm{FGWF}}}}}\right\rangle\, \approx \,0.961,\\ &&\left\langle {\phi }_{{e}_{1}}^{{{{\rm{MLWF}}}}}| {\phi }_{{e}_{1}}^{{{{\rm{FGWF}}}}}\right\rangle\, \approx \,0.987.\\ \end{array}$$
Fig. 8: Single-particle properties.
Fig. 8: Single-particle properties.
Full size image

Left: DFT Kohn-Sham energies. Red lines mark single-particle energies resulting from diagonalization of the downfolded Hamiltonian using first-guess Wannier functions. Right: First-guess Wannier functions used within the downfolding procedure.

Model Hamiltonian matrix elements and cRPA screening

With these Wannier basis sets, we evaluate the hopping matrix elements tij = 〈ϕi∣HDFT∣ϕj〉 and the bare vijkl = 〈ϕiϕj∣v∣ϕkϕl〉 as well as the statically cRPA-screened Uijkl = 〈ϕiϕj∣U(ω = 0)∣ϕkϕl〉 Coulomb matrix elements within RESPACK68. The resulting single-particle energies are given in Table 1, and the density-density and Hund’s exchange matrix elements are listed in Tables 2 and 3. We see that density-density elements are screened by about 55 to 60% (up to 7.8 eV), while Hund’s elements are reduced by only 5 to 10% (up to 60 meV). Analysing the leading and sub-leading screening contributions yields a screening of ε1 = εmono ≈ 2.44 in the predominate monopole channel and εi>1 = εmulti ≈ 1.04 to 1.21 in the sub-leading multipole channels. These screening channels are defined by εi = vi/Ui with vi being the eigenvalues of the full vijkl tensor evaluated in a product basis. Ui is the approximated eigenvalues of Uijkl obtained using the eigenbasis of the bare vijkl69. The full cRPA screening is thus most dominant in the monopole channel, which mostly affects density-density interactions, while the screening becomes small in the multipole channels, which mostly affect Hund’s exchange elements.

Inspired by Scott and Booth61 we also construct a cRPA Coulomb tensor, which is screened mostly in the leading monopole channel, such that only ε1 > 1, while all other εi>1 = 1. The resulting matrix elements are listed in Table 2. As expected, the approx. “monopole” cRPA Coulomb matrix elements agree with the full cRPA ones in the density-density and approx. with the bare ones in the Hund’s exchange channels. However, note that the monopole-screened Hund’s exchange elements are up to 77 meV larger than the fully screened ones, which is relevant for the spin-flip excitation energies.

For the analysis of finite background screening frequencies, we calculate Uijkl(iωn) = 〈ϕiϕj∣U(iωn)∣ϕkϕl〉 for a few Matsubara frequencies iωn. Afterwards we fit the leading eigenvalue to a single plasmon pole model of the form \({U}_{1}(i{\omega }_{n})={v}_{1}-{g}^{2}\frac{2{\omega }_{p}}{{\omega }_{n}^{2}+{\omega }_{p}^{2}}\) yielding g ≈ 17.6 eV and ωp ≈ 16.5eV. This allows us to adjust the real frequency of the leading eigenvalue as \({U}_{1}(\omega )\,\approx \,{v}_{1}+{g}^{2}\frac{2{\omega }_{p}}{{\omega }^{2}-{\omega }_{p}^{2}+i\delta }\) for which we use δ = 2.5 eV. This way, we can continuously adjust the leading eigenvalue in the monopole screening model as done for Fig. 7. The rather large δ = 2.5 eV is used here to mimic the effects of screening channels beyond the single plasmon pole model. The dependence of the density-density monopole-screened Coulomb interaction matrix elements is depicted in Fig. 9.

Fig. 9
Fig. 9
Full size image

Approximated frequency dependence of the monopole-screened cRPA Coulomb matrix elements in the density-density channel.

Although the FGWFs show deviations of just up to 6.4% in comparison to the MLWF ones (as measured by the overlaps given above), the resulting bare Coulomb matrix elements are different by up to 27% as a result of the \({\phi }_{i}^{4}\) dependence of v and as visible from Table 3. This is a result of the enhanced atomic character of the FGWFs and their reduced hybridization with the carbon rings, yielding a smaller spread and, thus, enhanced bare Coulomb matrix elements.

Double counting

We choose three orbital-dependent correction schemes, which all act on the single-particle tij hopping matrix elements according to the following terms, which we add to the model Hamiltonian:

$${H}_{{{{\rm{DC}}}}}^{{{{\rm{Har}}}}}=\sum\limits_{ij,\sigma }{c}_{i\sigma }^{{\dagger} }{c}_{j\sigma }\sum\limits_{kl}{\rho }_{kl}{U}_{iljk},$$
(2)

with ρkl the single-particle density-matrix in the Wannier basis obtained from the DFT calculation. This is the conventional Hartree double-counting correction as discussed in detail in refs. 20,55,56. We also apply ad hoc chosen crystal-field corrections of the form

$${H}_{{{{\rm{CFC}}}}}^{({a}_{1}/{e}_{1})}=\sum\limits_{i,\sigma }{\Delta }_{i}^{({a}_{1}/{e}_{1})}{c}_{i\sigma }^{{\dagger} }{c}_{i\sigma }$$
(3)

with only \({\Delta }_{i = {a}_{1}}^{({a}_{1})}=-0.143\) eV or \({\Delta }_{i = {e}_{1}}^{({e}_{1})}=-0.150\) eV non-zero elements for the respective HCFC. These corrections thus solely shift the a1 or e1 states, while they preserve all symmetries. With these, we aim to investigate the effects of possibly wrong single-particle energies.

Exact Solutions and Benchmark Quantities

We use the exact diagonalization (ED) routines (atom_diag) from TRIQS70 to solve the downfolded many-body Hamiltonian \(H={\hat{t}}_{\alpha \beta }{c}_{\alpha }^{{\dagger} }{c}_{\beta }+{\hat{U}}_{\alpha \beta \gamma \delta }{c}_{\alpha }^{{\dagger} }{c}_{\beta }^{{\dagger} }{c}_{\gamma }{c}_{\delta }\), giving us access to both the energies and the many-body wave functions in the target space. Here, we restrict ourselves to charge-neutral excitations, i.e., those with N = 3 electrons in total, such that the relevant many-body wave functions are of the form \(\left\vert \Psi \right\rangle =\mathop{\sum}\nolimits_{\alpha \beta \gamma }{A}_{\alpha \beta \gamma }{c}_{\alpha }^{{\dagger} }{c}_{\beta }^{{\dagger} }{c}_{\gamma }^{{\dagger} }\left\vert 0\right\rangle\), where the sum runs over spin and orbital indices. For three electrons, the possible values of Sz are ± 3/2 and ± 1/2. Note that our model has SU(2) spin symmetry, so the spin S is a good quantum number, and states with the same S and different Sz have the same energy. We can thus restrict ourselves to states with Sz = 1/2.

Using the Wannier functions ϕα(r), we calculate the electron density ρi(r) in real-space for a given many-body wave function i as \({\rho }_{i}({{{\bf{r}}}})=\mathop{\sum}\nolimits_{i,\alpha }\langle {\Psi }_{i}| {c}_{\alpha }^{{\dagger} }{c}_{\alpha }| {\Psi }_{i}\rangle | {\phi }_{\alpha }({{{\bf{r}}}}){| }^{2}\) using the many-body states Ψi as obtained from the exact diagonalization of the Hamiltonian. For the spin-resolved density \({\rho }_{i}^{\sigma }({{{\bf{r}}}})\) we use a similar expression yielding \(\int\,d{{{\bf{r}}}}{\rho }_{i}^{\uparrow }({{{\bf{r}}}})=2\) and \(\int\,d{{{\bf{r}}}}{\rho }_{i}^{\downarrow }({{{\bf{r}}}})=1\) in the Sz = 1/2 channel. In addition to this, we also calculate the 1-RDMs \({({\rho }_{i})}_{\alpha \beta }=\langle {\Psi }_{i}| {c}_{\alpha }^{{\dagger} }{c}_{\beta }| {\Psi }_{i}\rangle\) for each wave function Ψi.

Based on these quantities, that is, the many-body state energies, wave functions, 1-RDMs, and charge densities, we can unambiguously identify all states among all applied methods. A corresponding comparison between the many-body wave functions represented as linear combinations of Slater determinants from DFT+cRPA and the multi-Slater part of the trial wave functions for DMC is depicted in Fig. 10. The corresponding comparisons for the 1-RDMs and the many-body charge densities are given in the Supplemental Notes 1 and 2.

Fig. 10: Comparisons of many-body wave functions from quantum chemistry and downfolded model.
Fig. 10: Comparisons of many-body wave functions from quantum chemistry and downfolded model.
Full size image

a, b The squares of configuration interaction (exact diagonalization) coefficients (∣ci∣2) of the optimized multi-Slater-Jastrow wave functions obtained using variational Monte Carlo and eigenstates obtained from ED using the model given by the MLWFs. The four panels correspond to the 0th-order configuration interaction (purple), single excitations (green), double excitations (orange), and triple excitations (red) within the CAS(5, (2,1)) space. The tick labels on the x-axis, (i, j) (k), represent a Slater determinant \({c}_{k\downarrow }^{{\dagger} }{c}_{j\uparrow }^{{\dagger} }{c}_{i\uparrow }^{{\dagger} }\left\vert {\Phi }_{0}\right\rangle\) (see panel e for the MLWFs), where \(\left\vert {\Phi }_{0}\right\rangle\) is the Slater determinant of all the valence electrons doubly occupying the orbitals up to the V d shell. Panels c, d show the cumulated contributions in each order of CIs for the VMC wave functions and the model eigenstates. The black horizontal lines group the many-body states into ground state, spin-flip excitations, and crystal-field excitation, whereas the gray horizontal lines group the degenerate states together. We can see that states 3 and 4 (degenerate in energy and correspond to the 2nd excited state, labeled by blue dashed lines in Fig. 2) and all the crystal-field excitations show significant contributions from the double excitations.

Real-space fixed-node Diffusion Monte Carlo (DMC)

In order to obtain approximate many-body eigenstates using fixed-node diffusion Monte Carlo (DMC), we constructed the trial wave functions in the following multi-Slater-Jastrow form,

$$\begin{array}{ll}{\Psi }_{{{{\rm{T}}}}}({{{\bf{R}}}})={\Psi }_{{{{\rm{T}}}}}({{{{\bf{r}}}}}_{1},{{{{\bf{r}}}}}_{2},\cdots \,,{{{{\bf{r}}}}}_{N})\\ \qquad\quad\,\,={{{\mbox{e}}}}^{J({{{{\bf{r}}}}}_{1},{{{{\bf{r}}}}}_{2},\cdots ,{{{{\bf{r}}}}}_{N})}\sum\limits_{k}{c}_{k}{D}_{k}^{\uparrow }\left({\chi }_{i\uparrow }({{{{\bf{r}}}}}_{j})\right){D}_{k}^{\downarrow }\left({\chi }_{i\downarrow }({{{{\bf{r}}}}}_{j})\right),\end{array}$$
(4)

where ri represents the real-space coordinates of the ith electron and R encompasses the configurations of all the electrons. The multi-Slater-determinant expansion, given by \(\mathop{\sum}\nolimits_{k}{c}_{k}{D}_{k}^{\uparrow }({\chi }_{i\uparrow }({{{{\bf{r}}}}}_{j})){D}_{k}^{\downarrow }({\chi }_{i\downarrow }({{{{\bf{r}}}}}_{j}))\), was generated from restricted Hartree-Fock (RHF) followed by state-averaged complete active space self-consistent field (CASSCF)71 using the PySCF package72 with correlation-consistent effective core potentials, and a corresponding triple-zeta basis set39. The system consists of 63 valence electrons, and in CASSCF, we chose an active space that includes two spin-up electrons, one spin-down electron, and five molecular orbitals that are V-3d-like determined by the atomic valence active space (AVAS) procedure73. For all the eigenstates, the parameters in the two-body Jastrow factor J, all the determinant coefficients ck, and the molecular orbitals in the active space were fully optimized using variational Monte Carlo30 using the method described in refs. 74,75. The excited states were optimized with the constraint that they are orthogonal to the optimized lower-energy eigenstates32. After the trial wave functions ΨT (of either the ground state or the excited states) were fully optimized, the lowest-energy wave functions that have the same nodal structure were projected out using DMC30. We then compute the energies and reduced density matrices of the fixed-node wave functions of all the eigenstates of interest. All the variational and diffusion Monte Carlo calculations, including the optimization of trial wave functions and DMC, were performed using the PyQMC package76 interfaced with PySCF.

Using reduced density matrices in natural orbitals to characterize many-body eigenstates

For a wave function \(\left\vert \Psi \right\rangle\), its one-body and two-body reduced density matrices (1(2)-RDMs) in a given single-particle basis are defined as

$$\begin{array}{ll}\quad{\rho }_{ij}^{\sigma }[\Psi ]\,=\,\left\langle \Psi \left\vert {c}_{i\sigma }^{{\dagger} }{c}_{j\sigma }\right\vert \Psi \right\rangle,\\ \,\,{\rho }_{ijkl}^{\sigma {\sigma }^{{\prime} }}[\Psi ]\,=\,\left\langle \Psi \left\vert {c}_{i\sigma }^{{\dagger} }{c}_{k{\sigma }^{{\prime} }}^{{\dagger} }{c}_{l{\sigma }^{{\prime} }}{c}_{j\sigma }\right\vert \Psi \right\rangle ,\end{array}$$
(5)

where \({c}_{i\sigma }^{{\dagger} }\) (ciσ) creates (annihilates) an electron in single-particle orbital χiσ. Almost all the one- and two-body observables can be computed using the 1- and 2-RDMs. For example, the spin-resolved electronic density is evaluated as

$${\rho }^{\sigma }({{{\bf{r}}}})=\sum\limits_{i}{\chi }_{i\sigma }^{* }({{{\bf{r}}}}){\rho }_{ii}^{\sigma }\left[\Psi \right]{\chi }_{i\sigma }({{{\bf{r}}}}).$$
(6)

To directly compare the observables that characterize the fixed-node eigenstates with those of the downfolded eigenstates, it is crucial to ensure that the RDMs are calculated using an identical set of single-particle orbitals. Given that the downfolding and DMC calculations were performed using separated packages and employed a different basis to expand the active space (MLWFs for the downfolded results and CASSCF molecular orbitals for DMC), we rotate the RDMs obtained from each method to the same basis set to align the many-body eigenstates.

The natural orbitals are defined as the principal components, that correspond to the largest five singular values, of all the 1-RDMs up to the mth downfolded excited state77,

$$\begin{array}{ll}\varrho \,=\,U\Sigma V,\\ \varrho \,=\,\left({\rho }^{\uparrow }[{\Psi }_{0}],\cdots \,,{\rho }^{\uparrow }[{\Psi }_{m}],{\rho }^{\downarrow }[{\Psi }_{0}],\cdots \,,{\rho }^{\downarrow }[{\Psi }_{m}]\right),\end{array}$$
(7)

where ρσ[Ψa] is eigenstate Ψa’s 1-RDM written in the Wannier basis \(\left\{{\phi }_{i}({{{\bf{r}}}})\right\}\). We then transform all the 1- and 2-RDMs computed from both downfolding and DMC to these natural orbitals. This ensures that we characterize downfolded and DMC eigenstates within the same Hilbert space. We found \({{{\rm{Tr}}}}\left({\rho }_{{{{\rm{downfolded}}}}}\left[{\Psi }_{i}\right]\right)-{{{\rm{Tr}}}}\left({\rho }_{{{{\rm{DMC}}}}}\left[{\Psi }_{i}\right]\right)\) to be 0.1 ~ 0.2, for all the Ψi’s of interest (up to the 2nd charge excitation). With three electrons in the active space, this suggests that when using the MLWFs derived from Wannierization of Kohn-Sham orbitals to characterize the low-energy excitations in VCp2, we get about 5% error in terms of their 1-RDMs.

Auxiliary-field quantum Monte Carlo (AFQMC)

We computed the low-energy many-body spectrum, including spin-flip, and crystal-field excitations, of the vanadocene molecule using phaseless AFQMC29,78,79. All AFQMC calculations were performed using the Flatiron Institute, Center for Computational Quantum Physics’s production quality AFQMC code. A detailed description of the AFQMC method was recently reviewed in great detail78. AFQMC was recently used to compute the ionization potentials of several 3d-transition metal complexes, including vanadocene, producing results that compare favorably with experiment33. Our AFQMC procedure, especially regarding the choice of trial wave function, is guided by that benchmark. AFQMC calculations were performed using multi-Slater determinant trial wave functions which were derived from CASSCF calculations. The active space for CASSCF was chosen using the atomic valence active space (AVAS) procedure73 and the ANO-RCC basis to define V-3d reference atomic orbitals, with an AVAS threshold of 0.1. This yields an active space consisting of 13 active electrons and 15 active orbitals. CASSCF orbitals were optimized for the ground state, and excited-state CAS-wave functions were computed in one-shot state-specific CASCI calculations using the ground-state CASSCF orbitals to define the active space. AFQMC trial wave functions are truncated CAS-wave functions where Slater determinants with weight less than 0.0014 are discarded. For states with S = 3/2, including the ground state and both crystal-field excitations, the trial wave functions consisted of only about 100 Slater determinants. For all spin-flip excitations, the trial wave functions consisted of about 150 Slater determinants.

Equation-of-motion coupled cluster (EOMCC)

Equation-of-motion coupled-cluster calculations were performed at the single-, and double-excitation level (EOM-CCSD), as implemented in PySCF72. The correlation-consistent effective core potential (ccECP)/pseudopotential and the corresponding ccECP-cc-pVQZ basis were used for all atoms. The ROHF ground state determinant, with Sz = 3/2 and S2 = 3.75, was used as a reference for EOM-CCSD. Due to the very large dimension of the Hilbert space in the ccECP-cc-pVQZ basis, we performed EOM-CCSD in a CAS-like active space with 502 active orbitals and including all electrons in the active space. We checked that the excitation energies are converged by comparing them with similar calculations performed in an active space consisting of 452 orbitals. The average absolute deviation of the excitation energies, computed over the states in Fig. 2, between EOM-CCSD(502o,63e) and EOM-CCSD(452o,63e) is 28(15) meV with a maximum absolute deviation of 53 meV from the first crystal-field excitation.