Introduction

Aberration correction is an important component of high-resolution scanning transmission electron microscopy (STEM). Modern multipole aberration correctors effectively correct for the main spherical aberration (C3 in round electron lenses) but add smaller parasitic aberrations that also need to be corrected1,2,3,4,5. This is typically accomplished using an automated program based on the analysis of Ronchigrams (for example6) or a (CEOS) probe tableau to estimate aberration coefficients from a set of images at different defocii and beam tilts. The program then modifies multipole lenses to compensate for the measured aberrations7,8,9,10. Recognizable features in Ronchigrams (such as streaks) can also be used to manually correct lower-order aberrations such as 2-fold astigmatism (A1) and axial coma (B2). Microscope and corrector lens settings drift on the minutes to hours time scales and fine-tuning during an experiment is necessary to achieve optimal performance11. Methods based on the Ronchigram require a flat amorphous sample region (e.g. a thin carbon film), which can be difficult to find on monolithic crystalline films. For the probe tableau method, a special sample is required and is typically only used to tune the corrector at the start of an experiment. A general method for tuning aberrations that can be used on a wide range of samples is therefore desirable. Ideally, such a method would be fully automated and apply the least amount of beam dose possible to minimize time, damage (especially to beam-sensitive samples), and carbon contamination of a region of interest.

Recently, there has been promising progress in the development of new methods for automating aberration correction. DeepFocus uses a convolutional neural network to calculate the defocus and astigmatism in SEM images from two images at different focus values12. Convolutional neural networks have also been used to extract various metrics from Ronchigrams that can be used to correct first- and second-order aberrations in STEM, such as beam emittance growth13,14 and Strehl ratio15, as well as directly assess aberrations from the Ronchigrams themselves10. In ref. 1, Kirkland demonstrated via simulation that the normalized variance (σ2/μ2) of a high-angle annular dark field (HAADF-) STEM image increases when the magnitude of first, second and third-order aberrations decreases. Ishikawa et al. experimentally demonstrated the same relationship using image standard deviation (σ) to correct first- and second-order aberrations16. Together, these works demonstrate that adjusting aberrations to maximize σ or σ2/μ2 can effectively correct aberrations without directly measuring them. Kirkland used a simplex method to iteratively determine the set of aberration coefficients that maximizes σ2/μ2 using STEM image simulations1, while Ishikawa et al. developed a system that maximized σ for each aberration coefficient sequentially16. A similar concept is utilized by companies such as Thermo Fisher in their OptiSTEM+ software and has been shown to improve microscope alignment for atomic resolution imaging.

In this paper, we present BEACON—Bayesian-Enhanced Aberration Correction and Optimization Network—which uses Bayesian optimization to autonomously correct first- and second-order aberrations, both individually and collectively. Bayesian optimization can efficiently estimate unknown functions with a reduced set of measurements, making it useful in situations where each measurement is expensive in terms of time, cost and/or, in this case, beam dose. It is used for autonomous experiments in synchrotrons17, fMRI studies18 and scanning-probe microscopy19. We employ the metric of normalized variance (NV), proposed by Kirkland for HAADF-STEM and others for SEM, as a metric for image quality20,21,22,23,24,25. We demonstrate the use of BEACON at atomic- and nanoscale resolution using a polycrystalline hafnium dioxide (HfO2) thin film and a sample composed of 5 nm gold nanoparticles (Au NP) on a carbon support to show the method’s viability in different experimental scenarios. We demonstrate that BEACON efficiently compensates for aberrations, leading to higher quality and higher resolution images. Further, we show through ptychographic reconstruction of pre- and post-corrected data that BEACON indeed reduces the target aberrations. The simplicity and robustness of BEACON makes it useful for fine-tuning aberration corrector alignment in low- and high-resolution experiments without human intervention.

Results

Verifying image NV relationship

To verify the correlation between the image NV and aberration coefficient magnitude for both samples, we systematically varied corrector aberration values in small steps (called a “grid search”) and acquired HAADF-STEM images for each of the first- and second-order aberrations: defocus (C1), two-fold astigmatism (A1), axial coma (B2), and three-fold astigmatism (A2). Figure 1 (HfO2 thin film) and Fig. 2 (Au NPs) demonstrate that the image NV (top row) and visual image quality (bottom row) both vary inversely with the aberration coefficient magnitude16,26.

Fig. 1: Grid searches of individual aberrations on the HfO2 thin film.
figure 1

Grid searches of C1 (a, e), A1 (b, f), B2 (c, g), and A2 (d, h) for the HfO2 thin film. a–d Image NV normalized to the maximum value. e–h Tiled cutouts (4.3 nm field of view) of the corresponding full real space images. The zero aberration value was the initial well-corrected state as determined by the CEOS alignment software.

Full size image
Fig. 2: Grid searches of individual aberrations on the Au NP sample.
figure 2

Grid searches of C1 (a, e), A1 (b, f), B2 (c, g), and A2 (d, h) for the Au NP sample. a–d Image NV normalized to the maximum value. e–h Tiled cutouts (24.4 nm field of view) of the corresponding full real space images. The zero aberration value was the initial well-corrected state as determined by the CEOS alignment software.

Full size image

Individual aberrations

We initially tested BEACON on each first- and second-order aberration individually. We started from a corrected state before applying a known aberration value and running BEACON for a total of 35 iterations. Unless otherwise specified, we always initialized the model with 5 randomly chosen points followed by a specified number of iterations using the upper confidence bound (UCB) algorithm (see Section “Bayesian Optimization”). Images were taken before and after the runs to verify the results. The middle row of Fig. 3 shows the final model overlaid with each experimental measurement (colored dots). BEACON finds the peak of the surrogate model (background image), while the top and bottom rows show pre- and post-correction HAADF-STEM images of the HfO2 thin film, respectively, and demonstrate a visual improvement in the image quality. Supplementary Fig. 1 shows the same results for the Au NP sample.

Fig. 3: Correction of individual aberrations.
figure 3

BEACON corrections of C1 (offset = 5 nm) (a, e, i), A1 (x offset = –5 nm, y offset = 5 nm) (b, f, j), B2 (x offset = –150 nm, y offset = 150 nm) (c, g, k), and A2 (x offset = 100 nm, y offset = –150 nm) (d, h, l) aberrations on HfO2 sample. a–d Images before correction. e–h Surrogate models with sampled points marked by dots. i–l Images after correction.

Full size image

To quantitatively assess the performance and reliability of BEACON, we did 5 runs on each sample starting from the same initial aberration settings. We evaluated the correction precision by calculating the standard error of all final aberration coefficients compared to the mean, as a measure of self-consistency (see Table 1). We also assessed the convergence of BEACON by measuring the difference (ΔMM) between the final maximum of the surrogate model and the maximum of the surrogate model after every iteration, then calculating the standard deviations of these differences over the 5 runs (see Fig. 4 and Supplementary Fig. 2). We set benchmarks for a good corrector alignment as ± 1 nm for first-order corrections (C1, A1) and ±20 nm for second-order corrections (B2, A2). We defined the number of iterations needed for convergence as the iteration at which the standard deviation of the differences fell below these benchmarks (see Table 1).

Table 1 Correction precision and number of iterations required for convergence for individually corrected aberrations
Full size table
Fig. 4: Convergence of individual aberration runs.
figure 4

Plots of the average difference (ΔMM) between the model maximum and the final model maximum during BEACON runs on the HfO2 thin film for (a) C1, (b) A1, (c) B2 and (d) A2.

Full size image

Combined aberrations

In the previous section, we corrected each aberration separately with a maximum of two dimensions. However, Bayesian optimization can handle higher dimensional problems and each image NV provides information about multiple aberrations simultaneously.

We grouped first- (C1, A1) and second-order (B2, A2) aberrations because they affect image NV similarly, simplifying the procedure. Figure 5 shows the C1-A1 optimization (three-dimensional) before (Fig. 5a) and after (Fig. 5c) 65 iterations using the HfO2 thin film sample. Figure 5b shows the three-dimensional volumetric plot of the surrogate model of aberration space (estimated NV for each aberration set). A second optimization run for combined B2-A2 with 85 iterations is shown in Fig. 6. The aberration space plots in Fig. 6c–f) are three-dimensional plots obtained by fixing one of the aberration coefficients to its optimum value, enabling the visualization of the four-dimensional model. Equivalent low-magnification results using the Au NP sample are shown in Supplementary Figs. 3, 4. The convergence results of multiple runs on both samples are presented in Table 2. Figure 7 shows the differences between the model maxima and final model maxima (ΔMM). Equivalent low-magnification results using the Au NP sample are shown in Supplementary Fig. 5.

Fig. 5: Correction of combined C1-A1 aberrations.
figure 5

BEACON correction of combined C1 (offset = –5 nm) and A1 (x offset = 5 nm, y offset = –5 nm) on the HfO2 thin film. Magnified portion of the full image a before correction and c after correction. b Volumetric plot of the surrogate model.

Full size image
Fig. 6: Correction of combined B2-A2 aberrations.
figure 6

BEACON correction of combined B2 (x offset = 150 nm, y offset = –150 nm) and A2 (x offset = –150 nm, y offset = 150 nm) aberrations on the HfO2 thin film sample. Magnified portion of the full image a before and b after correction. c–f Three-dimensional volumetric plots of the four-dimensional surrogate model. Each plot was generated using the optimum value for c B2,x, d B2,y, e A2,x, f A2,y.

Full size image
Table 2 Precision and number of iterations required for convergence for combined first- and combined second-order aberration correction
Full size table
Table 3 Precision of C1-A1-B2-A2 aberration correction
Full size table
Fig. 7: Convergence of combined aberration runs.
figure 7

Plots of the average difference (ΔMM) between the model maximum and the final model maximum during optimization runs on HfO2 thin film for a C1-A1 correction and b B2-A2 correction.

Full size image

Finally, we optimized all first- and second-order aberrations simultaneously using 150 iterations initialized with 20 random points. Figure 8 shows the results of these corrections on the HfO2 thin film sample. Equivalent low-magnification results using the Au NP sample are shown in Supplementary Fig. 6. Table 3 includes the precision of these corrections but does not list the number of iterations needed for convergence as none of the standard deviations of the differences from the final model maximum fell below our benchmarks. Figure 9 shows the differences between the model maxima and final model maxima (ΔMM). Equivalent low-magnification results using the Au NP sample are shown in Supplementary Fig. 7. These were only saved every 10 iterations due to the large computation incurred to calculate the full 7-dimensional surrogate model.

Fig. 8: Correction of combined C1-A1-B2-A2 aberrations.
figure 8

BEACON correction of combined C1 (offset = –5 nm), A1 (x offset = –5 nm, y offset = 5 nm), B2 (x offset = –150 nm, y offset = 150 nm) and A2 (x offset = –150 nm, y offset = 150 nm) aberrations on HfO2 thin film sample. Zoomed portion of the full image a before and b after correction.

Full size image
Fig. 9: Convergence of combined C1-A1-B2-A2 runs.
figure 9

Plots of average difference (ΔMM) between model maximum and final model maximum during optimization runs on HfO2 thin film for C1-A1-B2-A2.

Full size image

Discussion

The before and after images in Figs. 3, 5, 6, 8 show qualitative visual improvement in image quality for every optimization performed on the HfO2 sample. Supplementary Figures 1, 3, 4, 6 show similar improvements for the Au NP sample.

Table 1 indicates that optimizations of individual aberrations reached below the aforementioned benchmarks within a reasonable number of iterations. The one exception is that the precision of the A2 correction on the Au NP sample was roughly double the benchmark. In our experience, this still constitutes an acceptable correction of A2 which is typically corrected to < 50 nm using the CEOS alignment software. With the exception of C1, convergence was slower for the HfO2 as compared to the Au NP sample. We attribute this to the different noise levels, which can be seen by comparing the scatter of NV measurements for different C1 values in Fig. 3 and Supplementary Fig. 1. We also note that the two samples required different noise functions (see Section “BEACON operation”). However, the number of iterations required to converge is relatively small for both samples, and BEACON achieves a corrected state with fewer images as compared to the traditional tableau method. Table 4 shows the time per iteration and full run time including additional overhead (normalization, plotting, etc.) for each type of optimization. For comparison, acquisition of a single tableau using the CEOS alignment software used for correcting higher-order aberrations requires 44 images. Each tableau takes 68 seconds and multiple passes are often required to correct second-order aberrations to within our benchmarks.

Table 4 Iteration times and full run times for each category of optimization
Full size table

Table 2 shows that the precision of the C1-A1 corrections was similar to individual C1 and A1 corrections for both samples. The precision of B2-A2 corrections is better than the individual B2 and A2 corrections for the Au NP sample but worse for the HfO2. In particular, the precision of the A2 corrections is three times our benchmark. Additionally, the number of iterations required to converge is larger than the sum of the two individual aberrations in all cases except the C1-A1 correction on the Au NPs. Here, the number of iterations required for C1 and A1 individually (21) is similar to the number required for the combination (20).

Table 3 shows that the precision of C1-A1-B2-A2 corrections was comparable to individual optimizations for the HfO2 thin film, with the exception of C1. However, the precision of corrections using this method on the Au NP sample is worse than corrections of each aberration individually or grouped by order. Furthermore, the standard deviation of the model maxima between different runs does not converge to below 1 nm for C1 and A1 or 20 nm for B2 and A2 for either sample. It is therefore more efficient to either correct the aberrations individually or grouped by order.

Overall, the results indicate that optimizing aberrations individually or grouped by order is more efficient and gives more precise corrections than optimizing all first- and second-order aberrations simultaneously. Still, correcting all first- and second-order aberrations collectively may be useful in certain circumstances. For example, as shown in Fig. 9, the combined C1-A1-B2-A2 optimization successfully improves the alignment, but it fails to converge in a reasonable amount of time. Such a coarse alignment could be followed by fine correction of each aberration individually. Additionally, further hyperparameter optimization may improve the performance of the combined correction method.

As an independent verification of the BEACON method, we used ptychography to estimate aberrations from datasets with intentionally applied aberrations. Electron ptychography is a phase retrieval technique27,28,29,30,31,32,33 that attempts to solve for both the object and the probe from a set of 2D diffraction patterns acquired at a set of probe positions, leading to the common term of 4D-STEM34. Ptychographic reconstructions rely on careful control of the electron beam shape in order to acquire redundant information from the sample. Most often defocus is added to the incident probe to facilitate real space overlap between adjacent probe positions. It is also possible to correct for other aberrations based on the retrieved phase of the probe35.

In this experiment, we added A1 or B2 to a well-corrected state before acquiring a 4D-STEM scan. We subsequently used BEACON to correct the aberrations and acquired a second dataset. We reconstructed both datasets using electron ptychography. Although it is possible to solve for an unknown probe with iterative ptychography, reconstructions converge more quickly with a good initial guess, which includes aberrations. Using phase retrieval methods implemented in py4DSTEM, we typically estimate the probe aberrations with either a tilt-corrected BF-STEM (or parallax imaging) reconstruction33,36,37, or using an independent Bayesian optimization algorithm33. The Bayesian optimization algorithm optimizes over self-consistency error of the ptychographic reconstruction, which in this case was used to estimate the initial aberrations in the probe. These fit aberration guesses provide an independent assessment of BEACON’s ability to correct aberrations beyond image NV and visual image quality.

Figure 10 shows the results of our reconstructions from ptychography experiments with Au NPs. Our initial probe for the first particle had large C1 and A1, as illustrated in Fig. 10a. The second column shows our second scan of the same particle after A1 aberration correction by BEACON. Our estimate is that A1 is reduced from 12 nm to 2 nm, as estimated from Bayesian optimization of probe parameters. The last two columns in Fig. 10 show a similar experiment with B2 instead of A1. In this case, the probe begins with approximately 554 nm of B2, which is decreased to 224 nm. Real space probes are shown in Supplementary Fig. 8. We note that aberration coefficients are defined differently by different groups and the CEOS coefficients used by BEACON are not expected to match the conventions used by py4DSTEM. While large aberration values would interfere with conventional imaging, the ptychographic approach yields high-quality reconstructions with clearly resolved atomic features and an estimate of the probe shape for all scans. Some small differences between subsequent images of the sample particle can be due to beam damage and carbon contamination. Comparing the fast Fourier transforms (FFT) between these two scans (Fig. 10i–j, k–l), we see similar spatial resolution but increased Thon ring contrast with the BEACON-corrected probes. An estimate for the improvement in Thon ring uniformity is shown in Supplementary Table 1.

Fig. 10: Verification by ptychography.
figure 10

Ptychographic reconstruction of two sets of particles: a–d probe in reciprocal space, e–h reconstructed object, and i–l object FFTs. Comparing a to b and c to d we observe reduction in aberrations after BEACON correction.

Full size image

It has been suggested that increasing the phase diversity of the probe, through physical devices or aberration modulation, can improve information in ptychography reconstructions35,38,39,40. However, it is often preferred to remove (especially unknown) higher-order aberrations from the incident beam. In this experiment, there was minimal impact on information transfer in the ptychographic reconstruction from the aberrated probe, but a clear reduction in the target aberration as evidenced by the final probe shape. Our ptychographic reconstructions highlight that BEACON lends itself especially well to the tuning of aberrations for many types of experiments, including conventional dark-field imaging and phase contrast experiments.

Although this work demonstrates a reliable automated aberration correction method that works on different samples, several factors affect its behavior and performance. Since BEACON relies solely on image NV, it is difficult to use in situations where image NV changes independent of aberration values. We found that imaging different sample regions (i.e. due to sample drift or beam shift) reduced the method’s reliability. BEACON incorporates a method based on cross-correlation to ensure the same region is imaged in every iteration. Further, electron beam-induced carbon contamination also tends to reduce the sensitivity of the method. Reducing the applied dose by using a lower magnification or lower beam current can mitigate this, but a clean sample is always preferred.

Additionally, the sample geometry and specific region chosen can affect the behavior and performance. The Au NP sample has many sharp edges and relatively large image contrast. The image NV varies above the noise level over a broad range of aberration magnitudes at low magnifications. Other nanoparticle samples should behave similarly. At atomic resolution, small changes in the imaged region unexpectedly resulted in relatively large changes in NV. Normalization (see equation (1)) is designed to compensate for this effect, but we found that using the same imaged region (identified by cross correlation) was an important feature to include in BEACON. Conversely, thin films such as the HfO2 sample are less sensitive to this effect, because the intensity is relatively uniform across the entire sample. However, for this sample geometry, the NV varies significantly only when atom columns or fringes are resolvable. Once atomic or lattice resolution is lost, NV is insensitive to aberration magnitude. Therefore, when the parameter bounds of the optimization are large and/or the optimization was performed far from the optimum condition, BEACON struggled to converge. In these cases, the NV peak is very narrow (approximating a Dirac delta function) compared to the parameter bounds. Tailoring the parameter bounds to approximately match the shape of the NV peak (which is sample-dependent) is essential to convergence. Further, samples with low intensity contrast in the field of view such as blank carbon films or amorphous thin-film materials cannot be used for this method. Lastly, single crystal samples which are tilted off-axis might also produce an erroneous result, especially for B2.

Finally, the choice of noise estimation function was an important aspect of this work. Corrections performed on HfO2 were relatively noisier when using the noise estimator designed for the Au NP sample. This was because the SNR of the HfO2 NV measurements was intrinsically lower and the noise was amplified by the normalization procedure. Thus, runs on the HfO2 sample used Eq. (5) and runs on the Au NP sample used Eq. (6). We find that Eq. (6) is an effective noise function for samples where the SNR of NV measurements is high, which typically occurs when there is large contrast within the field of view. Equation (5) is more effective for samples where the SNR of NV measurements is low, like for thin films.

To summarize these limitations, BEACON is most successful on clean samples with large contrast variations within the field of view, such as nanoparticles on a carbon film or well-resolved atomic lattices. Conditions where BEACON would fail include samples where carbon contamination builds up between images, beam-sensitive materials where the sample morphology changes under the beam, amorphous materials with low contrast variation and high-index zone axes with small spacings between atoms where the spacings are commensurate with the probe size. BEACON will also fail if the field of view drifts significantly during a single run, such as at extreme magnifications or cryo-cooled samples. Finally, success will also depend on the choice of the noise function used, which may depend on the specific sample characteristics and/or microscope properties.

Kirkland’s simulations demonstrated that image NV can be used to optimize third-order aberrations: spherical aberration (C3), four-fold astigmatism (A3) and axial star (S3)1. Experimental verification proved difficult in our setup. Our software is unable to change C3, but we note that C3 and A1 are coupled. A combined C3-A1 optimization could potentially correct both simultaneously essentially “learning” the relationship between the two aberrations. Also, changes to A3 induced a large shift of the field-of-view which proved difficult to compensate for even with our cross-correlation method. The large image shifts necessary to recenter the field of view induce higher off-axial aberrations making the process self-defeating. The effect of S3 on image NV is shown in Supplementary Fig. 9, but the effect was so subtle that BEACON was less effective than the CEOS alignment software. Third-order aberrations tend to drift much less than first- and second-order aberrations during experiments, and we thus concentrated on the lower orders in this manuscript.

Methods

Normalized variance (NV)

Normalized variance (NV) is expressed as:

$$f({{\boldsymbol{C}}}_{{\boldsymbol{a}}})=\frac{\sigma {({{\boldsymbol{C}}}_{{\boldsymbol{a}}})}^{2}}{\mu {({{\boldsymbol{C}}}_{{\boldsymbol{a}}})}^{2}}$$
(1)

where Ca = [C1, A1, B2, A2, ⋯ ] is the vector or set of aberration coefficients, possibly scaled to the tolerance for each order. σ2 is the variance of the pixel intensities of an image, and μ is the mean pixel intensity, such that

$$\mu =\frac{1}{N}\sum _{ij}{z}_{ij}({{\boldsymbol{C}}}_{{\boldsymbol{a}}})$$
(2)
$${\sigma }^{2}=\frac{1}{N}\sum _{ij}{({z}_{ij}({{\boldsymbol{C}}}_{{\boldsymbol{a}}})-\mu ({{\boldsymbol{C}}}_{{\boldsymbol{a}}}))}^{2},$$
(3)

where zij is the pixel intensity at position i,j and N is the total number of pixels1. Normalization is designed to reduce the sensitivity to specimen drift and changes in beam current. We note that the level of noise affects the image NV but does not vary with changes in aberrations. A greater concern is the build up of carbon during STEM scanning, which decreases the image NV. This can be mitigated by reducing dose by using shorter dwell times, fewer scan points, and minimizing the number of optimization iterations (i.e. the number of images acquired). SEM and HAADF-STEM are both essentially incoherent images and the NV behaves similarly. Traditional BF-CTEM is primarily coherent phase contrast, and NV has not been found to behave as described below. However, incoherent BF-CTEM most likely behaves similarly to HAADF-STEM.

Bayesian optimization

Bayesian optimization is a quick and efficient method of estimating expensive-to-compute unknown functions by minimizing uncertainty41. It generates a surrogate model as a proxy for the unknown function, usually using Gaussian processes (GP)41. If we interpret data as a set of noisy function evaluations of a data-generating, inaccessible, ground-truth latent function f(Ca), a GP assumes that a prior normal distribution can be placed over every finite subset of those function evaluations. This prior normal is fully defined by a prior mean and a covariance or kernel function42. This prior distribution can be optimized (trained) by maximizing the log marginal likelihood of the observed data. Once training is completed, conditioning on observations yields a posterior probability distribution for the latent function values across the domain. This posterior distribution provides a posterior mean (often considered the surrogate model) and posterior variances (often considered the uncertainties), both of which are used by the acquisition function such that maxima represent optimal new points for data collection. The choice of acquisition function defines the behavior of the optimization routine41. One commonly used acquisition function for locating the maximum of an unknown function is the upper confidence bound (UCB) method, where the next measurement point is chosen by the maximum of the UCB. This is mathematically represented by:

$${{\boldsymbol{C}}}_{{\boldsymbol{a}}}^{* }= \mathop{\arg\,\max }\limits_{{{\boldsymbol{C}}}_{{\boldsymbol{a}}}}[{m}_{GP}({{\boldsymbol{C}}}_{{\boldsymbol{a}}})+c{\sigma }_{GP}({{\boldsymbol{C}}}_{{\boldsymbol{a}}})],$$
(4)

where \({{\boldsymbol{C}}}_{{\boldsymbol{a}}}^{* }\) is the next sample point in aberration space, mGP(Ca) is the GP posterior mean, which is the predicted image quality (i.e. the image quality at aberration setting Ca), σGP(Ca) is the posterior uncertainty of the posterior image quality at aberration setting Ca, and c is a coefficient that determines the size of the confidence bound41. Increasing c favors exploration (searching new regions for a higher maximum) while decreasing c favors exploitation (searching closer to the current maximum for a better estimate)43. We used c = 2 throughout this work. Other acquisition functions for Bayesian optimization, such as predicted improvement and expected improvement44, can also be used, but we have found UCB to be most effective. Here, we use Bayesian optimization to determine the optimum aberration corrector alignment. The latent function for the GP framework is the image NV, which we maximize to correct aberrations.

GP calculations in this work were handled by gpCAM v 8.1.445. We used an anisotropic Matern kernel of first order differentiability (ν = 1.5).

BEACON operation

As shown in Fig. 11a, BEACON uses a client-server model to separate the Bayesian optimization process from the microscope control computer. First, the client requests a new measurement with a specific set of aberration coefficients. The server changes the state of the aberration corrector and acquires a new HAADF-STEM image. The server then calculates the image NV and returns this value to the client. The acquired image can also be returned to the client. The client updates the surrogate model of image quality for all sets of aberration coefficients as well as the model uncertainty. The UCB acquisition function then uses the surrogate model and the model uncertainty to choose the next set of aberration coefficients to measure. Figure 11b–d shows how the surrogate model evolves as more data points are added. The region of parameter space in which the peak resides is located with just 5 data points, as shown in Fig. 11b. Subsequent sampling efforts are focused in this region, and the approximate location of the peak is located with 15 data points, as shown in Fig. 11c. BEACON continues to search around this peak to accurately identify the peak, as shown in Fig. 11d. This shows how BEACON can efficiently optimize the aberrations without needing to sample large, extraneous portions of parameter space.

Fig. 11: Schematic of BEACON workflow.
figure 11

a Software schematic of aberration correction workflow. Evolution of an A1 surrogate model after b 5 iterations, c 15 iterations and d 35 iterations.

Full size image

We acquired 512 by 512-pixel images with a dwell time of 3 μs and a current of 70 pA. Large adjustments to aberrations, particularly B2 and A2, result in beam shifts that change the field of view. We use cross-correlation to ensure the same 256 by 256 pixel image region is used throughout an optimization run which also compensates for sample drift. The field of view was 7.8 nm for the HfO2 thin film and 62.4 nm for the Au NP sample. BEACON incorporates a graphical user interface (GUI) as shown in Supplementary Fig. 10 that enables users to quickly tune the optimization parameters for their sample. This GUI is included in the BEACON software, which is generally available (see Code Availability for details).

Bayesian optimization can incorporate noise variances into the calculation of surrogate models. Initial testing showed that proper noise estimation was vital to both the speed and precision of optimization. Measurements from the HfO2 sample were noisier than those from the Au NP sample, requiring the use of different noise estimation functions. For the HfO2 sample, the noise function S was:

$$S=0.0001\times ({y}_{max}-{y}_{min}).$$
(5)

For the Au NP sample, we found that the default noise estimation function used by gpCAM,

$$S=0.01\times | E9| ,$$
(6)

was effective.

Ptychographic reconstructions

For the ptychography experiments, scans were collected at 300 kV using a probe defined with a 25 mrad convergence angle. Approximately 400 Å of defocus was added, and the step size of the acquisition was 1.4 Å. A dose of approximately 2 × 104e−/Å2 was used. We used the 4D Camera to acquire scans46. We reconstructed both datasets using a gradient descent algorithm with batched updates implemented in the open-source toolkit py4DSTEM v 0.14.833,47. A batch size of 1024 measurements was used (1/16 the total number of probe positions), and 100 iterations were run to converge the mean squared error. The probe amplitude was fixed using a vacuum probe measurement taken during the same session. Position refinement and slight low pass filtering was applied during each update. The initial defocus was estimated with a parallax reconstruction. The A1 or B2 values were subsequently estimated using Bayesian optimization, minimizing the mean squared error with 25 initial, uniformly spaced points followed by 25 more optimized calls. More details about the reconstruction formalism can be found elsewhere33.