Introduction

It was shown that at the micron-scale and below crystalline materials (as well as many other heterogeneous materials) exhibit complex deformation behavior including size-related hardening and significant sample-to-sample variation in the plastic response1,2,3. In addition, in this regime deformation becomes intermittent and constitutes of a series of random strain bursts that make the details of the deformation process unpredictable both in time and space4,5. This intermittency and stochasticity originates from the sudden rearrangement events of the dislocation network, the so-called dislocation avalanches. Similar avalanche-like processes, such as the plastic deformation of amorphous materials6,7,8,9, the porous collapse10,11,12, the domain wall dynamics in ferroics13,14,15, the stick-slip behavior of granular matter16,17 or the fracture of paper18,19, are ubiquitous in nature and have gained massive interest in the last few decades as well as the universal features of avalanches20. In order to experimentally study the underlying physical process, that is, the source of the avalanche, it has to be connected to directly related proxies that are experimentally measurable with sufficient precision. This is usually a rather challenging task, since avalanches are fast and mainly occur inside the material below its surface.

Dislocation avalanches were shown to be accompanied by strong burst-like acoustic emission (AE) signals which are elastic waves triggered by the release of stored elastic energy during the event21,22,23,24. As such, AE is a natural candidate as the proxy of the avalanche activity in crystals. It was also demonstrated that other deformation mechanisms, e.g., twinning25,26,27 or martensitic transformation28,29,30 have similar concomitant AE signals. However, when it comes to evaluating the measured AE data, one has to face fundamental problems that stem from the complex nature of the involved physical processes. Firstly, the acoustic waves can be subjects of significant distortion during their travel in the specimen due to reflection and interference, thus, their final form upon reaching the detector depends on the location of their source, the geometry of the sample, the quality of the contact between the sample and detector, etc. Secondly, resonance occurs in the piezoelectric transducer that is unavoidably affected by its properties (geometry, material characteristics, etc.). Finally, the necessary strong electric amplification is influenced by the features of the data acquisition hardware and software. The whole process can be mathematically formulated as \({A_{\rm{AE}}(t)}\,=\,{V(t)}*{T(t)}\), where AAE(t) is the measured AE signal, V(t) is the source function (representing, e.g., the local deformation rate during an avalanche) which is convolved with a T(t) transfer function, that depends on all the specific details mentioned above31,32. The fundamental question of AE measurements is, whether and to what extent it is possible to infer V(t) from AAE(t), that is, how one should interpret the local deformation event from the measured AE signal.

A very common approach to investigate avalanches through the lens of AE is the analysis of some easily accessible features (such as, amplitude, duration, rise time, etc.) of signals12,31,33,34 which relies on a usually threshold-based, somewhat arbitrary AE event detection. Yet, these features do show clear connection with the properties of the avalanche source. A prominent example is the energy associated with AE events that was found to follow a robust power-law distribution for dislocation avalanches, a universal feature that was also recovered from a number of independent experiments, e.g.,17,21,35,36,37, and this was also confirmed by micromechanical experiments on other types of deformation processes and simulations4,5,36,38,39,40. But the rich AE signal (due to its unparalleled time resolution, usually recorded at MHz or higher) may contain far more information about the source than a few noise-affected and threshold-dependent parameters. The question naturally arises: What information (related to the particular avalanche source) is hidden in the acoustic data and how could it be deciphered?

In this work exactly this issue is addressed by analyzing experiments where both AAE(t) and V(t) is accessible. Two current set-ups exist that offer this possibility: (i) micropillar compression coupled with AE and carried out in situ in a scanning electron microscope (SEM) where local deformation can be detected both using the microdeformation stage and also visually from the SEM images36,40. (ii) And more recently Bronstein et al. designed a specific experiment where AE detection was coupled with acceleration measurements to reveal the avalanche source during twinning in Mg samples of cm size range32. Here we focus on the first method, in which the whole jerky stress-strain curve is available with clear signatures (stress drops) of avalanche activity. We address the general question posed above by defining a more specific, technologically rather relevant task: Whether and to what extent can one reconstruct the plastic response of a specimen merely from acoustic data?

To answer this question, a set of micropillar compression tests are utilized during which single crystalline Zn micropillars of various sizes (8, 16 and 32 μm) and oriented for basal slip are compressed and AE is measured simultaneously36,40. The sketch of the experimental set-up is shown in the left panel of Fig. 1. The micropillars (strongly magnified in the schematic) are carved in a macroscopic sample that is firmly attached to the AE sensor. The pillars are compressed using a flat punch diamond tip, that is attached to a force sensor. The latter is based on an elastic spring whose elongation is measured to obtain the acting force. During a measurement the top end of the spring is moved with constant velocity and no feedback for stress or strain control is applied. In this set-up strain bursts cause the elongation of the spring that are measured as sudden force drops (for a recorded jerky force-time curve see Fig. 1). For more details on the experimental set-up, see the Methods section and ref. 36 This set of experiments is ideal for our purpose for two reasons. Firstly, in the given material and orientation plastic deformation is solely caused by dislocation slip on the basal plane, and no other deformation mechanism (such as twinning or crack formation) is active. Secondly, because of the absence of forest dislocations (and any other relevant short-range interaction that would hinder dislocation motion and localize avalanches41,42,43) dislocation avalanches are large and, thus, produce strong, easily measurable AE signals24 (for an example see Fig. 1). To sum up, in these experiments both the force and the acoustic data are available making the prediction of the former based on the latter feasible.

Fig. 1: The workflow of the ML approach.
figure 1

Micropillar compression tests provide force and AE data which are, then, divided into equisized time windows. From the windows, frequency independent and dependent descriptors and force increments (ground truth) are extracted. These are utilized for the training of random forest ML models.

Full size image

Machine learning (ML) has emerged as a powerful tool for extracting relevant information from huge datasets. The richness of our experimental AE data makes application of ML feasible The feasibility is demonstrated by the successful utilization of ML for other AE-related tasks (e.g., for quality monitoring of samples) in the past44,45. For this task the random forest (RF) regressor46 is employed which is more straightforward to interpret compared to more complex neural network based methods. This enables us to analyze the importance of different features of the AE data in the prediction, as it will be shown in the paper. In order to carry out this prediction task two approaches are introduced in this paper. Both methods rely on the simultaneously measured force and AE data for the training of the ML model which is then utilized to predict the force-time response. For the visualization of the workflow see Fig. 1. The advantage of our approaches is that they do not rely on arbitrary thresholding that necessarily loses the information about acoustic signals below a certain size but are based on general descriptors of the AE timeseries such as its statistical moments (in the first approach) or features of its spectrograms (in the second approach).

The main objective of the paper is, therefore, to demonstrate the predictability of the plastic response based on merely the recorded AE signal. The potential foreseen application of such a finding would be deciphering AE data for bulk samples, where no concurrent measurements on plastic events are available. For this reason strong attention will be devoted to the question whether an ML model trained on samples of a certain size can make reasonable predictions for other specimens of unseen sizes.

Results

Relevant time scales of the experiment and the concept for prediction

In order to reveal the connection between the deformation processes and the AE activity, the workflow summarized in Fig. 1 is utilized to predict the force-time response merely from the AE data. As seen, descriptors are defined based on AE timeseries in equisized time windows. Before choosing the window width let us first investigate the relevant time scales of the experiment. (1) On a fine time scale the force-time curve is characterized by strong fluctuations due to force drops caused by dislocation avalanches. This scale can be related to the duration of the deformation processes (directly measurable via the timescale of the stress drops, see Suppl. Fig. S1), typically less than 0.3 s. A division below this timescale could lead to events split into several windows, which could potentially lead to performance loss in the prediction. (2) On a coarser scale the force-time curve is still fluctuating with alternating regimes of mainly strain hardening and softening. This scale can be roughly associated with typical waiting times between the force drops corresponding to duration of the quiescent periods between deformation events. These intervals can last up to ~ 100 s (see Suppl. Fig. S2). It is noted that this timescale is inversely proportional with the compression rate (or, more precisely, platen velocity)36.

The ML workflow should be able to reproduce both features, that is, the details of force drops on the fine scale as well as the overall shape of the force-time curve. To this end, predictions are made on both mentioned timescales. (1) On the fine scale the force change ΔF between the beginning and end of the time window is predicted based on descriptors calculated from the AE signal. The corresponding time window width Δt is determined based on Suppl. Fig. S1 and is chosen as Δt = 300 ms. (2) On the coarse scale the force F itself is predicted at the end of each time window of duration ΔT. For the coarse width ΔT = 50 s was chosen based on Suppl. Fig. S2). The combination of the two predictions leads to a final prediction which inherits both the fine fluctuations and the coarse shape of the curve (for more details on how the two predictions are combined, see the Methods section).

Machine learning model and feature selection

In order to predict the force increment (on the fine scale) or the force value (on the coarse scale) for the time windows, the AE activity within the respective time window has to be represented properly. To this end, firstly, the nature of the AE data at hand should be understood. Figure 2a shows a typical AE burst during the deformation of a 8 μm sample. At the beginning of the event, a medium-frequency ( ~ 250 kHz) pattern emerges from the high-frequency ( ~ 500 kHz) background noise. This is followed by an oscillation with somewhat lower amplitude and frequency ( ~ 100 kHz), presumably due to the mechanical resonance occurring in the piezoelectric transducer. The Fourier analysis of the ensemble of signals detected in 8 μm micropillars shows that there are indeed characteristic frequency ranges (around the three specific values mentioned above) indicated by peaks in the spectrum [see Fig. 2b]. These characteristic frequency ranges correspond to the three major regimes observed, that is, to the background noise, the initial response and the resonant part. This results in a strong similarity among acoustic signals as demonstrated by Fig. 2c. This observation motivates us to focus on information related to these specific frequencies when predicting plastic response based on AE. Therefore, two different approaches are applied: one which does not and one which does contain frequency information about the raw AE data. Accordingly, frequency independent and frequency dependent features are utilized to represent the AE information in these two cases.

Fig. 2: Acoustic signals from compression tests of micropillars with a diameter of 8 μm.
figure 2

a A representative AE signal that constitutes of a few characteristic frequencies. The noise have a mean frequency of ~ 500 kHz. When an event occurs a short ( ~ 20 μs long) signal with a frequency of ~ 250 kHz emerges from the noise which is followed by a longer periodic pattern of frequency ~ 100 kHz. b The absolute value of the Fourier transform \({C}_{{\mathcal{F}}}\) of the frequency f averaged on several time windows centered on signals. The color-coding is consistent with the legend of panel (a) and it highlights the peaks corresponding to the characteristic frequencies. c Typical signals shifted with t0 such that their starting points coincide at t − t0 = 0. These curves demonstrate that the signals and the involved frequencies are rather similar despite their different amplitude/energy.

Full size image

In the first approach statistical features of the AE timeseries (corresponding to time windows) are computed and used as input descriptors. The descriptors are normalized moments of form \(\sqrt[k]{\langle | V{| }^{k}\rangle }\). Here, V(t) is the raw AE signal and 〈•〉 is the average w.r.t. time within the relevant time window. Moments with low integer values of k (such as the mean and the standard deviation) are applied as well as the maximum of V (which is the k → ∞ limiting case). These moments encode information about the fluctuations in the AE signal, however, these descriptors completely lack frequency information.

The second approach is based on the spectrograms of the AE time windows obtained by wavelet transformation (for specific details see the Methods section). The transformation represents the timeseries in the 2D domain of time ⊗ frequency. After obtaining the spectrogram of the time windows, one-dimensional slices are taken corresponding to certain frequencies. Then, moments (of the same form as in the first approach) are computed from this 1D data. This procedure leads to a larger number of features (if the same amount of moments are used) and a richer input for the ML model since it also contains frequency information.

Figure 3 shows the course of the prediction process for an arbitrarily selected example experiment conducted on a micropillar of diameter d = 8 μm. The scatter plot of Fig. 3(a) compares the actual (ground truth) force increments ΔFground at the fine scale of Δt = 300 ms and the corresponding predicted values ΔFpred obtained with both the frequency independent and dependent prediction. The apparent correlation of the actual and the predicted force increment values indicate that the fine-scale prediction is able to uncover when the force drops occur based on the acoustic data. Supplementary Fig. S3 indicates that the performance score R2 (for its definition see the Methods section) is quite robust to the variation of the window width Δt above the duration of the longest force drops [ ~ 200 − 300 ms, see Suppl. Fig. S1(c)]. As it can be expected, however, if some force drops are longer than the window size, the R2 drops, that is, the prediction becomes worse for Δt → 0. These observations validate the choice of Δt = 300 ms which already falls into the regime where the performance scores are high and robust. It can be also noted that the frequency dependent approach achieves appreciably higher scores implying that the frequency information is valuable for the ML model. The coarse shape of the force-time curve can also be predicted well by the coarse-scale prediction scheme. It is demonstrated by the closeness of the predicted force values to the ground truth ones resulting in a predicted curve shape very similar to the ground truth as shown in Fig. 3(b). The final predictions are obtained by the combination of the predictions of the two scales (according to the procedure presented in the Methods section). Figure 3(c) shows that the predicted force-time curve (for both approaches) matches very well the ground truth curve. In particular, the overall shape of the curves is very similar on the coarse time scale and remarkable accordance is seen in terms of the fine serrations between the ground truth and the predictions. However, it should be noted that the prediction does not always closely match the ground truth curve. The comparison of the predictions (of both approaches) and the ground truth curves for all the individual experiments are presented in Suppl. Fig. S8.

Fig. 3: Double-scale prediction of the force-time response of micropillars.
figure 3

a Predictions of fine-scale response of a micropillar of diameter of 8 μm using frequency independent features. The data points correspond to force increments in Δt = 300 ms wide time windows. ΔFground and ΔFpred are the ground truth and the prediction of the force increment, respectively. b Prediction (using frequency independent features) of the coarse-scale behavior of the same micropillar with a time-resolution of 50 s. c Comparison of the final prediction (obtained by combining the fine- and coarse-scale prediction) with the ground truth for an 8-micron specimen.

Full size image

In the following, the importance of the features (of both approaches) in the fine-scale prediction is analyzed in Fig. 4. Specifically, their correlation is examined as well as the performance (in terms of R2 scores) of models trained on single features or on combinations of features. In the case of the frequency independent approach the following conclusions can be drawn. The Pearson correlation of the moments of type \(\sqrt[k]{\langle | V{| }^{k}\rangle }\) and the target variable (that is, the force increment) exhibits a non-trivial dependence on k saturating at a very strong anti-correlation for k → ∞ [see Fig. 4a]. Interestingly, when trained on single moments, the performance of our ML model shows a very different (but also non-monotonic) trend indicating that both the lower moments (such as the mean and the standard deviation) and the maximum contain a comparable amount of useful information [see Fig. 4b]. The analysis of R2 scores of models trained on subsets of moments reveal that the prediction quality quickly saturates (already at around 3 descriptors) if the number of features is increased and the descriptors are chosen correctly [see Fig. 4c]. That is, even with the utilization of a few moments the optimal predictive power of our approach can be reached. Figure 4d shows the optimal choice of features for given number of features n. The constitution of the ideal descriptor choices imply that while the lowest moments are typically very important, in order to achieve optimal performance, lower and higher moments have to be combined. A possible explanation for this observation is the following. In the experiments (on which our analysis is based) plasticity manifests in a sequence of events localised in time which are accompanied by acoustic bursts. If a higher moment (e.g., k = ∞) and a lower moment (for example, k = 1) are considered, their ratio characterises the localisation of the AE signal. For instance, if the lower moment is the same for two distinct time windows Wa and Wb, but the higher moment takes a larger value for Wa, then, the acoustic signal in Wa is more localised, that is, Wa is (more) likely to contain a burst (or bursts) corresponding to plastic activity.

Fig. 4: Feature importance for d = 8 μm samples.
figure 4

a Pearson correlation of the frequency independent features (of form \(\sqrt[k]{\left\langle | V{| }^{k}\right\rangle }\) where V is the raw voltage signal from the piezo-sensors) and the force increment (target variable) for individual 8-micron experiments (small markers) and the mean for the four experiments (large markers). k → ∞ corresponds to the maximum. b R2 score for prediction of force increment based on single features (with the same marker meaning as in (a). The features best-correlated with the target variable are typically the high-order moments, however, the best scores can be achieved using lower moments. c R2 score for prediction based on subsets of n features. All combinations are shown (light point clouds) and the best scores are highlighted with the darker markers. The best score already saturates at around n = 3. d Subset of features corresponding to the best score for each n. The lowest moment is of the utmost importance, however, the subset performing the best usually combines lower and higher moments. e Pearson correlation CPearson of the frequency dependent features and force increment averaged for all 8-micron experiments. Different colors correspond to features based on different order moments (mean, standard deviation, the fourth root of the kurtosis and maximum) of slices of the spectrograms of the AE data (see colorbar). f Mean R2 score for predictions based on single features. The correlations and scores clearly show that information of lower frequencies (100 – 300 kHz, especially around 100 kHz) that are characteristic to acoustic events is the most important. g R2 score for prediction based on subsets of n features. The features used are the moments of order k = 1, 2, 4, ∞ at frequencies 100 kHz, 250 kHz and 500 kHz [marked with dashed lines in panel (e, f)]. All combinations are shown (violin plots) and the best scores are highlighted with the markers. The best score already saturates at around n = 4. h The subset of features corresponding to the best score for each n. The optimal model compositions suggest that the combined use of 100 kHz and 250 kHz features is very beneficial.

Full size image

Similar conclusions can be drawn on the feature importance for the second approach. In this case moments (of order k) were computed based on the slices of the spectrograms corresponding to a varied frequency f. Due to the higher computational cost, in this case only a smaller subset of moments were utilised. The subset k = 1, 2, 4, ∞ was chosen in order include moments of widely changing k values that are expected to have high individual predictive power [based on Fig. 4b] to capture a great spread of information. The analysis of Pearson correlations and single-feature R2 scores highlights the importance of characteristic frequencies also observable for the individual the AE signals (see Fig. 2). Both the correlations and the scores (Fig. 4e, f, respectively) accentuate that the most important frequency range is around 100 kHz (characteristic to the low-frequency tail of the AE signals) and the higher frequencies (up to ≃ 350 kHz) corresponding to the first spikes of the signal also contain valuable information. The predictive power of very low and very high frequencies decays quickly. In order to test the synergy of descriptors four different moments were computed for three different frequencies: 100 kHz (low-frequency tail of signals), 250 kHz (high-frequency burst at signal onset) and 500 kHz (noise). Figure 4g demonstrates that, similarly to the frequency-independent case, the best R2 score can be closely approached with the utilization of very few (around 4) appropriately chosen features. The constitution of the best set of features (for a given number of features) confirms the superior importance of low-frequency features already implied earlier and that it is beneficial to combine low-frequency descriptors with middle-frequency ( ≃ 250 kHz) ones [see Fig. 4h].

From the definition of the features utilised, it is evident that two of them are closely related to traditional measures for characterizing acoustic bursts. Firstly, the moment-type feature of k = 2 is related to the time-integrated ∣V∣2 representing the dissipated energy associated with the acoustic event. Secondly, the case k = ∞ representing the maximum of ∣V∣ is known as the amplitude of the acoustic burst. These two characteristics of the acoustic signals are widely-used to quantify their properties26,31,47. While the amount of dissipated energy and the amplitude have been frequent choices as burst features due to their physical interpretability, our results shed light on their key importance in the prediction. Figure 4d shows that each optimal feature set of a size of n > 2 contains the energy-related k = 2 feature and either the amplitude (k = ∞) or the k = 10 feature strongly correlated (with a Pearson correlation coefficient of 0.995) to the amplitude. The importance of the energy and the amplitude is also accentuated by Fig. 4h. The figure reveals that the most important energy-type feature is the one corresponding to frequency 100 kHz which is understandable since the 100 kHz regime (which takes significantly longer than the initial 250 kHz oscillation, see Fig. 2) of a burst contributes most to the energy type feature. Out of the amplitude-type (k = ∞) features, the most important is naturally the one corresponding to 250 kHz since this is the frequency corresponding to the high-amplitude initial oscillations (see Fig. 2). It is particularly remarkable that these are exactly the two features (k = 2 at 100 kHz and k = ∞ at 250 kHz) that constitute the best performing n = 2 representation of the AE signal. Thus, it can be concluded that the often used energy and amplitude descriptors (combined with ML) can used to recover information about the deformation events very well, however, as Fig. 4 demonstrates, the inclusion of further features of the AE response is beneficial and can significantly improve our prediction further. It is also noted, that the lowest-order (that is, k = 1) moment-type feature is proportional to the characteristic termed as the measured area under the rectified signal envelope (MARSE)48,49,50. This descriptor of the AE burst is less frequently used (due to the lack of simple physical interpretation), however, its immense importance in the prediction [see Fig. 4d] suggests that it is closely related to the physical source (dislocation avalanche in our case) of the burst.

Transferability of the method

The capability of predicting the plastic response merely from the acoustic emission activity was demonstrated above for 8 μm specimens (see Fig. 4). In that setup, prior to the prediction, the ML model was trained on experiments with the same characteristics (same material, same pillar size, etc.) as the test experiment. For the possible applications, however, it is also important whether it is possible to transfer the trained model to other samples where the direct determination of details of the plastic activity is impossible or cumbersome (for instance, in the case of bulk specimens). In order to test the transferability of our method across different sample sizes the following approach is utilised. Sets of experiments are created all consisting of four experiments but with varying composition in terms of pillar diameter (8, 16 and 32 μm). Our ML model is trained on these sets separately and then is tested on a fifth different experiment. This is examined for different training and test set compositions (see Fig. 5).

Fig. 5: Transferability of the ML method across different sample sizes.
figure 5

The R2 score for different combinations of 4-experiment training sets and 1-experiment test sets. The R2 scores are evaluated for the fine-scale predictions. The combinations with the test set of the same micropillar diameter (8 μm, 16 μm or 32 μm) are arranged in distinct three panels. The colour of the violin plots in each panel indicates the test sample size (see colourbar). The size composition of the training sets are showed by the miniature pie charts which use the same colour-coding. The striped slices denote a training experiment that is either of one diameter or of another. In each scenario two point clouds and violin plots indicate the performance for the frequency independent and frequency dependent approaches. Each data point corresponds to the performance score R2 of an individual training-test set pair. The mean R2 scores are indicated by the orange and purple markers in the two cases of frequency independent and frequency dependent approaches. Generally, it can be observed that, even though, the prediction score improves as the similarity of the training and the testing data increases, the model performs reasonably well even if experiments of the same pillar size as in the test experiment were not shown. This observation holds regardless of the inclusion/exclusion of frequency information.

Full size image

The obtained results (quantified with R2 scores) are summarized in Fig. 5. As it can be expected, the scores indicate that the model performs better (on average) if experiments of the same type are included in the training set (or if the ratio of this type is increased). It is remarkable, however, that the scores only moderately lower when the training and validation sets are disjoint in terms of pillar diameter. That is, the model can extrapolate (either to smaller or larger scales) and interpolate to pillars that are of previously unseen type. These results are promising in regards with the potential application of the method to unseen specimen sizes – possibly even to bulk samples. However, it should be remarked that in specimens of larger size regimes several effects can be present (including sound attenuation, varying arrival times from different locations, event clustering, wave propagation effects, different mechanisms, etc.) which could possibly make the application of the present method more challenging.

Discussion

The AE technique is widely utilized for detecting and monitoring damage progression in various structures. It has gained recognition in the engineering community as one of the most reliable and well-established methods in non-destructive testing (NDT). AE technology has proved highly efficient and effective for identifying fracture behavior and fatigue in materials such as metals, fiberglass, wood, composites, ceramics, concrete, and plastics51. However, as it was elucidated in the introduction, AE measurements are inherently characterized by a significant loss of information due to various physical and technological reasons. This loss of information is encompassed in the transfer function. Yet, the success of the various AE applications means that AE data still contains significant and relevant information on damage progression. The facts that AE datasets are huge due to the large sampling rate and that relevant information is obscured in a noisy and fluctuating signal calls for the application of advanced data science methods. Indeed, analysis of AE data is an ideal application for ML, as shown by the recent sheer increase in its applications including damage mechanism identification in composites52,53, crack classification54,55, leakage detection56,57, life time prediction of bearings58, real-time quality monitoring in additive manufacturing45 and welding59. All these applications have in common, that the source of the AE signals (fiber debonding, crack propagation, etc.) cannot be directly observed, so the AE analysis is inherently phenomenological and mainly focuses on classification of events and or detection of undesired behavior. In other words, these methods are unable to characterize the actual transfer function and, consequently, to assess properties of the individual micromechanical damage events.

ML applications on AE employ various ML models to capture the information hidden in acoustic signals. The approaches include ones based on traditional ML algorithms such as random forest44,57,60, support vector machine54,56,58 or k-nearest neighbors52,54,57 and more complex approaches based on artificial neural networks57,58,61, convolutional neural networks45,62 and recurrent neural networks63,64. For these ML applications the acoustic data is typically represented by conventional signal characteristics (such as duration or peak frequency)52,53,59, statistical moments of the timeseries44,57 or by spectrograms45,52,65. The latter two approaches were employed in our two ML methods. Our method fits into this zoo of different strategies by applying a random forest approach to data represented by statistical moments and features extracted from spectrograms. The choice of ML model was motivated by the small number of individual experiments which results in relatively low variety in the data. A simpler model such as the random forest typically outperforms more data-hungry neural networks in such a scenario. If, however, significantly larger number of experiments were available, more complex neural network based approaches would be expected to take advantage of the rich AE data better. It is also noted that similarly to this paper, another work also demonstrated the importance of lower moments (e.g., the variance) in capturing the relevant information in AE data which could be used to, e.g., predict the time to failure in sheared fault gouge materials44. This suggests that representing the AE timeseries by its moments (especially by the lower ones) may be a good choice in general.

Our work focused on a case when dislocation slip was basically the sole deformation mechanism and we investigated the deformation of microsamples. In this case we were able to decipher the AE waveforms of individual AE events in the sense, that we could predict the corresponding plastic strain increments with high fidelity and also the general shape of the force-time curve. Although our results correspond to a specific case of dislocation mediated plasticity, we believe that these results represent a new direction in the utilization of acoustic data. Namely, in an analogous experimental set up where there is a different dominant deformation mechanism (e.g., twinning or fracture), our methodology has a great potential to recognize and quantify those deformation events. Collecting such data of different deformation mechanisms could, then, lead to a more general model that can detect and classify AE events originating from different, coexisting deformation mechanisms based not on implicit assumptions but verified training data provided by in-situ experiments. This model would, at the same time, quantify the individual plastic events that are the sources of the AE events, which would represent a great leap in NDT methodologies both from the theoretical and technological perspective. Other future directions could be transferring our method to other experimental setups and to specimens of different materials and geometries. Testing this methodology on numerous different data sets in the future could shed light on how generalizable the method is and how universal the conclusions of the present work (e.g., the benefit from combining lower and higher order moments) are.

Methods

Experiment and data preparation

Micromechanical compression tests of single crystalline zinc micropillars oriented for basal slip were conducted at room temperature while recording both the force and the acoustic emission induced by dislocation avalanches. The micropillars had a square-shaped cross-section and an aspect ratio of height to side of 3:1. The diameter of micropillars was either 8 μm, 16 μm or 32 μm in the experiments. Each compression test was performed with a platen velocity of 10 nm/s and a spring constant of 10 mN/μm of a customized nanoindenter66. Indentation depth and load was monitored with an accuracy of ~ 1 nm and ~ 1 μN, respectively. The force was recorded with a sampling rate of 200 Hz. An acoustic emission detector of Physical Acoustics Corporation (PAC) with a wide-band (100–1000 kHz) was mechanically bonded on the bottom of the micropillar while ensuring a constant contact pressure and recorded acoustic data with a sampling rate of 2.5 MHz. It is important to note, that all the micropillars were milled into the same sample and the experiments were conducted during one measurement event, that is, the sample was not detached from the AE transducer between the tests and the same conditions/settings applied to all of the experiments. More detailed information about the experimental setup of the micropillar compression tests such as the acoustic emission detector, the amplifier and the data acquisition system, can be found in Ispánovity et al.36.

Before the extraction of features from the AE data and apply machine learning methods, the recorded AE data of each experiment was normalized by the mean of its absolute values. The AE and force data of each experiment was subdivided into equisized time windows Δt. Subsequently, various statistical frequency independent and frequency dependent features were extracted from each window of the acoustic emission data (as described in the main text) and the force increment as well as the force value was extracted from each window of the force data.

Wavelet transformation

Wavelet transformation was conducted to decompose the AE signal into a localized frequency-time domain. Formally, a wavelet transformation W(a, b) at scale a (dilatation) and position b (translation) is expressed as

$$W(a,b)=\frac{1}{w(a)}\mathop{\int}\nolimits_{-\infty }^{\infty }x(t)\cdot \psi \left(\frac{t-b}{a}\right)\,dt$$
(1)

where x(t) is the signal of time t, ψ(t) is the wavelet function and w(a) is a weighting function, which is set to \(1/\sqrt{a}\) in this work to ensure energy conservation67. In this work, we applied a continuous wavelet transformation (CWT) based on the third-order Gaussian wavelet function, which is defined as

$$\psi (t)=\frac{{\partial }^{3}}{\partial {t}^{3}}C{e}^{-{t}^{2}}=-4Ct(2{t}^{2}-3){e}^{-{t}^{2}}$$
(2)

where C is a normalization constant. This wavelet function was chosen due to its asymmetry and its shape fitting well to our detected AE signals (cp. Suppl. Fig. S4 and Suppl. Fig. S5). In this work, we employ the PyWavelets implementation (gaus3) of the third-order Gaussian wavelet68 for the CWT of the AE. Further information on wavelets, the choice of the wavelet function and the wavelet transformation is provided in Supplementary note 2.

For applying the CWT, we generated a custom set of scales a, which inversely relates to a set of frequencies f characteristic to the wavelet function. The power spectra of the AE signals of different specimen sizes are shown in Suppl. Fig. S6. Thus, we chose a based on a frequency range from 50 kHz to 800 kHz to incorporate all characteristic frequencies of dislocation avalanches.

To create a time-frequency dependent power spectrum, which leads to a so-called spectrogram, each AE time window w is transformed by the CWT. An example of an AE signal as well as the corresponding spectrograms during a regime without any detected dislocation motion and during a dislocation avalanche are presented in Supplementary note 2. After the creation of spectrograms, they are utilised to extract frequency dependent descriptors as described in the main text.

Machine learning at the fine-scale

To predict the force increment for each time window based on its extracted features, the regression model called Random Forest (RF) regressor46 was utilized with the scikit-learn python implementation. The train-test split was executed by using a set of experiments as the training data and one experiment as the test data. For the analysis of the transferability capability of the transfer learning approach (see Fig. 5), we additionally ensured that the amount of training data remains constant throughout all permutations. This was achieved by keeping the number of considered time windows nw constant (based on the shortest experiment). Based on the constant nw, only the last nw time windows for each experiment were included for training. This leads to a train:test ratio of 80%:20% for the transfer learning approach. Additionally for the transfer learning approach, to reduce the number of features of the wavelet-based approach, only the three most important frequencies 100 kHz, 250 kHz and 500 kHz were included. Hyperparameters of the frequency independent and the frequency dependent approaches were evaluated based on a grid search five-fold cross-validation and were frozen after finding the best set. For measuring the accuracy of the regression model, we applied the coefficient of determination R2 to the predicted and the actual force increments. This measure is defined as

$${R}^{2}({\bf{y}},{{\bf{y}}}^{({\bf{p}})})=1-\frac{\mathop{\sum }\nolimits_{i = 1}^{N}{\left({y}_{i}-{y}_{i}^{({\rm{p}})}\right)}^{2}}{\mathop{\sum }\nolimits_{i = 1}^{N}{\left({y}_{i}-\overline{y}\right)}^{2}}$$
(3)

where y and y(p) are the vectors of the ground truth and predicted values of the target variable (in our case, the force increment), respectively, N denotes the number of data point in the test set and \(\overline{y}=\frac{1}{N}\mathop{\sum }\nolimits_{i = 1}^{N}{y}_{i}\).

Machine learning at the coarse-scale

At the coarse-scale the force value is predicted at certain times instead of force increments. For this prediction, additional features in combination with the aforementioned frequency independent features (i.e., moments of the AE timeseries) used at the fine-scale are utilized. Firstly, the number of AE signals is counted (based on thresholding), and secondly, the total duration of these AE signals is detected, i.e., features representing the number and the length of dislocation avalanches occurring in a coarse time window are used. To ensure that the size effect is captured, the pillar diameter is added as a feature at the coarse-scale. The window width chosen is ΔT = 50 s. Due to the large window width overlapping windows are used for training to avoid insufficient amount of training data. Adjacent windows are shifted from each other by ΔT/10 = 5 s. To predict the shape of the force-time curve of a certain experiment at the coarse-scale all these coarse-scale windows of all other experiments are utilized to train a random forest regressor model. An example of the final coarse-scale prediction is shown in Fig. 3b capturing the overall shape of the force-time curve.

Combination of fine- and coarse-scale prediction

In order to keep the fine-scale force drop structure while obtaining force-time curve that has approximately correct coarse-scale shape, the fine- and coarse-scale predictions are combined as follows. Corrections are made to the fine-scale results in a piecewise manner, that is, independently in each disjoint coarse window. Let F denote the coarse-scale force prediction and f the force prediction obtained by summing up the force increments originating from the fine-scale prediction. Let us now consider the nth coarse window and denote the corresponding time interval with \({{\mathcal{T}}}_{n}=[(n-1)\Delta T,n\Delta T]\). At the end of this window the two predictions provide F((n − 1)ΔT) and f((n − 1)ΔT). Let Δn denote their difference:

$${\Delta }_{n}=F((n-1)\Delta T)-f((n-1)\Delta T).$$
(4)

By adding Δn to the original fine-scale curve (only within the interval \({{\mathcal{T}}}_{n}\)) we get the modified function

$${f}_{n}^{* }(t)=f(t)+{\Delta }_{n},\quad (t\in {{\mathcal{T}}}_{n})$$
(5)

for which \({f}_{n}^{* }((n-1)\Delta T)=F((n-1)\Delta T)\), however, generally \({f}_{n}^{* }(n\Delta T)\ne F(n\Delta T)\). Then,

$${\delta }_{n}=\frac{F(n\Delta T)-{f}_{n}^{* }(n\Delta T)}{\Delta T}=\frac{F(n\Delta T)-f(n\Delta T)-{\Delta }_{n}}{\Delta T}$$
(6)

is computed. Our final prediction within the coarse window is obtained as

$$\begin{array}{l}{f}_{n}^{* * }(t)={f}_{n}^{* }(t)+{\delta }_{n}[t-(n-1)\Delta T]=f(t)+{\Delta }_{n}\\\qquad\quad\;\;\;+\,{\delta }_{n}[t-(n-1)\Delta T],\quad (t\in {{\mathcal{T}}}_{n}),\end{array}$$
(7)

which guarantees that \({f}_{n}^{* * }((n-1)\Delta T)=F((n-1)\Delta T)\) and \({f}_{n}^{* * }(n\Delta T)=F(n\Delta T)\). This is performed for all coarse time windows (all values for n).

This method ensures that the predicted stress–time curve goes through the (t,σ) points provided by the coarse-scale prediction, which is achieved by adding a constant value to the force increments obtained by the fine-scale prediction. This values is constant within a coarse time windows but can vary window to window. Since this procedure only adds a constant slope to the curve within a coarse time window, the curve is adjusted to follow the coarse-scale prediction while the relative differences between the increments on the fine-scale are kept unchanged within a coarse window. Consequently, the method leaves the fine structures revealed by the fine-scale prediction unaffected.