Accurate and uncertainty-aware multi-task prediction of HEA properties using prior-guided deep Gaussian processes

Abstract

Surrogate modeling techniques have become indispensable in accelerating the discovery and optimization of high-entropy alloys (HEAs), especially when integrating computational predictions with sparse experimental observations. This study systematically evaluates the training and testing performance of four prominent surrogate models—conventional Gaussian processes (cGP), Deep Gaussian processes (DGP), encoder-decoder neural networks for multi-output regression and eXtreme Gradient Boosting (XGBoost)—applied to a hybrid dataset of experimental and computational properties of the 8-component HEA system Al-Co-Cr-Cu-Fe-Mn-Ni-V. We specifically assess their capabilities in predicting correlated material properties, including yield strength, hardness, modulus, ultimate tensile strength, elongation, and average hardness under dynamic/quasi-static conditions, alongside auxiliary computational properties. The comparison highlights the strengths of hierarchical deep modeling approaches in handling heteroscedastic, heterotopic, and incomplete data commonly encountered in materials science. Our findings illustrate that combined surrogate models such as DGPs infused with machine-learned priors outperform other surrogates by effectively capturing inter-property correlations and by assimilating prior knowledge. This enhanced predictive accuracy positions the combined surrogate models as powerful tools for robust and data-efficient materials design.

Introduction

High-entropy alloys (HEAs) are multicomponent alloys with five or more principal elements in high concentrations that offer a vast compositional design space and often exhibit exceptional mechanical properties^1,2. The exploration of this space is challenged by limited experimental data and complex composition-property relationships^1,3. Accordingly, data-driven surrogate models have become invaluable in accelerating HEA discovery by predicting material properties^3,4,5,6.

Surrogate models serve as computationally efficient approximations of complex structure-property relationships and are widely used to accelerate materials discovery. Gaussian processes (GPs) provide calibrated uncertainty estimates, making them effective for sparse, high-fidelity datasets⁷, while multi-task extensions and co-kriging allow for correlated output modeling⁸. Deep Gaussian Processes (DGPs) extend this capability to hierarchical, nonlinear functions⁹, offering a potential advantage in capturing complex material behavior. In contrast, tree-based methods like eXtreme Gradient Boosting (XGBoost)¹⁰ or encoder-decoder mapping of inputs to outputs¹¹ are often easier to scale and tune but lack native uncertainty quantification unless modified. As a result, model selection should be guided by data availability, dimensionality, and the need for uncertainty modeling in the optimization loop. Hybrid modeling strategies that combine the strengths of neural networks (for representation learning) and probabilistic models (for uncertainty quantification) offer a promising path forward–leveraging expressive power while retaining decision-making confidence in sparse-data regimes.

Recent studies have combined machine learning with high-throughput calculations and experiments to identify novel alloys with targeted properties¹². In particular, Bayesian optimization (BO) frameworks for materials design rely on accurate surrogate models to guide experiments toward optimal candidates¹³. In our previous work, we addressed the discovery of HEAs with specific properties by using advanced GP models (conventional Gaussian process (cGP), multi-task Gaussian process (MTGP), and DGP) within a BO loop¹⁴. We also explored DGP surrogates in that context, which—along with MTGPs—were able to capture the coupled trends in bulk modulus and coefficient of thermal expansion, while also accelerating the search for promising alloy compositions in Fe-Cr-Ni-Co-Cu HEA space. In particular, Khatamsaz et al.¹⁵ employed a MTGP approach to jointly model the yield strength (YS), the Pugh ratio and the Cauchy pressure in a simulated Mo-Ti-Nb-V-W alloy system, enabling efficient multi-objective optimization for high strength and ductility. Building on the successes of these optimization-focused studies, we now shift our attention toward a detailed evaluation of surrogate modeling techniques. Moving beyond optimization, the present work assesses how well different models fit a hybrid data set consisting of both experimental measurements and computational predictions. In particular, we compare multiple modeling approaches, conventional single-layer cGPs, DGPs, a custom encoder-decoder neural network for multi-output regression, and XGBoost, on their ability to learn the composition-property relationships in the Al-Co-Cr-Cu-Fe-Mn-Ni-V HEA system. DGPs are a hierarchical extension of GPs that can capture complex, nonlinear mappings by composing multiple GP layers⁹, and we hypothesize that this added depth enables DGPs to model heteroscedastic and nonstationary behavior often observed in materials data better than a standard cGP. The encoder-decoder model, on the other hand, represents a deterministic deep learning approach: it “encodes” alloy compositions into a learned feature representation and “decodes” this latent vector to simultaneously predict multiple properties¹⁶.

As a baseline, we include a conventional XGBoost model for each property to illustrate the limitations of ignoring inter-property correlations and data uncertainty. The material data set used in this study spans the Al-Co-Cr-Cu-Fe-Mn-Ni-V composition space, an 8-element HEA system of interest for its high configurational entropy and the potential for exceptional mechanical performance. This data set is assembled from disparate sources, having experimentally calculated properties as “main tasks” and some additional computationally estimated properties as “auxiliary tasks”. The inclusion of these additional “tasks” provides additional information that can be used by multiple output models to improve predictions of the main experimental properties, analogous to multifidelity or multisource learning in other contexts^17,18.

Given the diverse origins of this dataset, not every alloy composition has a complete set of measurements; some samples have missing experimental values, while corresponding computational predictions may exist, and vice versa. This reflects a common situation in materials science, where gathering a complete set of property data for every sample is infeasible, and one must utilize incomplete and noisy information from various sources. Consequently, the challenges posed by this HEA dataset underscore the need for surrogate models capable of handling correlated output, heteroscedastic uncertainties, and missing data. Moreover, many of the mechanical properties in HEAs are interdependent—for example, hardness and YS are often correlated since they both relate to underlying strengthening mechanisms. A multi-output model can exploit such correlations to improve the prediction accuracy for each task, especially when one property has abundant data and another is data sparse^8,16.

In addition to the challenges of missing data, noise levels, and predictive difficulties can vary between properties. Experimental measurements like elongation may have higher uncertainty or variability than, say, valence electron concentration (VEC, which is deterministically computed), leading to heteroscedastic behavior. The sources and treatment of uncertainty differ across the various properties considered in this work. For experimentally measured properties–such as hardness, modulus, or elongation–uncertainty is quantified using the standard deviation across three replicate measurements per alloy, capturing variability due to factors like local microstructural differences and measurement noise. These properties can exhibit heteroscedasticity across the dataset as their variance may depend on composition or processing conditions. In contrast, some descriptors used in this study are computed rather than experimentally measured, and have different uncertainty profiles. For example, VEC is computed deterministically using a rule-of-mixtures approach, scaled by a factor of 100 prior to modeling for numerical stability, and currently does not include an associated uncertainty estimate in our framework. However, we recognize that VEC, like any empirical descriptor, has model-form uncertainty based on its simplifying assumptions. Similarly, stacking fault energy (SFE) values used in this study were generated using a machine learning model trained on density functional theory (DFT) data, as described by Khan et al.¹⁹, while YS estimates were computed using the Curtin–Varvenne solid solution strengthening model²⁰. Both of these computed descriptors inherit uncertainty from their underlying models and training data, although explicit uncertainty quantification was not included. To support future use and analysis, we plan to release uncertainty estimates alongside the dataset–including statistical uncertainties for experimental data and model-based estimates for computed properties–to enable more robust uncertainty-aware modeling and optimization across the HEA design space. Traditional single-task cGPs can struggle in this setting because they assume a single noise variance per task and cannot borrow strength from related tasks. In contrast, co-regionalized DGP frameworks can naturally handle task-specific noise and allow information transfer across properties²¹. Likewise, DGPs can effectively model input-dependent noise by virtue of their layered construction—the output of one GP layer (with its own variance) feeds into the next, allowing the model to represent variability that changes with composition²². Finally, neural networks can also accommodate heteroscedasticity by learning complex nonlinear interactions, though they typically require larger datasets to generalize well and often treat noise implicitly (e.g., through regularization techniques). Importantly, both the DGP and the encoder-decoder model can handle missing outputs during training, as they are trained with a loss (or likelihood) that includes only observed data for each task, enabling the use of all available partial information.

Despite the advances in surrogate modeling and optimization for HEAs, a notable gap remains: while numerous studies have demonstrated the successful application of individual surrogate models for property prediction and optimization, a robust, non-parametric, multi-property model being able to predict uncertainty as well on a unified, multi-property alloy dataset is lacking^3,4,5,6. In this paper, our aim is to fill this gap by systematically benchmarking DGPs, DGPs with encoder-decoder prior, encoder-decoder multi-output architectures, cGPs, and XGBoost on the BIRDSHOT dataset. Through this comparison, we evaluate the ability of each method to fit the complex property landscape of HEAs and faithfully capture property correlations and uncertainties. The insights from this study provide guidance on selecting surrogate models for future materials informatics efforts, especially in scenarios with sparse, noisy, and multisource data. We show that appropriately accounting for task correlations and data heterogeneity—via advanced models like DGPs—can markedly improve predictive performance, thereby enabling more reliable downstream tasks such as Bayesian alloy design and multiobjective optimization.

Results

BIRDSHOT data

The BIRDSHOT dataset is a comprehensive, high-fidelity collection of mechanical, structural, and compositional data covering over 100 distinct high-entropy alloy (HEA) compositions in the Al-Co-Cr-Cu-Fe-Mn-Ni-V system^23,24—see Fig. 1. It is the product of two complementary experimental campaigns—the first campaign has already been thoroughly described in ref. ²³—each targeting slightly different regions of the compositional space and emphasizing different mechanical performance metrics. The first campaign focused on a six-element FCC HEA system (Al-Co-Cr-Fe-Ni-V), while the second campaign expanded the design space to include Mn and Cu. Together, these efforts explored a filtered design space consisting of tens of thousands of feasible alloys, ultimately selecting and synthesizing more than 100 unique compositions. These alloys were chosen to span a broad and representative portion of the chemically complex HEA landscape, under multiple design constraints related to phase stability, thermal, and mechanical properties.

Fig. 1: Schematic overview of the BIRDSHOT data acquisition workflow.

The dataset results from a multi-institutional effort combining alloy synthesis, mechanical testing across various strain rates (including nanoindentation, strain rate jump tests, small punch tests, and SHPB), and structural and compositional verification (EBSD, XRD, SEM-EDS). This integrated workflow ensures fidelity and consistency across the high-dimensional alloy space explored.

Alloys in the dataset were synthesized using vacuum arc melting (VAM) under strictly controlled and standardized protocols²³. Each 30–35 g ingot was produced from high-purity elements, flipped and remelted multiple times to ensure chemical homogeneity, and subjected to homogenization followed by mechanical working (hot forging or cold rolling and recrystallization). These protocols were applied uniformly across both campaigns, ensuring consistency in processing history. Compositional accuracy was confirmed via SEM-EDS with average deviations from targets below 1%, while structural phase verification was carried out using XRD and EBSD. This uniformity enables direct comparison of properties across compositions without the confounding effects of processing variability.

Each alloy underwent a suite of structural and mechanical characterizations, with properties measured from subsamples taken from the same physical ingot. This eliminates ambiguity often introduced when comparing properties measured on nominally identical but independently prepared samples. As a result, the dataset provides true cross-property, multimodal measurements–such as YS, ultimate tensile strength (UTS), hardness, strain-rate sensitivity, and modulus–directly coupled to each alloy’s verified composition and microstructure.

Mechanical behavior was characterized across a wide range of strain rates. Miniaturized tensile tests were conducted to extract yield and ultimate strengths and strain hardening ratios. High-strain-rate nanoindentation, including strain rate jump tests, was used to determine rate-sensitive hardness and modulus. Dynamic performance was further evaluated via small punch tests, split Hopkinson pressure bar (SHPB) compression testing, and Laser-Induced Projectile Impact Testing (LIPIT), enabling mechanical assessment from 10⁻⁴ to 10⁷ s⁻¹. Microstructural analyses using SEM, EBSD, and XRD verified single-phase FCC stability and ruled out the presence of deleterious secondary phases.

A key innovation of the BIRDSHOT effort lies in its use of a Bayesian discovery framework²⁵ to guide experimental exploration. This closed-loop approach integrates machine learning, physical modeling, and experimental data to iteratively select the most informative alloy compositions. At each iteration, a surrogate model–typically a cGP–is trained on prior experimental data to predict material properties across the compositional space, along with associated uncertainty. An acquisition function, such as Expected Hypervolume Improvement (EHVI), is used to rank candidate alloys by their potential to improve the Pareto front defined by multiple objectives. A batch of promising compositions is selected, synthesized, tested, and then fed back into the model for the next iteration. This strategy balances exploration of under-sampled regions with exploitation of high-performing areas and led to the discovery of a robust multi-objective Pareto set while sampling less than 0.2% of the filtered design space²³.

The two campaigns established a rigorously curated, multi-modal, and multi-objective dataset that not only provides broad coverage of a chemically rich compositional space, but also captures cross-property correlations at an unprecedented level of fidelity. This makes the BIRDSHOT dataset a valuable resource for both mechanistic understanding and data-driven modeling of complex alloy systems—in this specific case, the FCC HEA space.

In addition to experimental data, the dataset includes a suite of simulation-derived descriptors computed for each alloy composition. VEC was calculated using a rule-of-mixtures approximation, based on atomic fractions of constituent elements, and subsequently scaled by a factor of 100 prior to model training for numerical stability. YS estimates were obtained using the Curtin–Varvenne model²⁰, which computes solid solution strengthening based on atomic size and modulus mismatch among elements. SFE values were predicted using a machine learning model trained on DFT data, developed by Khan et al. ¹⁹, providing composition-sensitive estimates of SFE across the HEA design space.

Depth of penetration, representing a measure of ballistic resistance, was predicted through finite element simulations using ABAQUS. These simulations modeled the impact of a spherical alumina projectile onto a cylindrical HEA target traveling at 500 m/s. Each alloy was modeled using a Cowper-Symonds viscoplasticity framework, calibrated with experimental stress-strain data from quasi-static tensile and SHPB experiments. While descriptors such as VEC, SFE, and predicted YS are available for all alloy compositions in the dataset, depth of penetration is currently available for a subset of alloys tested for SHPB tests.

Effect of elemental composition on experimental properties

The BIRDSHOT dataset²⁴ currently contains detailed information on 147 non-equimolar Cantor HEAs, focusing on their composition, processing parameters, and mechanical properties. Each alloy is characterized by its elemental fractions (Al, Co, Cr, Cu, Fe, Mn, Ni, V) and evaluated for key properties such as YS, UTS, tension elongation, and hardness. The dataset also includes computationally derived parameters such as SFE, VEC, and Varvenne YS providing insights into the alloys’ mechanical behavior and stability. For example, the YS of the alloys ranges from 310 to 537 MPa, while tension elongation varies from 18.3% to 25.7%. First-order statistics for this dataset is summarized in Table 1.

Table 1 Birdshot dataset: first-order statistical summary including skewness and kurtosis for experimental data

The dataset contains 22 features, capturing both experimental and computational results, and is suitable for developing predictive models, investigating structure-property relationships, and advancing the discovery of novel high-performance alloys. Despite its comprehensive nature, the dataset includes minor missing data across key properties. Measured properties like YS and UTS have 141 complete entries (~96%), while computed parameters such as SFE and VEC; and nano-indentation hardness are available for 131 samples (~89%). Feature complexity was calculated as the sum of the absolute skewness and the absolute excess kurtosis (∣skewness∣ + ∣kurtosis − 3∣) for each feature.

The experimental data set contains data points scattered throughout the composition space. The selection strategy of the data points is based on BO of two campaigns, as mentioned in the methods section. The available data points are described in the above-mentioned Table 1.

To further investigate correlations among properties, we report pairwise plots in Fig. 2. Our goal is to demonstrate that considering correlation among the properties yields a better fit for the surrogates. The pairplot reveals strong positive correlations between YS, hardness, and UTS, emphasizing their interdependent nature. Ductility (Elong_T) shows a clear inverse relationship with these strength-related properties, highlighting the common strength-ductility trade-off observed in HEAs¹.

Fig. 2: Pairwise plot illustrating correlations between different experimental properties.

This plot shows pairwise relations between yield strength, ultimate tensile strength, ratio of ultimate tensile strength and yield strength, elongation, hardness, modulus and ratio of hardness under dynamic and quasi-static conditions.

These visual analyses establish the foundational understanding of the underlying correlations and elemental influences critical to interpreting the surrogate modeling performance discussed subsequently.

Comparison of model performances

In this study, we systematically evaluated seven distinct surrogate modeling approaches to accurately predict material properties within the Al-Co-Cr-Cu-Fe-Mn-Ni-V HEA space Fig. 3. The methods assessed include: (i) DGP incorporating priors trained on all tasks (ii) DGP incorporating priors trained on only main objectives, (iii) DGP without priors utilizing all tasks, (iv) DGP without priors trained solely on main objectives, (v) cGP with no correlated kernel, (vi) XGBoost and (vii) encoder-decoder model used for regression tasks¹¹.

Fig. 3: Schematic representation of the two-layered variational deep Gaussian process (DGP) architecture with encoder-decoder based prior injection.

Each variational multi-task Gaussian process (MTGP) layer reduces dimensionality from input to latent representation, governed by the reduction parameter.

A unique aspect of our modeling approach involves incorporating prior knowledge derived from a previously trained encoder-decoder neural regression model. We fixed the train-test split between the two models (DGP and encoder-decoder) for the cases where DGPs use encoder-decoder as prior. This prevents data leakage, ensuring that our test set is totally unseen by the DGP models during training.

This prior was injected into the DGP models by subtracting predicted prior values from the training data outputs, thereby focusing the DGP on modeling residuals, i.e., delta learning. After the prediction phase, the priors were re-added to generate the final predictions, which were then visualized and evaluated. This technique leverages existing domain knowledge and has the potential to significantly enhance predictive performance, especially in sparse and noisy data environments^16,26.

To comprehensively evaluate model performance, we utilized multiple standard regression metrics, including the coefficient of determination (R²), root mean squared error (RMSE), symmetric mean absolute percentage error (SMAPE), and mean averaged error (MAE). Additionally, Spearman’s rank correlation coefficient was computed to assess the consistency of predicted rankings relative to actual data, providing insights into the models’ abilities to preserve order, which is critical for practical materials selection scenarios.

Table 2 final reported metrics were obtained by averaging results from five different randomized train-test splits to ensure robust and reliable performance evaluation. Specifically, an 80-20 train-test split was employed for the DGP, GP, XGBoost models, and encoder-decoder model. For the DGP models, the optimal reduction parameter identified during the hyperparameter tuning stage was consistently applied across these evaluations.The values of the metrics reported in Table 3 contain the mean and the standard deviation over the five train-test splits in the form of “mean ± standard deviation”.

Table 2 Summary of surrogate modeling strategies evaluated in this study

Table 3 Test performance metrics

In the following section, we will provide detailed comparisons and discussions of the performance metrics across the seven evaluated surrogate models, elucidating the strengths, limitations, and practical implications of each approach for multi-task prediction in HEAy datasets.

The comparative analysis across surrogate models, as detailed in Table 3, provides valuable insights into the efficacy and suitability of different modeling strategies within HEA surrogate modeling. Among the evaluated models–XGBoost, cGP, DGP variations (HDGP NP-All, HDGP NP-Main, HDGP P-All, HDGP P-Main), and the encoder-decoder neural network–the HDGP models incorporating prior knowledge (HDGP P-Main, HDGP P-All) consistently exhibited superior performance across the majority of tasks of primary experimental interest.

Specifically, the HDGP-P-Main and HDGP-P-All models showed marked improvements in prediction accuracy, reflected by the lowest RMSE and SMAPE values, alongside the highest R² and Spearman rank correlation coefficients for critical experimental properties such as YS, UTS, elongation, hardness, and average hardness under dynamic/quasi-static conditions. Both of these models have mostly similar performance. For depth of penetration, modulus, hardness, elongation, YS, and SFE, HDGP-P-All performed best. For UTS/YS, UTS and avg HDYN/HQS HDGP-P-Main performed better. We note that HDGP-P-All had encoder-decoder priors for all tasks except Varveillien YS, SFE, and VEC. The HDGP-P-All surpassed the performance of HDGP-P-Main in most of the tasks and it can possibly be attributed to the addition of auxiliary tasks. Auxiliary task may help the DGP generalize better by giving information about the correlation between tasks¹⁴. The exceptional performance of HDGP P-All and HDGP-P-Main over non-prior models can be primarily attributed to the integration of encoder-decoder model as informative priors in the training process¹¹. Any GP-based model without prior defaults to constant mean in unseen region. The addition of prior makes the model converge to the prior at a specific input compositions rather than one constant value at all compositions²⁶. This makes the predictions more robust for out-of-distribution points. It also helps in leveraging expert knowledge or other parametric ML models, like encoder-decoder in our case. This is also illustrated in Fig. 4 for the case of hardness in HDGP-NP-All and HDGP-P-All. For points inside the red circle, in the model with prior(b), the predictions don’t default to a constant value in far-away regions. Thus, they are close to the actual values. Whereas in the model without prior(a), the mean tend to move toward a constant value, making the predictions defer from the actual values.

Fig. 4: The figure illustrates difference in fitting performance for hardness between two DGP-all models.

a Without prior injection, the points within the red circle are further from the actual value and b With prior injection, the points within the red circle tend to be closer to the actual value.

Interestingly, the performance of prior-based models was followed by encoder-decoder, HDGP-NP-Main, and HDGP-NP-All, having intermediate performance. They performed better than XGBoost and cGP but performed worse than DGP models having priors. In between these two DGPs, HDGP-NP-All was the better one for most of the tasks. It had Varveillien YS, SFE, and VEC as auxiliary tasks. Due to the hierarchical and deep structure of DGPs, the HDGP model effectively leveraged these auxiliary tasks, exploiting inter-task correlations to enhance predictive accuracy for main tasks compared to HDGP-NP-Main. This multi-task learning capability, intrinsic to DGP frameworks, allows the model to extract valuable latent information from auxiliary properties, thus significantly improving prediction robustness in sparse and noisy experimental settings^9,22. Encoder-decoder model performed better than HDGP-NP-Main and HDGP-NP-All, but it does not have the capacity of uncertainty quantification, so may not be useful in materials design or discovery campaigns on its own. Moreover, the encoder-decoder neural network model, while theoretically capable of capturing complex nonlinearities, exhibited poorer generalization performance compared to DGPs with prior. This underperformance likely stems from its deterministic and parametric nature, sensitivity to hyperparameter settings, and susceptibility to overfitting when trained on smaller datasets¹⁶. Also, the encoder-decoder model had much larger number parameters, making it prone to overfitting for a small dataset like ours.

Contrastingly, the cGP and XGBoost models demonstrated several limitations despite their frequent use in regression tasks. Although XGBoost delivered competitive performance in specific auxiliary tasks (e.g., VarvYS), it notably lacked the flexibility and uncertainty quantification capabilities inherent to GP-based methods. XGBoost, being a parametric, tree-based ensemble model, can easily overfit limited datasets, reducing its generalizability in complex materials design problems¹⁰. Similarly, cGP, despite their nonparametric Bayesian framework and robust uncertainty quantification, failed to match the performance of HDGP variants due to their inability to explicitly model inter-task correlations and input-dependent uncertainty through hierarchical structures. cGP assumes independence among outputs, limiting its ability to transfer knowledge effectively across related properties⁷.

This trend in performance is also visible in train set metrics as attached in the Supplementary File. Although XGBoost had the best training metric mostly, it was very closely followed by HDGP-P-All and HDGP-P-Main. These two DGP models performed better than cGP and encoder-decoder model in fitting the train data as well, affirming their superior generalization capability.

It is important to contextualize our modeling approach within the framework of BO for materials discovery, as this fundamentally differs from traditional machine learning paradigms that emphasize broad generalization. The 147-composition BIRDSHOT dataset, while appearing modest in size, is entirely typical for BO-driven materials discovery campaigns where each experimental data point represents substantial synthesis and characterization investment^27,28. In BO applications, surrogate models serve not as standalone predictive tools but as acquisition function components that guide sequential experimental design to efficiently navigate complex design spaces²⁹. For this purpose, the model’s ability to correctly rank candidate materials by their predicted performance and associated uncertainty–quantified through Spearman rank correlation–is far more critical than absolute predictive accuracy³⁰. Our results demonstrate Spearman correlations consistently above 0.8-0.9 for critical properties, indicating excellent ranking performance suitable for guiding experimental campaigns. Similar BO studies in materials science routinely operate with comparable dataset sizes, as the methodology explicitly prioritizes efficient exploration over comprehensive coverage^31,32. Rather than claiming broad extrapolative generalization across the entire 8-component HEA space, our approach demonstrates effective interpolative prediction within the explored compositional region, combined with well-calibrated uncertainty estimates that naturally increase for compositions distant from training data. This represents the standard operating paradigm for automated experimentation and self-driving laboratory applications, where datasets of 100-200 samples are typical³³.

To address the interpretability of our best-performing HDGP-P-All model and understand the underlying relationships between elemental composition and material properties, we conducted a comprehensive feature importance analysis using SHapley Additive exPlanations (SHAP)³⁴. SHAP values provide a unified framework for interpreting machine learning model predictions by quantifying the contribution of each input feature to individual predictions, thereby offering insights into the model’s decision-making process. Figure 5 presents SHAP summary plots for all predicted properties in our HEA system, where each point represents a single prediction, with the x-axis showing the SHAP value (impact on model output) and the color indicating the feature value magnitude. Positive SHAP values increase the predicted property value, while negative values decrease it. We used a model-agnostic KernelSHAP estimator on the trained HDGP-P-ALL posterior for each target,

Fig. 5: SHAP summary plots for all predicted properties using the HDGP-P-ALL model.

The figures illustrate SHAP values for (a) varveillian yield strength, (b) SFE, (c) average VEC, (d) YS, (e) UTS, (f) UTS/YS ratio, (g) elongation, (h) hardness, (i) modulus, (j) average HDYN/HQS and (k) depth of penetration. Each dot represents a single prediction, with the x-axis showing the SHAP value (impact on model output) and color indicating feature value magnitude (red = high, blue = low). Features are ordered by importance (sum of absolute SHAP values).

Across nearly all mechanical targets, Al and V exhibit the largest absolute SHAP magnitudes, indicating they are primary drivers of model outputs. Cr and Cu frequently contribute negatively to strength-related metrics but can aid ductility-related ones (e.g., Elongation). UTS/YS is negatively influenced by higher Al and V, but positively by Fe. Ni, Co, and Mn play secondary yet property-dependent roles, often modulating predictions in narrower bands. Collectively, these patterns align with metallurgical intuition: strong solid-solution strengtheners (e.g., V, Al) raise yield/ultimate strengths, whereas elements that reduce SFE (e.g., Cu, Cr) can trade off strength for ductility and hardness. These SHAP analyses (i) validate that HDGP-P-ALL captures physically plausible element-property relationships and (ii) offer actionable guidance for alloy design (e.g., increasing V/Al to boost strength while monitoring Cr/Cu levels to balance ductility).

In summary, HDGP P-All emerges as the optimal modeling choice for predicting correlated material properties within HEA datasets, efficiently addressing common challenges of sparsity, noise, and missing data. Its hierarchical, Bayesian nature ensures robust uncertainty quantification and superior predictive performance, making it particularly suitable for guiding experimental design in materials informatics. It utilizes auxiliary tasks to generalize better. Also, it leverages the knowledge of encoder-decoder model as prior to perform better, providing promise as a hybrid model capable of uncertainty estimation and better generalization. Moreover, Fig. 6 illustrates the parity plots and relevant test metrics for one specific train-test split in the case of HDGP-P-All model, showing excellent accuracy. It is to be noted that the error bars in the plots are representative of the standard deviation as predicted by the model. The parity plots for all the other models and tables of train metrics are provided in the Supplementary File.

Fig. 6: HDGP-P-All model results.

a Predicted VarvYS at 298 K; b SFE calculation; c Average VEC; d Yield strength (MPa); e True UTS (MPa); f UTS/YS ratio; g Elongation; h Hardness (GPa); i Modulus (GPa); j Average HDYN HQS; k Depth of penetration (mm).

Discussion

In this work, we systematically evaluated several surrogate modeling techniques, including cGP, DGP, encoder-decoder neural networks, and XGBoost, to predict multiple correlated material properties in the complex Al-Co-Cr-Cu-Fe-Mn-Ni-V HEA system. Our analysis demonstrated that the hierarchical DGP model, which incorporates informative priors and is trained across both main and auxiliary tasks (HDGP P-All), consistently outperformed other models in key mechanical properties, such as YS, UTS, elongation, hardness, modulus, and dynamic/quasistatic hardness ratios.

The superior performance of HDGP P-All can be primarily attributed to its hierarchical architecture and effective leverage of inter-task correlations, as well as the strategic integration of prior knowledge derived from a pre-trained encoder-decoder model. This combination enabled the model to robustly address common challenges inherent in materials informatics datasets, including heteroscedastic uncertainties, heterotopic observations, and missing data. In contrast, cGP, XGBoost, and deterministic encoder-decoder networks exhibited notable limitations, primarily due to their inability to explicitly quantify uncertainties, susceptibility to overfitting, and limited capacity for modeling correlations among multiple outputs.

In general, our findings strongly advocate the adoption of advanced hierarchical surrogate models with uncertainty-awareness, such as DGP in materials discovery and optimization campaigns. These models provide substantial advantages by effectively capturing complex property interdependencies, robustly managing uncertainty, and maximizing predictive accuracy even with sparse and heterogeneous data sets. Future research should further optimize the integration of domain-specific prior knowledge and enhance computational efficiency to facilitate broader applications of these advanced methods in accelerated materials development.

Methods

Conventional Gaussian processes (cGP)

cGPs provide a flexible Bayesian framework widely used for regression tasks due to their ability to explicitly quantify predictive uncertainty⁷. Given a set of observations ${\mathcal{D}}=\{{{\bf{X}}}_{N},{{\bf{y}}}_{N}\}$, where X_N = (x₁, x₂, …, x_N) are the input points and y_N = (y₁, y₂, …, y_N) are their corresponding outputs, a GP is fully defined by its mean function μ(x) and the covariance function $k({\bf{x}},{{\bf{x}}}^{{\prime} })$. For an unseen input point x^*, the GP posterior prediction is given by:

$$p(f({{\bf{x}}}^{* })| {{\bf{X}}}_{N},{{\bf{y}}}_{N})={\mathcal{N}}(\mu ({{\bf{x}}}^{* }),{\sigma }^{2}({{\bf{x}}}^{* }))$$

(1)

with predictive mean and variance:

$$\mu ({{\bf{x}}}^{* })={\bf{k}}{({{\bf{x}}}^{* },{{\bf{X}}}_{N})}^{\top }{\left({\bf{K}}({{\bf{X}}}_{N},{{\bf{X}}}_{N})+{\sigma }_{n}^{2}{\bf{I}}\right)}^{-1}{{\bf{y}}}_{N},$$

(2)

$${\sigma }^{2}({{\bf{x}}}^{* })=k({{\bf{x}}}^{* },{{\bf{x}}}^{* })-{\bf{k}}{({{\bf{x}}}^{* },{{\bf{X}}}_{N})}^{\top }{\left({\bf{K}}({{\bf{X}}}_{N},{{\bf{X}}}_{N})+{\sigma }_{n}^{2}{\bf{I}}\right)}^{-1}{\bf{k}}({{\bf{x}}}^{* },{{\bf{X}}}_{N})$$

(3)

Here, K(X_N, X_N) denotes the covariance matrix computed from the kernel k for the training inputs, k(x^*, X_N) is the covariance vector between the test point x^* and the training inputs, and ${\sigma }_{n}^{2}$ represents the observation noise variance. A commonly adopted kernel is the squared exponential (RBF) kernel:

$$k({\bf{x}},{{\bf{x}}}^{{\prime} })=\exp \left(-\frac{| | {\bf{x}}-{{\bf{x}}}^{{\prime} }| {| }^{2}}{2{\ell }^{2}}\right)$$

(4)

which characterizes the similarity between input points based on their Euclidean distance, with ℓ controlling the length scale. This description of conventional GPs lays the foundational groundwork for understanding more complex modeling approaches.

Deep Gaussian processes (DGP)

Building directly on the fundamental concepts introduced above, DGPs extend conventional GPs by incorporating a hierarchical structure. This hierarchical arrangement enables the modeling of more intricate, non-linear relationships and input-dependent uncertainties, thereby addressing challenges that arise in complex datasets^9,22. In a DGP with L hidden layers, the generative model is described by:

$${{\bf{y}}}_{n}={f}^{(L)}({{\bf{h}}}_{n}^{(L-1)})+{\epsilon }_{n}^{(L)},\quad {\epsilon }_{n}^{(L)} \sim {\mathcal{N}}(0,{\sigma }_{L}^{2}I),$$

(5)

$${{\bf{h}}}_{n}^{(L-1)}={f}^{(L-1)}({{\bf{h}}}_{n}^{(L-2)})+{\epsilon }_{n}^{(L-1)},\quad {\epsilon }_{n}^{(L-1)} \sim {\mathcal{N}}(0,{\sigma }_{L-1}^{2}I),$$

(6)

$$\vdots$$

(7)

$${{\bf{h}}}_{n}^{(1)}={f}^{(1)}({{\bf{x}}}_{n})+{\epsilon }_{n}^{(1)},\quad {\epsilon }_{n}^{(1)} \sim {\mathcal{N}}(0,{\sigma }_{1}^{2}I)$$

(8)

Each function f⁽ⁱ⁾ is modeled as a GP with its own covariance structure, effectively capturing different levels of abstraction from the input data. Given the increased computational complexity of DGPs, training often involves variational inference techniques. Specifically, the model is optimized by maximizing the Evidence Lower Bound (ELBO):

$${\mathcal{L}}=\,\int\,Q\log \frac{p({\bf{Y}},{\bf{H}},{\bf{F}})}{Q},$$

(9)

where Q represents the variational distribution that approximates the true posterior. Gradient-based optimization methods are typically employed to balance model fit and complexity^22,35. This progression from conventional to deep GP structures demonstrates the evolution from simple probabilistic modeling to a framework capable of handling greater nonlinearity and uncertainty.

Isotopic vs. Heterotopic data

Continuing from the dataset description, we now address a key data structuring concern relevant to the modeling techniques presented earlier. In multi-output GP modeling, distinguishing between isotopic and heterotopic data is essential due to its implications on model performance and uncertainty quantification²¹.

Isotopic data, where all tasks are observed at the same set of input points, provides a straightforward scenario:

$${{\mathcal{D}}}_{{\rm{iso}}}={\{({{\bf{x}}}_{i},{f}_{1}({{\bf{x}}}_{i}),\ldots,{f}_{T}({{\bf{x}}}_{i}))\}}_{i = 1}^{N},$$

(10)

where T is the number of tasks. Conversely, heterotopic data arises when tasks are observed at different input points:

$${{\mathcal{D}}}_{{\rm{hetero}}}=\mathop{\bigcup }\limits_{t=1}^{T}{\{({{\bf{x}}}_{i,t},{f}_{t}({{\bf{x}}}_{i,t}))\}}_{i = 1}^{{N}_{t}},$$

(11)

leading to sparsely and partially missing observations for some tasks. This characteristic is particularly common in materials informatics due to the varying feasibility or cost of data collection for different properties^17,18.

By incorporating the capability to handle heterotopic data, DGP models offer a significant advantage. Their ability to leverage inter-task correlations not only maximizes information utility from incomplete datasets but also enhances predictive accuracy and uncertainty quantification—key factors for accelerating materials discovery. The clear delineation between isotopic and heterotopic data thus bridges the theoretical modeling framework with the practical challenges encountered in experimental datasets.

The DGP models implemented in this work consist of two-layered variational GPs, each configured with 10 latent GPs and implemented using BoTorch³⁶. To ensure numerical stability and consistent predictions, all input and output variables were scaled prior to model fitting and subsequently descaled to original units during result visualization.

A critical hyperparameter specific to the DGP architectures evaluated is the reduction parameter, illustrated schematically in Fig. 7. The reduction parameter dictates the dimensionality reduction performed between the first and second layers of the DGP, calculated as the number of tasks minus a chosen integer. Selecting an optimal reduction parameter is essential, as it influences the capacity of the DGP to capture inter-task correlations and the complexity of latent representations. We systematically explored different values of this hyperparameter by fitting models with varying reduction parameters and selected the best-performing value based on cross-validation accuracy metrics. We plotted the Spearman rank coefficient and RMSE over different reduction parameters and chose the reduction parameter that yielded the highest Spearman rank coefficient and lowest RMSE value. The results for HDGP-P-All are reported in Fig. 8. The plot shows that a reduction parameter of 5 achieves the lowest RMSE and highest Spearman's rank co-efficient over the test set on average over all tasks. So, reduction parameter of 5 was chosen for the model. The other relevant hyperparameters of the DGP models include kernels, the optimizer used for training and the number of latent GPs in each layer. We used RBF kernel for the first layer and a Matern kernel for the second layer. 10 latent GPs were used in each layer. We used the Adam optimizer with 0.01 learning rate for training. The models were trained using NVIDIA A100 GPU. Training of each instance of the DGP models takes roughly 5 min with CUDA acceleration.

Fig. 7: Schematic representation of the two-layered variational deep Gaussian process (DGP) architecture.

Each variational multi-task Gaussian process (MTGP) layer reduces dimensionality from input to latent representation, governed by the reduction parameter.

Fig. 8: The figure illustrates RMSE and spearman rank coefficient for different reduction parameters.

Figure at the top shows mean RMSE for test set over all tasks with respect to different reduction parameter for HDGP-NP-All. Bottom figure shows mean spearman rank coefficient over all tasks for test set with respect to different reduction parameters.

Encoder-decoder model for tabular data learning

The encoder-decoder framework is a powerful approach for learning complex input-output relationships in regression tasks and has shown competitive performance compared to tree-based methods on small datasets¹¹. Its performance improves with larger datasets, enabling the model to capture more nuanced patterns and generalize better to unseen inputs. Although the BIRDSHOT dataset is relatively small, we used the encoder-decoder model to benchmark its performance against deep GPs, as both are capable of modeling non-linear mappings and uncertainty in high-dimensional spaces. Training involves minimizing the difference between the predicted and actual values using a loss function, typically optimized with gradient-based methods. After training, the model can generalize to new inputs and produce corresponding predictions. Although the model offers strong predictive performance, its internal representations are often not interpretable. However, explainability can be improved using attention mechanisms, feature attribution methods, and visualization tools. Different neural architectures, including but not limited to dense networks, disjunctive normal form networks (DNF-Nets), and convolutional neural networks (CNNs), can be used at the core of the encoder and decoder. These architectures can enhance accuracy by capturing detailed patterns and filtering noise. In this study, we used regularized dense networks, also known as feedforward networks. Non-linear activation functions like ReLU or sigmoid introduced flexibility and regularization methods such as L2 penalties were used to reduce overfitting.