Introduction

Global warming is one of the most pressing and critical problems facing the world today. It is mainly caused by the increase in greenhouse gases in the atmosphere, such as carbon dioxide (CO2), methane (CH4) and nitrous oxide (N2O), resulting from human activities such as burning fossil fuels, deforestation and intensive agriculture. Climate change is causing global temperatures to rise, which in turn directly affects biological processes in soils1. Soil microbes decompose organic matter and release CO2 as a by-product of this process (microbial respiration). The rate of this CO2 release is strongly influenced by soil temperature, and Q10 indicates how much this respiration increases with increasing temperature2,3. Q10 is widely recognized as a critical metric for quantifying the sensitivity of soil respiration across various ecosystems, including forests, grasslands, and agricultural soils4,5,6,7. Q10 is an important parameter in ecology and biology, used to describe the sensitivity of a biological or chemical process to temperature changes. Specifically, Q10 measures how much the rate of a biological reaction (such as microbial respiration in soil) increases with a a temperature increase of \(10 ^\circ\) Celsius8,9. Q10 is often used to describe how much soil microbes increase their respiration activity (and therefore CO2 release) in response to warming. A Q10 of 2, for example, means that for every \(10^{\circ }\hbox {C}\) increase in temperature, the rate of respiration doubles. Soils with low Q10 are less responsive to temperature changes. With a low Q10, CO2 release due to microbial respiration is less pronounced even under warming conditions. A low Q10 indicates greater resilience to temperature changes, with a lower potential for CO2 release and a higher capacity to retain carbon. This makes such soils crucial for climate change mitigation strategies and sustainable ecosystem management. A high Q10 indicates that microbial respiration increases rapidly with increasing temperature. Increased microbial respiration leads to accelerated CO2 release and subsequent degradation of soil quality due to the positive feedback between temperature and microbial respiration10,11. Soils with a high Q10 should be monitored carefully, as they are more vulnerable to warming and may require more active soil management strategies to avoid excessive CO2 release. For example, agricultural practices that enhance carbon sequestration, such as growing cover crops or adding compost, could be adopted. Q10 is particularly important in the context of climate change because it helps us understand how global warming will affect carbon cycling in soils. Since soils store large amounts of carbon in the form of organic matter, increased microbial respiration due to rising temperatures could lead to accelerated release of CO2 into the atmosphere, further exacerbating global warming. Therefore, understanding and predicting Q10 in different soil types is essential to assess the risk of positive climate feedback: more CO2 released from soils could accelerate warming, which in turn could further stimulate microbial respiration.

In recent years, machine learning (ML) and explainable artificial intelligence (XAI) have entered various areas of the agricultural field, providing new ways to improve efficiency, precision, and sustainability12. These technologies are now being used to address complex agricultural challenges, including the prediction of soil health and the development of data-driven strategies for mitigating the impact of climate change. By making models more interpretable, XAI allows for actionable insights that can be directly applied to agricultural practices13.

Previous studies have explored the role of machine learning in predicting soil respiration dynamics, but these efforts primarily focused on traditional black-box models that limit interpretability14,15,16. This lack of transparency can be a major drawback when applying machine learning in critical areas like agriculture and environmental science. Moreover, these studies often miss the opportunity to uncover more granular insights about specific subgroups of soils that may be at higher risk under warming. As a result, there remains a gap in our ability to identify subtle variations within low Q10 soils that could lead to increased sensitivity to temperature.

By utilizing Machine Learning (ML) techniques, we can delve deeper into the complex interactions between soil properties, microbial communities17, and environmental factors to identify which soils are higher risk of increasing the sensitivity of soil microbial respiration to warming18.

Explainable artificial intelligence (XAI) can play a key role in applying machine learning (ML) models to analyze and predict the sensitivity of soil microbial respiration to warming (Q10). The main goal of XAI is to make ML models transparent and interpretable, allowing us to understand why and how models reach certain conclusions19. This is particularly useful in contexts such as climate change and soil biology, where decisions based on black-box models may not be reliable or understandable to domain experts. XAI enables the interpretation of ML models by identifying the most influential soil characteristics (e.g. organic content, pH) in predicting high Q10 values. With techniques such as SHAP (SHapley Additive exPlanations) it is possible to see the importance and impact of each factor on each individual prediction of the model20.

If the ML model predicts that a particular soil is at high risk of increasing microbial respiration due to warming, XAI can reveal that this is mainly due to low soil organic matter or high nitrogen content. This helps focus attention on those aspects that need to be monitored or mitigated. With interpretable models, XAI could help scientists make more robust hypotheses about underlying microbial mechanisms21. Knowing exactly why a model predicts a high Q10 could reveal new insights into the interactions between microbes, soil, and temperature, improving our understanding of soil carbon cycling. XAI improves confidence in model outputs. In a scenario where ML model outputs are used to inform land management policies or climate change mitigation strategies, decision makers need to understand how predictions are generated. For example, an ML system that recommends actions to reduce the increase in soil microbial respiration needs to be able to explain the rationale behind those recommendations22. This increases the acceptance and adoption of models by experts.

One of the strengths of using SHAP values within the XAI framework is that they provide a detailed understanding of feature importance, which can be leveraged to create an embedding space where the separation between classes is more distinct. This embedding, generated from SHAP values, is particularly suited for classification tasks, as it enhances the distinction between different groups, making it easier to apply further analyses like clustering23. By creating such an embedding, we can not only improve classification accuracy but also explore more granular patterns within the data.

In the context of soil respiration sensitivity (Q10), this study uses SHAP-based embeddings to distinguish soils with high Q10 from those with low Q10. Once this separation was achieved, the objective was to focus on soils classified as low Q10 and determine which among them have characteristics similar to those with high Q10. Through clustering analysis on this embedding, we aimed to identify subgroups of low Q10 soils that may be at higher risk of transitioning to high Q10. This approach allows us to pinpoint the factors most responsible for increasing the risk, providing valuable insights into soil management strategies aimed at mitigating warming-induced CO2 release.

The objectives of this study are twofold: first, to utilize interpretable machine learning models to assess the extent to which biochemical, microbiome, and environmental soil characteristics are associated with higher or lower Q10 values; and second, to identify subgroups of low Q10 soils that are at greater risk of transitioning to higher Q10 values. By leveraging SHAP-based embeddings and clustering techniques, this research aims to uncover granular patterns within low Q10 soils, providing actionable insights for targeted soil management strategies to mitigate warming-induced CO2 release.

Materials and methods

In Table 1 detailed information about each input variable is provided. These variables were downloaded from a public repository and have been used for a similar analysis, offering a comprehensive summary of the diverse climate conditions and soil characteristics across various countries globally.

The dataset includes 332 unique soil samples collected from a global survey spanning multiple ecosystems. No duplicate samples were included in the analysis, and each sample represents a distinct soil environment. Sampling points span 29 countries and seven continents, with geographic locations ranging between \(67^\circ \hbox {N}\) and \(62^\circ \hbox {S}\) latitude and \(153^\circ \hbox {E}\) to \(159^\circ \hbox {W}\) longitude, encompassing diverse environmental conditions and vegetation types, including forests, grasslands, and shrublands. Field surveys were conducted between 2016 and 2019. Soil samples were collected at a depth of 0–10 cm, a biologically active soil layer, to capture relevant microbial and biochemical activities. Each site included composite samples derived from multiple soil cores, which were homogenized to ensure representative sampling. The climatic conditions represented in the dataset include tropical, temperate, continental, polar, and arid biomes, with mean annual temperatures (MAT) ranging from \(-7^\circ \hbox {C}\) to \(30^\circ \hbox {C}\) and mean annual precipitation (MAP) ranging from 50 to 4,500 mm/year24.

For each soil sample, 27 distinct factors were utilized, encompassing environmental traits, soil microorganisms, biochemical stability, substrate levels, and mineral protection elements. Additionally, each sample is associated with a Q10 value, providing an in-depth perspective on the sensitivity of soil respiration to temperature variations.

The environmental traits cover soil-specific characteristics and the geographical location of samples. These parameters include longitude, forest presence, MAP, MAT, plant richness, plant cover, pH, electrical conductivity, clay and silt percentage, total phosphorus content, and the Soil C ratio, based on total organic carbon and total nitrogen levels.

Additionally, biochemical stability refers to the resistance of organic compounds to degradation and includes factors such as the proportions of aromatic compounds, alkanes, polysaccharides, and amides. Mineral protection factors emphasize the proportions of particulate organic carbon (POC) and mineral-associated organic carbon (MAOC), which play a role in how organic matter interacts with minerals and its stabilization over time.

Finally, the dataset incorporates information about the soil microbiome, capturing microbial diversity and activity. This includes measures like microbial biomass (estimated via glucose-induced respiration), bacterial, fungal, and protist richness, as well as standardized ratios of bacterial, fungal, and protist taxa associated either positively or negatively with Q10. These microbiome factors provide key insights into how microbial communities respond to soil warming, ultimately affecting soil respiration processes.

The environmental auxiliary variables were derived as follows:

  • Climatic variables, such as mean annual temperature (MAT) and mean annual precipitation (MAP), were obtained from the WorldClim database v225;

  • Soil-specific properties, including pH, electrical conductivity, and fine texture, were derived from direct measurements on the soil samples following standardized laboratory protocols24.

Table 1 Description of the input variables used in the analysis.
Full size table

Data preparation

To evaluate biomarkers distinguishing soils with high and low Q10 values, we partitioned the soil samples into two categories based on the median Q10 value. The median split approach was chosen for statistical reasons, as it is a widely used method for dichotomizing continuous variables while being less sensitive to outliers compared to the mean26. This method ensures a balanced classification while accounting for the broad range of temperature, vegetation types, and soil characteristics represented in our dataset. The group with low Q10 consisted of 166 soil samples, while the group with high Q10 also contained 166 soil samples. This binary classification served as the foundation for further analysis, particularly in identifying the factors contributing to high Q10 and determining soils at greater risk.

Machine learning based classification

Fig. 1
figure 1

Workflow for predicting Q10 sensitivity in soils. Environmental, biochemical, and microbiome variables are used as input features. The Extra Trees Classifier distinguishes between high and low Q10 soils, with SHAP values providing interpretability. Clustering of low-Q10 soils identifies subgroups at higher risk of transitioning to high Q10. The histogram uses a total of 24 bins.

Full size image

The workflow adopted for the analysis is depicted in Figure 1. The illustration shows the mechanism of soil respiration and the process of Q10 prediction. The histogram presents the distribution of the two groups, divided by the median line into high Q10 and low Q10 categories, providing initial insight into the dataset. Subsequently, all variables related to these soils were merged to form the input table for further analysis.

The workflow consisted of two main phases. The first phase applied machine learning (ML) techniques using a classifier to distinguish soils with high Q10 from those with low Q10. In the second phase, explainable artificial intelligence (XAI) methods, specifically SHAP (SHapley Additive Explanations) values, were used to enhance transparency. SHAP values were crucial in identifying new biomarkers and understanding the importance of each feature in the model. Clustering analysis was then performed on the low Q10 group to identify subgroups of soils at higher risk of having high Q10 values.

To ensure the reliability and consistency of our results, we applied a 10-fold nested cross-validation procedure, repeated 20 times. During this process, the parameters of the machine learning algorithm were fine-tuned within the inner loop of the cross-validation. After the model was trained, SHAP values were calculated to provide greater transparency in the model’s decision-making process. These values helped explain the importance and influence of each variable on the prediction outcomes.

To evaluate the machine learning performance for predicting high and low Q10 values, the Extra Trees Classifier was selected because its performance was shown to be superior in the referenced article13. The Extra Trees Classifier is an ensemble learning algorithm designed for classification tasks, based on decision tree methodologies. It uses randomization techniques to enhance accuracy and minimize variance. Similar to other ensemble methods, the Extra Trees Classifier constructs multiple decision trees. Each tree is trained on a randomly chosen subset of the data, with random features selected at each split. This approach of random node splitting and data sampling helps the Extra Trees Classifier to be more resistant to overfitting compared to standard decision trees. As a result, it performs well in scenarios where the data is noisy or highly variable. Additionally, Extra Trees is adept at managing datasets with numerous features, making it ideal for tackling complex challenges27. Moreover, the random feature selection during tree construction enables the model to estimate the significance of each feature.

The parameters grid for the ensemble models are as follows:

  • max_depth \(\in\) {None, 3, 5, 7, 9},

  • n_estimators \(\in\) {50, 100, 150, 200, 250}.

We systematically tested a range of parameter values to enhance the classifiers’ generalization to unseen data and improve their overall predictive accuracy.

Evaluation metrics

Evaluation metrics are crucial tools for assessing the performance and effectiveness of machine learning models28. These metrics provide quantitative measures that help quantify how well a model is performing on a given task. The most commonly used evaluation metrics in classification analysis are:

  • Accuracy:

    It is defined as the proportion of correct predictions made by the model out of the total number of predictions

    $$\begin{aligned} ACC=\frac{Number of Correct Predictions }{Total Number of Predictions } \end{aligned}$$
    (1)

  • Precision:

    It measures how many of the instances predicted as positive are actually positive

    $$\begin{aligned} PREC=\frac{True Positives (TP)}{True Positives (TP) + False Positives (FP)} \end{aligned}$$
    (2)

  • Area Under the ROC Curve (AUC-ROC):

    The ROC (Receiver Operating Characteristic) curve and AUC (Area Under the Curve) are key metrics used to assess the performance of binary classification models. The ROC curve visually represents the relationship between a model’s sensitivity (true positive rate) and specificity (true negative rate) across different decision thresholds. The AUC provides a numerical measure of this performance, representing the area under the ROC curve: a value near 1 indicates excellent model accuracy, whereas a value near 0.5 indicates that the model’s predictions are no better than random guessing29.

  • Area Under the PRC Curve (AUC-PRC):

    The Precision-Recall Curve (PRC) and the Area Under the Curve (AUC) are used to evaluate the effectiveness of binary classification models. The PRC illustrates how precision (the proportion of true positives among all positive predictions) and recall (the proportion of actual positives correctly identified) vary across different threshold settings. The AUC of the PRC provides a summary measure of the model’s performance in balancing precision and recall. A higher AUC reflects better overall performance, with a perfect model achieving an AUC of 130.

Explainable artificial intelligence (XAI)

Explainable Artificial Intelligence (XAI) represents a crucial development in AI, aimed at making models more transparent and understandable to humans. One of the main technique is SHapley Additive exPlanations (SHAP) which provides a structured method for interpreting machine learning model outputs by assigning importance to each feature in relation to the model’s predictions31,32,33. By evaluating the effect of each feature on the predictions, SHAP values offer significant insights into the decision-making process of the model. One key advantage of SHAP is the ability to generate local explanations, where each SHAP value corresponds to an individual instance in the dataset. This allows for a granular understanding of how specific features contribute to the prediction for each instance. The SHAP value for the jth feature of the instance x is calculated by aggregating it across all possible subsets according to the formula:

$$\begin{aligned} \Phi _j(x)=\sum _{F\subseteq S- \{j\}}\frac{|F|!(|S|-|F|-1)!}{|S|!}[f_{x}(F\cup j)-f_{x}(F)] \end{aligned}$$
(3)

|F|! denotes the permutations of features in the subset F, \((|S|-|F|-1)!\) represents the permutations of features in the subset \(S-(F\cup \{j\})\), and |S|! signifies the total number of feature permutations34.

The SHAP values, calculated for each individual soil sample, provided a basis for generating embeddings that captured the contribution of each feature at a local level.

In this study, SHAP values explain probabilities, where positive values indicate an increase in the probability of a soil sample being classified as high Q10, and negative values indicate a decrease in this probability. The magnitude of the SHAP value reflects the strength of the feature’s contribution to the prediction. This enables a granular understanding of how specific features influence prediction probabilities.

Clustering analysis

T-distributed Stochastic Neighbor Embedding (t-SNE) is a widely used technique for dimensionality reduction, particularly in data visualization and exploration. It is designed to project high-dimensional data into lower dimensions (typically two or three) while preserving the relative distances between data points. Unlike linear dimensionality reduction techniques like PCA, t-SNE is capable of capturing complex, non-linear relationships in the data, making it an ideal choice for visualizing clusters that may not be evident in high-dimensional space. t-SNE operates by minimizing the divergence between two probability distributions: one representing pairwise similarities in the high-dimensional space and the other representing these similarities in the lower-dimensional space35,36.

In this study, t-SNE was used to visualize the data by reducing the dimensionality of both the input feature space and the SHAP value space23. By applying t-SNE to both, we aimed to identify the embedding that best separates the two classes (low Q10 and high Q10). This embedding was then used to perform clustering on the low Q10 soils, identifying subgroups at higher risk of transitioning to high Q10, using three unsupervised clustering algorithms: KMeans, Agglomerative Clustering, and Birch Clustering.

The classification probabilities used to assess the risk of transitioning to high Q10 were derived from the machine learning model trained to distinguish between high-Q10 and low-Q10 soils. Each soil sample was assigned a probability score representing its likelihood of being classified as high Q10. After clustering the low-Q10 soils using SHAP-based embeddings, we analyzed the distribution of these classification probabilities within each cluster. This allowed us to identify subgroups of low-Q10 soils with higher predicted probabilities of transitioning to high Q10, indicating potential heterogeneity within this category.

  • KMeans Clustering: KMeans is a widely used clustering algorithm that partitions data into a predefined number of clusters (K). It works by minimizing the within-cluster variance by iteratively assigning data points to the nearest cluster centroid and then updating the centroids based on the mean of the points in each cluster37.

  • Agglomerative Clustering: This is a hierarchical clustering method that builds nested clusters by merging or splitting them successively. In this study, we used a bottom-up approach, starting with each sample as its own cluster and then successively merging the closest pairs of clusters based on a defined distance metric. This process continues until a desired number of clusters is reached38.

  • Birch Clustering: Birch (Balanced Iterative Reducing and Clustering using Hierarchies) is particularly effective for large datasets. It builds a clustering feature tree to summarize the dataset and then applies hierarchical clustering to create clusters. Birch is efficient and scalable, making it suitable for complex, high-dimensional datasets39.

The number of clusters was determined using the Silhouette Score, a metric that evaluates the consistency and quality of the clustering. The Silhouette Score is calculated by measuring the mean intra-cluster distance (how close each point in a cluster is to other points in the same cluster) and the mean nearest-cluster distance (how close each point is to points in the nearest different cluster). The higher the Silhouette Coefficient, the better defined the clusters are. The clustering method that yielded the highest Silhouette Score was selected as the optimal clustering solution40,41.

Results

This study applied interpretable machine learning models to analyze the relationships between biochemical, microbiome, and environmental soil characteristics and Q10 values. Through clustering analyses of low Q10 soils using SHAP-based embeddings, we identified distinct subgroups with varying probabilities of transitioning to high Q10.

The average performance of the machine learning classifiers, calculated as the mean performance obtained during the 10-fold cross-validation repeated 20 times, is summarized as follows:

  • Accuracy (ACC): \(0.813 \pm 0.007\);

  • Precision: \(0.812 \pm 0.008\);

  • Area Under the ROC Curve (AuROC): \(0.884 \pm 0.003\);

  • Area Under the Precision-Recall Curve (AuPRC): \(0.883 \pm 0.007\).

This approach was chosen to ensure the robustness and generalizability of the machine learning model.

Fig. 2
figure 2

ROC (a) and Precision-Recall (b) curves showing the performance of the machine learning classifiers. The bold blue line represents the average curve across 20 repetitions, while the shaded region corresponds to one standard deviation around the mean. The individual-colored curves represent the ROC and Precision-Recall curves from each of the 20 repetitions, illustrating the variability in model performance across iterations. The dashed red line indicates the performance of a random classifier (chance level).

Full size image

Figure 2 displays the ROC (a) and Precision-Recall (b) curves along with their respective areas under the curve. The figure shows the average curves with standard deviations derived from the 20 repetitions.

Fig. 3
figure 3

SHAP summary plot of feature importance in predicting Q10 sensitivity. The x-axis represents each feature’s contribution to the model, where positive values indicate an increased likelihood of high Q10. The color gradient (blue = low, red = high) represents feature values. The most influential predictors include “Bacteria_Positive” and “Microbial_Biomass”, followed by “Alkane” and “SOC” (soil organic carbon).

Full size image

In Figure 3, the global feature importance is represented using the SHAP summary plot. This plot visualizes the impact of each feature on the model’s output based on the calculation of SHAP values. Each point on the plot corresponds to a SHAP value for a particular feature and a specific soil in the dataset. The color gradient represents the feature value, with blue indicating lower feature values and red representing higher feature values. The position along the x-axis indicates whether the effect of the feature increases or decreases the model’s prediction of Q10 sensitivity.

The two most important variables, as indicated by their impact on the model output, are the proportion of bacterial taxa positively associated with Q10 (“Bacteria_Positive”) and the microbial biomass (“Microbial_Biomass”), estimated via glucose-induced soil respiration. Both variables show a strong influence on the prediction of Q10, suggesting that soils with higher values for these features tend to have increased sensitivity to temperature changes.

Other significant features include the percentage of alkanes (“Alkane”) and soil organic carbon content (“SOC”). These features also play a crucial role in distinguishing soils with higher and lower Q10 values.

Among the environmental variables, forest presence, fine soil texture, and soil pH showed notable associations with Q10 sensitivity. As illustrated in the SHAP summary plot (Figure 3), forested soils (represented by red-colored points for the Forest variable) were associated with a higher likelihood of high Q10 values, indicating greater temperature sensitivity. Similarly, soils with a higher proportion of fine texture (clay and silt) demonstrated an increased likelihood of high Q10 sensitivity. Moreover, soils with lower pH values were more likely to be classified as low Q10, highlighting the influence of soil acidity on temperature sensitivity.

Fig. 4
figure 4

t-SNE projections of input features and SHAP values. (a) Input feature-based t-SNE shows overlap between high-Q10 (orange) and low-Q10 (blue) soils. (b) SHAP value-based t-SNE reveals improved separation between high and low Q10 soils. The Euclidean distance between the centroids of high and low \(\hbox {Q}_{10}\) soils in the input feature space was 5.22, whereas in the SHAP-based t-SNE embeddings, it increased to 20.59, highlighting improved class separability.

Full size image

In this study, a comparison was made between the outcomes of t-SNE conducted on the input features and those resulting from t-SNE applied to the SHAP values. After computing the SHAP values, t-SNE was performed on these values, and the first two components, t-SNE component 1 and t-SNE component 2, were calculated. The visualization of the results is shown in Figure 4 (b). For comparison, Figure 4 (a) also includes the t-SNE results obtained from the input features. Figure 4 (b) presents the projection of the SHAP values onto the two principal components. A distinct pattern was observed, where there is a clear separation between high and low Q10 soils along t-SNE comp.1. In contrast, the t-SNE results based on the input features (Figure 4 (a)) do not exhibit such clear differentiation between the two classes. The SHAP value embeddings, also referred to as local explanation embeddings, outperform the original input features by better capturing the differences between the two soil categories (high Q10 vs. low Q10). This highlights the effectiveness of SHAP values in capturing key distinctions within the dataset. To quantitatively assess the separability between soils with high and low Q10 values, we calculated the centroids for each class in both projections and computed the Euclidean distance between these centroids. In the input feature space (Figure 4(a)), the centroid distance was 5.22, indicating limited class separation. In contrast, the SHAP-based t-SNE embeddings (Figure 4(b)) showed a centroid distance of 20.59, highlighting a marked improvement in separability.

Fig. 5
figure 5

Clustering of low-Q10 soils based on SHAP-derived t-SNE embeddings. (a) K-means clustering achieved the highest Silhouette score. (b) Visualization of six identified clusters. (c) Predicted probabilities of transitioning to high Q10 across clusters, with Clusters 4, 6, and 2 showing the highest risk.

Full size image

Based on the clear separation between high and low Q10 soils observed in the previous analysis, it became reasonable to further cluster the soils classified as low Q10 into distinct subgroups, with the assumption that these subgroups might exhibit varying probabilities of transitioning to high Q10. To explore this hypothesis, clustering techniques were applied to the t-SNE components (comp.1 and comp.2) derived from the SHAP values. The optimal number of clusters was determined using the Silhouette score (Figure 5(a)), which evaluated different clustering algorithms: K-means, Birch, and Agglomerative Clustering. Among these methods, K-means produced the highest Silhouette score, indicating that six clusters offered the best-defined grouping for the low Q10 soils.

The six clusters identified by K-means are shown in Figure 5(b). These clusters represent distinct subgroups within the low Q10 soils. As depicted in Figure 5(c), Clusters 4, 6, and 2 have a higher probability of being classified as high Q10 soils compared to the other clusters. Conversely, soils in Clusters 1 and 3 demonstrate the lowest median probability, suggesting they are less likely to transition into high Q10 soils.

Fig. 6
figure 6

Box plots of the four most important variables identified by the SHAP summary plot across the six clusters. (a) Distribution of “Microbial_Biomass” values across clusters, showing that Clusters 4 and 6 have significantly higher values. (b) The distribution of “Bacteria Positive” values shows that Clusters 4 and 6 exhibit higher levels. (c) and (d) display the distributions for “Alkane” and “SOC” (Soil Organic Carbon), respectively, highlighting differences in these key variables across the identified clusters. These higher values in Clusters 4 and 6 are associated with an increased probability of transitioning to high Q10 soils.

Full size image

Figure 6 presents the box plots of the four most important variables, as identified by the SHAP summary plot (Figure 3). The figure illustrates the distribution of these key variables across the different clusters obtained from the clustering analysis. Notably, in clusters with the highest predicted probabilities of transitioning to high Q10 soils, such as Cluster 4 and Cluster 6, certain variables show distinct patterns.

For instance, in Figures 6(a) and (b), the variables “Microbial_biomass ” and “Bacteria Positive” exhibit significantly higher values in Clusters 4 and 6 compared to the other clusters. This indicates a strong association between elevated levels of these variables and the increased probability of a soil transitioning from low Q10 to high Q10. Conversely, the other clusters, particularly those with lower probabilities (e.g., Clusters 1 and 3), tend to show lower median values for these variables. This trend reinforces the importance of microbial biomass and “Bacteria_Positive” in driving the differences between soils more resistant to temperature sensitivity and those more susceptible.

Discussion

This study makes several significant contributions to the field of environmental research, particularly in predicting soil microbial respiration sensitivity (Q10) using machine learning and explainable AI (XAI) techniques. By leveraging the power of machine learning, we achieved high predictive performance in estimating Q10 values, demonstrating the potential of these models to capture complex soil-climate interactions.

First, our research underscores the critical role of machine learning in predicting Q10 sensitivity with high accuracy across diverse soil types. The high predictive accuracy of our models, as evidenced by metrics such as an AUC-ROC of 0.884 and an AUC-PRC of 0.883, highlights the effectiveness of machine learning in capturing the intricate relationships between soil properties, microbial communities, and environmental factors.

Second, the explainable machine learning approach used in this study successfully uncovered important biochemical, microbiome, and environmental factors that influence Q10 values. These factors, such as glucose-induced soil respiration and the proportion of bacterial taxa positively associated with Q10, emerged as key drivers of soil sensitivity to temperature42,43,44,45. Our findings align with those of Sáez-Sandino et al. (2023)24. However, while Sáez-Sandino et al. used feature importance embedded within a Random Forest model, our study leverages an explainable AI approach based on post-hoc explanations. This methodological shift offers three key advantages: (1) Model Agnosticism, as XAI can be applied to any machine learning model, enhancing flexibility and generalizability; (2) Local Feature Importance, allowing for the interpretation of individual predictions rather than just global trends; and (3) Directional Insights, providing information on whether a variable positively or negatively influences Q10 in different contexts. The identification of these variables not only deepens our scientific understanding of soil-microbiome-climate interactions but also has practical implications. By highlighting the soil characteristics that contribute to higher Q10 values, this research provides a foundation for developing customized agricultural strategies. These strategies could be used to reduce CO2 emissions from soils, enhancing carbon sequestration and mitigating the impact of climate change.

Among the environmental variables, forest presence, fine soil texture, and soil pH exhibited associations with Q10 sensitivity. Forested soils were associated with an increased likelihood of high Q10 values. The organic matter-rich environment of forest soils promotes microbial activity, which may lead to greater temperature sensitivity46. Similarly, fine-textured soils (high in clay and silt content) showed a positive relationship with Q10, supporting the idea that soil texture influences microbial access to substrates and respiration processes. Prior research suggests that finer-textured soils tend to retain moisture and organic matter, creating conditions that can amplify microbial respiration responses to temperature changes47. Additionally, soil pH was found to influence Q10 sensitivity, with lower pH values being associated with a higher likelihood of classification as low-Q10 soils. Soil acidification is known to affect microbial community composition and metabolic processes, potentially leading to reduced respiration sensitivity48. Acidic soils may limit microbial activity and enzymatic efficiency, thereby lowering Q10 values and altering carbon cycling dynamics.

Third, by leveraging local explanation embeddings derived from SHAP values and applying unsupervised clustering methods, this study identified distinct subgroups within the low Q10 soils that are at higher risk of temperature sensitivity. Clusters 4 and 6, in particular, were found to have significantly higher probabilities of transitioning to high Q10 soils, making them prime candidates for targeted interventions. This clustering approach represents a significant advancement over traditional methods, as it not only provides a nuanced categorization of soils but also elucidates the reasons why certain soils are more prone to warming-induced sensitivity. Unlike conventional t-SNE analyses based solely on feature variables, which failed to clearly distinguish between high and low Q10 soils, the SHAP-based embeddings effectively captured the intricate patterns and interactions driving soil sensitivity. This enables the identification of high-risk subgroups within the low Q10 category, facilitating targeted interventions and improving our understanding of the soil-microbiome-climate nexus.

Moreover, our study demonstrates the broader potential of combining XAI with machine learning in environmental research. The transparent nature of XAI models bridges the gap between complex machine learning predictions and practical, actionable insights. This is crucial for gaining the trust of stakeholders, including land managers and policymakers, who require clear, evidence-based guidance for implementing sustainable practices. The integration of XAI into environmental models could also facilitate more effective communication of scientific findings, fostering a wider adoption of AI-driven solutions in agriculture and climate change mitigation.

Overall, this work provides a robust, explainable framework for predicting soil sensitivity to temperature increases, contributing to both the scientific understanding and practical management of soil ecosystems. Future research should integrate this approach with long-term field data, exploring the interactions of Q10 with other soil health indicators, and applying the framework to different biomes. By doing so, we can further enhance our ability to predict and manage soil responses to climate change, ultimately supporting more resilient agricultural systems and ecosystems.

Our findings reveal that the presence of forests, fine soil texture, and soil pH are key environmental variables influencing Q10 sensitivity. However, the absence of explicit soil type classifications in the dataset limited our ability to directly associate Q10 variations with conventional soil types (e.g., sandy, clay, loam). Incorporating detailed metadata on soil classifications in future studies would enable a more comprehensive understanding of how specific soil types influence Q10. Additionally, analyzing Q10 variations within these types could provide more targeted insights into soil management strategies for mitigating climate change.

A key limitation of this study is the representativeness of the dataset, which, although global in scope, may not fully capture the diversity of microbiomes and soil environments across all ecosystems. Certain underrepresented soil types or environmental conditions may introduce biases in the model predictions. Additionally, the reliance on SHAP embeddings and clustering techniques involves assumptions regarding the separability of soils in the reduced-dimensionality space. These methods may be sensitive to the chosen dimensionality reduction parameters and clustering algorithms, potentially impacting the robustness of the identified subgroups. Future studies should explore alternative embeddings and validation techniques to minimize these uncertainties and enhance generalizability.

In conclusion, this study highlights the power of explainable AI in uncovering the key drivers of soil respiration sensitivity to temperature. By providing both accurate predictions and interpretability, our approach bridges the gap between complex machine learning models and practical applications in soil management. However, while these findings offer actionable insights, their practical implementation requires validation through field experiments across diverse environmental conditions and soil types. As the impacts of climate change continue to escalate, the integration of XAI into environmental research will be crucial for developing effective mitigation strategies and ensuring the sustainability of ecosystems.