Fig. 3
From: XElemNet: towards explainable AI for deep neural networks in materials science

Overview of XElemNet framework. The framework includes post-hoc and transparency explanation methods. ElemNet is used as the base model for generating the secondary datasets for compound stability analysis and convex hulls assessment.The transparency explanation component is depicted at the bottom. Surrogate model fitting is performed to generate the decision tree, which is then used for feature importance analysis.