Fig. 1: An overview of how data are used in Matbench Discovery.

a, A conventional prototype-based discovery workflow where different elemental assignments to the sites in a known prototype are used to create a candidate structure. This candidate is relaxed using DFT to arrive at a relaxed structure that can be compared against a reference convex hull. This sort of workflow was used to construct the WBM dataset. b, Databases such as the MP provide a rich set of data that different academic groups have used to explore different types of models. While earlier work tended to focus on individual modalities, our framework enables consistent model comparisons across modalities. c, The proposed test evaluation framework where the end user takes an ML model and uses it to predict a relaxed energy given an initial structure (IS2RE). This energy is then used to make a prediction as to whether the material will be stable or unstable with respect to a reference convex hull. From an applications perspective, this classification performance is better aligned with intended use cases in screening workflows.