Fig. 4: Evaluation of metrics.
From: Cryo-EM model validation recommendations based on outcomes of the 2019 EMDataResource challenge
Model metrics (Table 2) were compared with each other to assess how similarly they performed in scoring the challenge models. a–d, Fit-to-Map metrics analyses. a, Pairwise correlations of scores for all models across all map targets (n = 63). b, Average correlation of scores per target (average over four correlation coefficients, one for each map target with T1, n = 16; T2, n = 15; T3, n = 15; T4, n = 17). Correlation-based metrics are identified by bold labels. In a, table order is based on a hierarchical cluster analysis (Methods). Three red-outlined boxes along the table diagonal correspond to identified clusters (no. c1–c3). For ease of comparison, order in b is identical to a. c, Representative score distributions are plotted by map target, ordered by map target resolution (see legend at bottom; T1, n = 16; T2, n = 15; T4, n = 17; T3, n = 15). Each row represents one of the three clusters defined in (a). Each score distribution is represented in box-and-whisker format (left) along with points for each individual score (right). Lower boxes represent Q1–Q2 (25th–50th percentile, in target color as shown in legend); upper boxes represent Q2–Q3 (25th–75th percentile, dark gray). Boxes do not appear when quartile limits are identical. Whiskers span 10th to 90th percentile. To improve visualization of closely clustered scores, individual scores (y values) are plotted against slightly dithered x values. d, Scores for one representative pair of metrics are plotted against each other (CCbox from cluster 1 and Q-score from Cluster 2). Diagonal lines represent linear fits by map target. e, Coordinates-only metrics comparison. f, Fit-to-Map, Coordinates-only and Comparison-to-Reference metrics comparison. Correlation levels in a,b,e,f are indicated by shading (see legend at top). See the Methods for additional details.
