Extended Data Table 3 Overview of sources of pitfalls for object detection metrics ((a): detection metrics, (b): localization criteria) related to poor metric selection [P2]

From: Understanding metric-related pitfalls in image analysis validation

  1. A warning sign indicates a potential pitfall for the metric in the corresponding column, in case the property represented by the respective row holds true. Comprehensive illustrations of pitfalls are available in Supplementary Note 2. A comprehensive list of pitfalls is provided separately for each metrics in the metrics cheat sheets (Supplementary Note 3). Note that we only list sources of pitfalls relevant to the considered metrics. Other sources of pitfalls are neglected for this table. (a) Detection metrics. Considered counting metrics: Fβ Score (Fig. SN 3.43), Positive Predictive Value (PPV) (Fig. SN 3.51), Sensitivity (Sens) (Fig. SN 3.52). Considered multi-threshold metrics: Average Precision (AP) (Fig. SN 3.56) and Free-Response Receiver Operating Characteristic (FROC) (Fig. SN 3.57). (b) Localization criteria. Considered localization criteria: Box/Approx IoU (Fig. SN 3.74), Center Distance (Fig. SN 3.72), Mask IoU > 0 (Fig. SN 3.75), and Point inside Mask/ Box/ Approx (Fig. SN 3.76).
Back to article page