Extended Data Table 4 Overview of sources of pitfalls for instance segmentation metrics (Part 1) ((a): detection metrics, (b): localization criteria) related to poor metric selection [P2]

From: Understanding metric-related pitfalls in image analysis validation

  1. A warning sign indicates a potential pitfall for the metric in the corresponding column, in case the property represented by the respective row holds true. Comprehensive illustrations of pitfalls are available in Supplementary Note 2. A comprehensive list of pitfalls is provided separately for each metrics in the metrics cheat sheets (Supplementary Note 3). Note that we only list sources of pitfalls relevant to the considered metrics. Other sources of pitfalls are neglected for this table. (a) Detection metrics. Considered counting metrics: Fβ Score (Fig. SN 3.43), Positive Predictive Value (PPV) (Fig. SN 3.51), Panoptic Quality (Fig. SN 3.49) Sensitivity (Sens) (Fig. SN 3.52). Considered multi-threshold metrics: Average Precision (AP) (Fig. SN 3.56) and Free-Response Receiver Operating Characteristic (FROC) (Fig. SN 3.57). (b) Localization criteria. Considered localization criteria: Boundary Intersection over Union (IoU) (Fig. SN 3.59), Intersection over Reference (IoR) (Fig. SN 3.73), Mask IoU (Fig. SN 3.74).
Back to article page