Figure 2

Detection performance of EAD participants on the test dataset. (a) Plot of mean IoU vs mAP, (see also Table 2). Each point represents a team plotted with decreasing marker size with decreasing order of detection score, score_d. Points that lie along the same black dashed lines have the same score_d but show a different trade-off between mAP and IoU. The red line highlight the best performing teams (to the right). Box-plot of test detection score, score_d for individual artefact classes sorted by increasing % of training boxes, (b) and by the normalized box area (area after box width and height have been normalized by the respective image width and height) for all images in the training dataset, (c) White and black horizontal lines indicate the mean and median of boxes. Whiskers are plotted at 1.5 × inter-quartile range of upper and lower quartiles. (d) Error bars and swarm plots of IoU (top), mAP (middle) and the final detection score, (score_d, bottom) for each team and baseline methods. Teams are arranged by decreasing score_d. Error bars show ± 1 standard deviation relative to the mean score across artefact classes. For better visualization, points are adjusted such that they do not overlap in the x-axis. Filled square and diamond markers mark teams whose average ranked performance is significantly different to respective Faster R-CNN and Retinanet methods following Friedman Bonferroni-Dunn post-hoc testing with p < 0.05. (e) Average rank performance of individual methods considering artefact classes independently with detection scores. Solid black lines join methods with no significant rank difference with Friedman Nemenyi post-hoc analysis and p < 0.05. Color bars (b,c) and color points (d) constituting of red, green, blue, violet, orange, yellow and brown represent specularity, artefact, saturation, blur, contrast, bubbles, and instrument classes, respectively.