Fig. 4: PL exponent (α) versus layer id for VGG, ResNet, and DenseNet. | Nature Communications

Fig. 4: PL exponent (α) versus layer id for VGG, ResNet, and DenseNet.

From: Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data

Fig. 4

PL exponent (α) versus layer id, for the least and the most accurate models in VGG (a), ResNet (b), and DenseNet (c) series. (VGG is without BN; and note that the Y axes on each plot are different.) Subfigure (d) displays the ResNet models (b), zoomed in to α ∈ [1, 5], and with the layer ids overlaid on the X-axis, from smallest to largest, to allow a more detailed analysis of the most strongly correlated layers. Notice that ResNet152 exhibits different and much more stable behavior of α across layers. This contrasts with how both VGG models gradually worsen in deeper layers and how the DenseNet models are much more erratic. In the text, this is interpreted in terms of Correlation Flow.

Back to article page