Fig. 7: Correlations between manifolds.
From: Separability and geometry of object manifolds in deep neural networks
Changes of mean between-manifold correlations along the layers of AlexNet. a Center correlations for top 10% point-cloud manifolds in fully trained network (full line), randomly initialized network (dashed line) or randomly shuffled object manifolds (dotted line). b Center correlations for smooth 2-d shear manifolds in fully trained network (full line) or randomly initialized network (dashed line). Line and markers indicate mean value over different choices of objects; surrounding shaded areas indicate 95% confidence interval. The x-axis labels provides abbreviation of the layer types. Marker shape represents layer type (circle—pixel layer, square—convolution layer, right-triangle—max-pooling layer, hexagon—fully connected layer). Features in linear layers are extracted after a ReLU nonlinearity. Color changes from dark to light along the network. Center correlations are \({\rho }_{CC}={<| {\overrightarrow{x}}^{\mu }\cdot {\overrightarrow{x}}^{\nu }| /| | {\overrightarrow{x}}^{\mu }| | \cdot | | {\overrightarrow{x}}^{\nu }| | > }_{\mu \ne \nu }\) where \({\overrightarrow{x}}^{\mu }\) is the center of object μ (Supplementary Eq. (1)).
