Figure 1
From: Learning on tree architectures outperforms a convolutional feedforward network
Examined convolutional LeNet-5 and Tree-3 architectures. (a) The convolutional LeNet-5 for the CIFAR-10 database consists of 32 × 32 RGB input images belonging to the 10 output labels. Layer 1 consists of six (5 × 5) convolutional filters followed by (2 × 2) max-pooling, Layer 2 consists of 16 (5 × 5) convolutional filters and Layers 3–5 have three fully connected hidden layers of sizes 400, 120, and 84, which are finally connected to the 10 output units. (b) Scheme of the routes affecting the updating of a weight belonging to Layer 1 in panel (a) (dashed red line) during the BP procedure. The weight is connected to one of the output units via multiple routes (dashed red lines) and can exceed one million (“Methods” section). Note that all weights in Layer 1 are equalized to 6 × (5 × 5) weights belonging to the six convolutional filters. (c) Examined Tree-3 structure consisting of M branches with the same 32 × 32 RGB input images. Layer 1 consists of 3 × K (5 × 5) convolutional filters and K filters for each of the RGB input images. Each branch consists of the same 3 × K filters, followed by (2 × 2) max-pooling which results in (14 × 14) output units for each filter. Layer 2 consists of a tree (non-overlapping) sampling (2 × 2 × 7 units) across the K filters for each RGB color in each branch, resulting in 21 (7 × 3) outputs for each branch. Layer 3 fully connects the 21 × M outputs of the M branches of Layer 2 to the 10 output units. (d) Scheme of a single route (dashed black line) connecting an updated weight in Layer 1 (in panel c), during the BP procedure, to an output unit.
