Table 2 Applicability of augmented DFA to practical network models

From: Physical deep learning with biologically inspired training method: gradient-free approach for physical hardware

 

MNIST

CIFAR-10

BP

DFA (Ours)

Augmented DFA (Ours)

Train only final layer

BP

DFA (Ours)

Augmented DFA (Ours)

Train only final layer

Custom MLP-mixer-3

98.90%

98.36%

98.49%

92.85%

76.50%

62.12%

62.01%

43.73%

ResNet-18

99.51

99.41%

99.23%

81.70%

91.86%

81.68%

80.38%

34.92%

ViT-3 (w/o fine tuning)

98.23%

98.04%

98.03%

47.6%

73.91%

59.00% (DFA+BP)

58.78% (DFA+BP)

32.50%

SNN

97.90% (best)

86.13% (ave.)

96.73% (best)

93.19% (ave.)

98.14% (best)

98.05% (ave.)

92.21% (best)

91.95% (ave.)

—

—

—

—

  1. Scores for MNIST and CIFAR-10 for various ANN models trained by BP, standard DFA, and augmented DFA. As references, results for the models trained by the final layer only (the other layers are randomly fixed) are shown. Accuracies for DFA and augmented DFA higher than that for final layer-only training indicate the training works effectively. Since the training became unstable with BP and the DFA algorithm when using a derivative function of spiking neurons, both the best and average scores are shown for SNN setups for fair comparison (see Supplementary Information S1.4 for the details).
Back to article page