Table 3 Algorithm performance comparison and statistical significance test results.
Algorithm | Datasets and distribution | Average accuracy | Average train time | ||||
|---|---|---|---|---|---|---|---|
ACC(%) | t-statistic | P value | train time(s) | t-statistic | P value | ||
SaAS-FL | MNIST-iid | 98.09 ± 0.04 | – | – | 861.40 ± 22.20 | – | – |
MNIST-noniid (alpha = 0.5) | 97.53 ± 0.14 | – | – | 842.6 ± 28.54 | – | – | |
CIFAR10-iid | 74.65 ± 0.59 | – | – | 1825.80 ± 18.25 | – | – | |
CIFAR10-noniid (alpha = 0.5) | 70.97 ± 0.53 | – | – | 1848.80 ± 21.73 | – | – | |
FedAvg | MNIST-iid | 98.47 ± 0.44 | − 1.922 | 0.1270 | 4622.20 ± 77.48 | − 99.142 | 1 × 10–7 |
MNIST-noniid (alpha = 0.5) | 98.66 ± 0.09 | − 14.897 | 0.0001 | 4683.0 ± 187.44 | − 43.501 | 2 × 10–6 | |
CIFAR10-iid | 85.33 ± 0.08 | − 43.641 | 2 × 10–6 | 13,400.40 ± 25.26 | − 1240.500 | 3 × 10–12 | |
CIFAR10-noniid (alpha = 0.5) | 82.65 ± 0.46 | − 53.423 | 1 × 10–6 | 13,325.00 ± 914.81 | − 28.013 | 1 × 10–5 | |
FedAsync | MNIST-iid | 76.80 ± 3.96 | + 11.980 | 0.0003 | 2136.40 ± 43.95 | − 48.437 | 1 × 10–6 |
MNIST-noniid (alpha = 0.5) | 45.60 ± 9.56 | + 12.300 | 0.0003 | 2125.2 ± 26.39 | − 58.772 | 1 × 10–6 | |
CIFAR10-iid | 41.60 ± 2.07 | + 28.222 | 9 × 10–6 | 2834.20 ± 32.00 | − 102.780 | 5 × 10–8 | |
CIFAR10-noniid (alpha = 0.5) | 35.80 ± 2.77 | + 24.113 | 2 × 10–5 | 2631.80 ± 71.36 | − 21.657 | 3 × 10–5 | |
