Table 5 Understanding the effectiveness of LEMONADE and other SOTA NAS frameworks utilizing the GM metric.
From: An automated multi parameter neural architecture discovery framework using ChatGPT in the backend
Method | Search energy (kWh-PUE) | GM values | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
\(X_A=0.8\), \(X_E=0.2\) | \(X_A=0.5\), \(X_E=0.5\) | \(X_A=0.2\), \(X_E=0.8\) | ||||||||
CIFAR-10 | CIFAR-100 | ImageNet 16-120 | CIFAR-10 | CIFAR-100 | ImageNet 16-120 | CIFAR-10 | CIFAR-100 | ImageNet 16-120 | ||
DeepMaker43 | 35.55 | 0.925 | 0.781 | – | 0.915 | 0.825 | – | 0.906 | 0.870 | – |
CGP-CNN44 | 352.66 | 0.752 | 0.586 | – | 0.470 | 0.367 | – | 0.188 | 0.147 | – |
EIGEN45 | 9.48 | 0.952 | – | – | 0.960 | – | – | 0.968 | – | – |
47.40 | – | 0.798 | – | – | 0.823 | – | – | 0.849 | – | |
GeNet46 | 161.16 | 0.852 | 0.676 | – | 0.736 | 0.626 | – | 0.620 | 0.577 | – |
NSGANet47 | 307.15 | 0.788 | 0.624 | – | 0.541 | 0.439 | – | 0.294 | 0.253 | – |
NASHBOT48 | 19.34 | 0.920 | – | – | 0.929 | – | – | 0.939 | – | – |
NASH-Net49 | 11.38 | 0.952 | – | – | 0.958 | – | – | 0.964 | – | – |
GDAS50 | 4.12 | 0.947 | 0.760 | 0.531 | 0.962 | 0.846 | 0.703 | 0.978 | 0.932 | 0.874 |
DARTS-51 | 1.52 | 0.950 | 0.771 | 0.560 | 0.967 | 0.856 | 0.724 | 0.985 | 0.940 | 0.887 |
DrNAS52 | 0.57 | 0.955 | 0.788 | 0.570 | 0.971 | 0.867 | 0.731 | 0.988 | 0.946 | 0.892 |
SE-NAS53 | 1.39 | 0.947 | – | 0.565 | 0.966 | – | 0.727 | 0.984 | – | 0.889 |
FairNAS54 | 1.29 | 0.945 | – | 0.537 | 0.965 | – | 0.709 | 0.984 | – | 0.882 |
Shapley-NAS55 | 3.41 | 0.953 | – | 0.573 | 0.967 | – | 0.730 | 0.981 | – | 0.886 |
DC56 | 1.85 | 0.953 | – | 0.570 | 0.969 | – | 0.730 | 0.985 | – | 0.889 |
RMI58 | 0.16 | 0.954 | 0.787 | 0.571 | 0.971 | 0.867 | 0.732 | 0.989 | 0.947 | 0.893 |
LEMONADE (Proposed) | 1.20 | 0.964 | – | – | 0.976 | – | – | 0.989 | – | – |
2.34 | – | 0.834 | – | – | 0.894 | – |  | 0.954 | – | |
5.45 | – | – | 0.541 | – | – | 0.707 | – | – | 0.874 | |
