Table 4 Comparison of adversarial texts’ perturbation rate. The minimum value in each row is emphasized in bold.
From: Hard label adversarial attack with high query efficiency against NLP models
Data | Model | Attack | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
TextFooler | GA-A | PSO-A | LSHA | HLBB | TextHoaxer | LeapAttack | SSPAttack | QEAttack | ||
AG | BERT | 21.65 | 16.45 | 15.56 | 20.32 | 13.15 | 15.72 | 18.37 | 12.22 | 12.43 |
WordCNN | 13.22 | 14.91 | 12.44 | 14.43 | 10.97 | 12.97 | 14.81 | 9.96 | 9.91 | |
WordLSTM | 18.96 | 16.11 | 15.55 | 20.79 | 13.54 | 16.24 | 18.28 | 12.62 | 12.49 | |
MR | BERT | 15.25 | 16.18 | 11.27 | 14.44 | 13.86 | 12.31 | 16.58 | 12.61 | 13.45 |
WordCNN | 13.75 | 15.86 | 11.77 | 14.40 | 13.74 | 12.49 | 15.44 | 12.86 | 12.95 | |
WordLSTM | 13.95 | 17.23 | 11.57 | 15.02 | 14.28 | 12.59 | 16.09 | 13.07 | 13.52 | |
Yelp | BERT | 9.07 | 11.09 | 7.77 | 8.63 | 7.65 | 9.77 | 11.02 | 7.49 | 6.97 |
WordCNN | 6.81 | 11.63 | 7.55 | 6.96 | 7.20 | 8.90 | 9.58 | 6.81 | 6.58 | |
WordLSTM | 7.18 | 10.72 | 7.53 | 7.79 | 6.71 | 8.32 | 9.01 | 6.37 | 6.22 | |
Yahoo | BERT | 8.81 | 8.55 | 6.25 | 7.62 | 6.52 | 7.30 | 8.63 | 6.23 | 6.10 |
WordCNN | 8.60 | 9.15 | 6.96 | 7.91 | 7.20 | 8.22 | 9.17 | 6.76 | 6.44 | |
WordLSTM | 10.51 | 9.51 | 7.70 | 9.86 | 7.59 | 8.83 | 10.20 | 7.31 | 6.92 | |
IMDB | BERT | 3.50 | 6.32 | 4.44 | 3.44 | 3.88 | 5.40 | 5.77 | 3.79 | 3.23 |
WordCNN | 3.00 | 6.86 | 4.36 | 2.89 | 3.59 | 4.72 | 4.59 | 3.33 | 2.84 | |
WordLSTM | 2.96 | 6.33 | 4.18 | 3.39 | 3.52 | 4.46 | 4.59 | 3.31 | 2.77 | |
Average | 10.48 | 11.79 | 8.99 | 10.53 | 8.89 | 9.88 | 11.48 | 8.32 | 8.19 | |
