Table 9 Rejection rates (Total, Correct Prediction, and Incorrect Prediction) for various confidence and uncertainty measures when ResNet50, trained on Stable Diffusion, is tested on Stable Diffusion (SD), Midjourney (MJ), GLIDE (GD), VQDM (VQ),BigGAN (BG) and StyleGAN3 (SG)
Measure | Dataset | Nature | AI | ||||
|---|---|---|---|---|---|---|---|
Total | Correct | Incorrect | Total | Correct | Incorrect | ||
Probability | SD | 1.45 | 1.28 | 93.33 | 1.85 | 1.62 | 100.00 |
MJ | 1.50 | 1.25 | 88.24 | 39.10 | 52.37 | 32.76 | |
GD | 1.70 | 1.49 | 92.86 | 31.22 | 33.04 | 29.57 | |
VQ | 1.50 | 1.22 | 80.95 | 13.55 | 39.61 | 9.06 | |
BG | 1.47 | 1.29 | 73.33 | 24.13 | 67.76 | 18.07 | |
SG | 0.17 | 0.17 | 0.00 | 0.85 | 0.00 | 0.85 | |
Total Fisher Information | SD | 1.40 | 1.23 | 93.33 | 2.20 | 1.98 | 94.74 |
MJ | 1.43 | 1.19 | 88.24 | 38.48 | 55.36 | 30.42 | |
GD | 1.60 | 1.39 | 92.86 | 31.33 | 34.55 | 28.43 | |
VQ | 1.48 | 1.19 | 85.71 | 13.88 | 43.02 | 8.87 | |
BG | 1.42 | 1.20 | 86.67 | 23.03 | 69.67 | 16.55 | |
SG | 0.17 | 0.17 | 0.00 | 0.85 | 0.00 | 0.85 | |
Fisher Frobenius Norm | SD | 1.43 | 1.24 | 100.00 | 2.39 | 2.17 | 94.74 |
MJ | 1.53 | 1.27 | 94.12 | 39.72 | 56.96 | 31.48 | |
GD | 1.65 | 1.44 | 92.86 | 31.75 | 34.80 | 29.00 | |
VQ | 1.48 | 1.19 | 85.71 | 13.67 | 41.88 | 8.81 | |
BG | 1.48 | 1.27 | 86.67 | 23.88 | 69.67 | 17.52 | |
SG | 0.17 | 0.17 | 0.00 | 0.85 | 0.00 | 0.85 | |
Fisher Entropy | SD | 1.19 | 1.00 | 100.00 | 28.36 | 28.24 | 78.95 |
MJ | 1.45 | 1.17 | 100.00 | 50.25 | 94.07 | 29.31 | |
GD | 1.30 | 1.07 | 100.00 | 51.75 | 75.84 | 29.98 | |
VQ | 1.57 | 1.22 | 100.00 | 19.83 | 78.21 | 9.79 | |
BG | 1.30 | 1.05 | 100.00 | 27.35 | 94.95 | 17.96 | |
SG | 0.17 | 0.17 | 0.00 | 2.32 | 0.00 | 2.32 | |
Gaussian Process | SD | 4.25 | 4.08 | 93.33 | 2.08 | 1.84 | 100.00 |
MJ | 5.45 | 5.18 | 100.00 | 47.77 | 63.51 | 40.25 | |
GD | 4.80 | 4.63 | 78.57 | 38.68 | 35.18 | 41.85 | |
VQ | 4.60 | 4.35 | 76.19 | 23.03 | 49.04 | 18.56 | |
BG | 4.80 | 4.59 | 86.67 | 30.82 | 78.14 | 24.24 | |
SG | 0.68 | 0.68 | 0.00 | 12.61 | 0.00 | 12.61 | |
Entropy of Expected | SD | 74.06 | 74.01 | 100.00 | 84.50 | 84.46 | 100.00 |
MJ | 72.75 | 72.67 | 100.00 | 99.55 | 99.95 | 99.36 | |
GD | 73.57 | 73.50 | 100.00 | 99.75 | 99.93 | 99.59 | |
VQ | 72.92 | 72.82 | 100.00 | 82.70 | 99.55 | 79.80 | |
BG | 73.55 | 73.48 | 100.00 | 99.88 | 100.00 | 99.87 | |
SG | 82.34 | 82.34 | 0.00 | 58.38 | 0.00 | 58.38 | |
Knowledge Uncertainty | SD | 72.47 | 72.42 | 100.00 | 81.94 | 81.89 | 100.00 |
MJ | 75.28 | 75.21 | 100.00 | 99.65 | 99.95 | 99.51 | |
GD | 75.28 | 75.23 | 100.00 | 99.75 | 99.93 | 99.59 | |
VQ | 69.47 | 69.38 | 95.24 | 81.15 | 98.75 | 78.12 | |
BG | 73.78 | 73.72 | 100.00 | 99.87 | 100.00 | 99.85 | |
SG | 87.27 | 87.27 | 0.00 | 62.25 | 0.00 | 62.25 | |
Combined Uncertainty | SD | 32.51 | 32.39 | 100.00 | 33.91 | 33.76 | 100.00 |
MJ | 32.35 | 32.16 | 100.00 | 90.58 | 97.32 | 87.36 | |
GD | 32.12 | 31.96 | 100.00 | 88.75 | 89.75 | 87.85 | |
VQ | 31.43 | 31.19 | 100.00 | 59.15 | 87.17 | 54.33 | |
BG | 31.38 | 31.21 | 100.00 | 93.43 | 99.04 | 92.65 | |
SG | 14.52 | 14.52 | 0.00 | 29.08 | 0.00 | 29.08 | |
