Fig. 2: Comparison of autoencoder learning curve with generalizability of a fully connected network.
From: Autoencoders for sample size estimation for fully connected neural network classifiers
f is a fully connected network with input dimension of 784 and output dimension 10, and g an autoencoder with an input dimension of 784 and a latent space of dimension 3. a The loss of the autoencoder displays a curve split into two phases: the quick phase and the slow phase. b The first derivative of the autoencoder loss function displays a decay phase and a growth phase. c The second derivative reveals a sharp inflection point where the slope changes from sharply decreasing to sharply increasing. d The area-under-the-curve metric on the test set displays a biphasic structure: a rapid growth phase and a slow growth phase. e The first derivative of the AUC curve reveals a rapidly increasing phase followed by a decreasing phase. f The second derivative of the AUC curve reveals an inflection point as a mirror image of the autoencoder loss curve.
