Figure 5

Examples of how loss and accuracy change with epochs during training for CNN4 for properly tuned and improperly tuned CNNs. Loss is measured as cross entropy normalized by its maximum value while the training accuracy is measured by the number of training samples correctly identified at the end of each epoch. Hyperparameters α, λ, and p are, respectively, the initial learning rate, regularization constant, and dropout probability. (a) α = 0.001, λ = 0.2 and p = 0.5 for summers with the test accuracy of 93.3%. (b) α = 0.001, λ = 0.15 and p = 0.5 for winters with the test accuracy of 93.8%. (c) α = 0.01, λ = 0.01 and p = 0.01 for summers with the test accuracy of 25%. (d) α = 0.01, λ = 0.01 and p = 0.01 for winters with the test accuracy of 60%). Several kernel sizes were tried and it was found that 5 × 5 kernel size gives the best validation accuracy and consequently the best test accuracy.