Fig. 4: Bias-variance decomposition of generalization error.

a Average estimator for kernel regression with \(K(x,x^{\prime} )=\mathop{\sum }\nolimits_{k = 1}^{N}\cos (k(x-x^{\prime} ))\) on target function \(\bar{f}(x)={\mathrm{e}}^{4(\cos x-1)}\) with mean subtracted for different values of α = P/N when λ = σ² = 0. Estimator linearly approaches to the target function and estimates it perfectly when α = 1. Dashed lines are theory. b With the same setting in Fig. 3, when λ = 0 and σ² = 0, the bias is a monotonically decreasing function of α while variance has a peak at α = 1/2 yet it does not diverge. c When λ = 0 and σ² = 0.2, we observe that the divergence of E_g is only due to the diverging variance of the estimator. In b, c, solid lines are theory, dots are experiments. Error bars represent the standard deviation over 15 trials.