Figure 2
Grid search reveals optimal rate constants improve performance. Data were generated for six different combinations of target rate constants. A grid search was performed for each of these target pairs, as shown in the different panels (the target rate constants are marked by the green circle in every panel). The dashed lines indicate the approximation where either \({\hat{\alpha }}_s= 1\) or \({\hat{\alpha }}_r= 1\). The intersection of both dashed lines indicates the Elman solution (blue circle). The region of lower loss for all generated data examples centers around the actual values (\(\alpha _s, \alpha _r\)) used to generate the data. This indicates that there is a performance benefit from choosing the correct rate constants that is not compensated for by the recurrent interactions between neurons.
