Fig. 8: hcDRQN shows better generalisation to maze length, cue removal, distractors and sensory noise.
From: Hippocampus supports multi-task reinforcement learning under partial observability
a T-maze setup for increased length of the middle corridor, cue removal, distractor cue and random noise. b, Performance decrease over gradual increase of the maze length shows that hcDRQN can handle middle maze length being 32 steps. Black arrows on the X-axis represent maze corridor length of 4 steps utilised during training. c,d, Animal and model performance when cues are removed from the environment. Both hcDRQN (light green) and animal (blue) allocentric navigation are highly dependent on cues while egocentric is not affected by cue removal. On the other hand, hcDQN fails to solve both allocentric tasks. Data represent mean ± SEM (n = 12, pallo = 0.0004, pego = 0.53, two-sided Wilcoxon rank-sum tests). ***: p < 0.001, ns indicates no significant. e When adding a distractor cue, hcDQN drops to chance level while hcDRQN can still solve most of the tasks. Significance stars indicate two-sided independent-samples t-tests across models (p = 2.01 × 10−15). ****: p < 0.0001. f, When adding white Gaussian noise to the cues hcDRQN is more stable and robust when compared to hcDQN. g T-maze setup with full observability. This setup was used to evaluate generalisation capabilities following cue removal (h). h Compared to models with partial observability, all models with full observability show a drop in performance when cues are removed. Data are presented as mean values ± SEM over 10 different initial conditions for (d, e, h) and over 5 different initial conditions for (b, f). Source data are provided as a Source Data file. Icons used in panels a,g are released by OpenMOJI under a Creative Commons Attribution ShareAlike license 4.0. Panel c was adapted from H Malagon-Vina, S Ciocchi, J Passecker, G Dorffner, T Klausberger. Fluid network dynamics in the prefrontal cortex during multiple strategy switching. Nature Comms 309 (2018); reprinted with permission from Springer Nature under a Creative Commons Attribution 4.0 International License.
