Table 1 Model parameters

N_inp

120

–

Number of neurons in external network

N

120

–

Number of neurons in hippocampal network

N_act

2

–

Number of neurons in action network

\({dx}/{dt}\)

1

dv (arb.)

Velocity of agent in maze

L

100

dx (arb.)

Length of maze

σ

5

dx (arb.)

Standard deviation of input Gaussians

τ_e, τ_ea

10, 20

dt (arb.)

Time constants of eligibility traces

τ_a,τ_v

40, 10

dt (arb.)

Time constants of apical compartment, action neurons

σ_v

N(0,0.75)

–

Noise, action neuron activity

σ_p

N(0,5)

–

Noise, policy selection

m_a, m_β

1, 5

–

Stretch coefficient, activation function

c_a, c_β

5, 1.4

–

Offset, activation function

\({{{t}}}_{{{{{\mathbb{choice}}}}}}\)

60

dt (arb.)

Time of turn choice

p_rand

10

%

Chance of random turn

n_trials

4000

–

Number of trials

\({{{{{r}}}}_{{{{0}}}}{{{,}}}{{{r}}}}_{{{{q}}}}\)

0.5,0.6

–

Reward expectation

\({{{t}}}_{{{\mathbb{plateau}}}}^{{{{i}}}}\)

i

dt (arb.)

Time of induced plateau for neuron i

M_ik

δ_ik, U(0,\({1{{{{{\rm{e}}}}}}}^{-4}\)) (S2 only)

–

Weight matrix, input layer to representation layer. Identity matrix (fixed) for all but Supplementary Fig. 2)

W_ij

U(0,\({1{{{{{\rm{e}}}}}}}^{-4}\))

–

Initial values, recurrent weight matrix, representation layer (before learning). Uniform distribution between 0 and \({1{{{{{\rm{e}}}}}}}^{-4}\)

Q_li

U(0,\({1{{{{{\rm{e}}}}}}}^{-4}\))

–

Initial values, weight matrix, representation layer to action layer (before learning). All initial values the same

I_ml

\(\left(\begin{array}{cc}0 & -0.125\\ -0.125 & 0\end{array}\right)\)

–

Recurrent weight matrix, action layer (fixed)

M_max

0.75

–

Upper bound for input weight (Supplementary Fig. 2 only)

W_max

0.15

–

Upper bound, recurrent weights

Q_min,Q_max

−0.15, 0.15

–

Lower and upper bounds, action weights

\({{{{{\eta }}}}}^{{{{{{\boldsymbol{W}}}}}}}\), \({{{{{\eta }}}}}^{{{{{{\boldsymbol{Q}}}}}}}\), \({{{{{\eta }}}}}^{{{{{{\boldsymbol{M}}}}}}}\)

0.0006, 0.0003, 0.15

–

Learning rates, recurrent weights, state-action weights and input weights (input learning for Supplementary Fig. 2 only)

\({{{{\lambda }}}}_{{{{w}}}}\)

0.025

–

Decay constant, recurrent weights