Fig. 3: Behavior and representation iteratively improve each other.

a Schematic of representation (left) and agent behavior (right), before (top) and after (bottom) learning the cue-dependent task. Initially, the agent cannot implement the optimal policy at C since a single state cannot incorporate two separate state-action values. After training, the state has been split, and the policy has been learned. b Top, difference in mean firing rates (activity) on A and B cue trials (“splitness”) of the two populations over the course of learning. Activity is plotted as a running mean over 100 trials. Bottom, behavioral performance, shown as the percentage of correct turns over the course of learning. Behavioral performance is plotted as a running mean over 100 trials. c Left inset, neuron index corresponding to a given location in the track. Right, evolution of population activity on B trials over the course of training. Odd numbered neurons are induced on B trials, while even numbered neurons are induced on A trials. Splitters emerge in a “zipper”-like fashion, starting from the part of C closest to the cue zone (neuron 30), and propagating along until the end of the track nearest to the reward (neuron 90). This zippering creates two feed-forward “chains” of splitters, one propagating the memory of cue A, and the other propagating the memory of cue B.