Fig. 7: Applying Rastermap to artificial neural networks.

We visualized activations from QR-DQNs that were trained to play various Atari games⁵². Superneurons were z-scored across the recording; white represents 0, and black represents 2.5. a, An agent trained on the Pong Atari game. Left: example frames from an episode of the agent playing Pong. Middle: activations of units from the agent’s neural network during the episode, sorted by Rastermap. Right: positions in the sorting colored by the layer in the network. b–d, Same as a, for agents trained to play Space Invaders (b), Enduro (c) and Seaquest (d). Blue vertical dashed lines denote the times of example frames illustrated in the left panels.