Extended Data Fig. 4: Further observations.
From: Magnetic control of tokamak plasmas through deep reinforcement learning
a, When asked to stabilize the plasma without further specifications, the agent creates a round shape. The agent is in control from t = 0.45 and changes the shape while trying to attain Ra and Za targets. This discovered behaviour is indeed a good solution, as this round plasma is intrinsically stable with a growth rate γ < 0. b, When not given a reward to have similar current on both ohmic coils, the algorithm tended to use the E coils to obtain the same effect as the OH001 coil. This is indeed possible, as can be seen by the coil positions in Fig. 1g, but causes electromagnetic forces on the machine structures. Therefore, in later shots, a reward was added to keep the current in both ohmic coils close together. c, Voltage requests by the policy to avoid the E3 coil from sticking when crossing 0 A. As can be seen in, for example, Extended Data Fig. 4b, the currents can get stuck on 0 A for low voltage requests, a consequence of how these requests are handled by the power system. As this behaviour was hard to model, we introduced a reward to keep the coil currents away from 0 A. The control policy produces a high voltage request to move through this region quickly. d, An illustration of the difference in cross sections between two different shots, in which the only difference is that the policy on the right was trained with a further reward for avoiding X-points in vacuum.
