Figure 1

Schematics of a pair of three-sphere microswimmers with colinear arrangement and Actor-Critic neural network architecture. (a) Schematic of environment setup for reinforcement learning. Each swimmer consists of three rigid spheres with radius R and two extensible arms, and two identical swimmers are arranged colinearly. We distinguish the two swimmers by marking the spheres of the swimmer at the back as red and the spheres of the swimmer at the front as blue. The lengths of the extensible arms are denoted by \(L_{i}\) (i = 1,2,3,4) and the positions of the spheres’ centers are denoted by \({\textbf {r}}_i\) (i = 1,2,...,6). The closest distance between two swimmers is denoted as d, which is defined as the distance between \({\textbf {r}}_3\) and \({\textbf {r}}_4\). (b) The deep neural network has an Actor-Critic structure, in which the Actor-network memorizes and updates the learning policy, and the Critic-network estimates a value function to evaluate the performance of the policy. (c) Schematic showing the transition of the swimmer’s configuration due to its actuation. The swimmer can either extend or contract one of its two links at a step and each swimmer has a total of 4 possible configurations.