Fig. 6: Autonomous navigation of a microrobot upstream in a flow environment.
From: Model-based reinforcement learning for ultrasound-driven autonomous microrobots
a, Schematic of the reward function adjusted to promote microrobot navigation close to the wall to minimize drag. b, Graph showing reward progression over time for microrobots in normal (blue line) and stronger (green line) flow conditions, highlighting differences in learning and adaptation. Solid lines represent the EWMAs (α = 0.0015) of the reward. In the normal flow, rewards steadily improve and stabilize around 200,000 steps. In the stronger flow, initial difficulties lead to more negative rewards, but the algorithm shows notable improvement by 400,000 steps. c, Schematic illustrating the behaviour of the microrobot attached to the wall to avoid drag and move against the flow. d, Image sequence showing a microrobot navigating within a microfluidic channel under flow conditions. Initially, when the microrobot was at the centre of the channel (0.0 to 6.7 s), it encountered maximum drag before moving towards the wall (7.8 to 31.4 s), where the drag was minimal. The reduced drag forces at the wall facilitated more stable and controlled navigation. The red arrows indicate the direction of the fluid flow. Scale bar, 200 µm.
