Fig. 3: Three selected experimental results of autonomous EM manipulation, where the demanded tasks from a user’s voice commands.

a ‘please check Alice’s breathing status’ : (Top) the graph representation of planned subtasks, where two subtasks are organized in a sequential manner. For each subtask, the action function, SPM device, and the required semantic coding pattern are marked. For subtask-1, 2.4 GHz SPM and 20 random coding patterns are assigned for reconstructing the 3D human skeleton through the microwave imaging strategy; for subtask-2, 5.5 GHz SPM and a focusing coding pattern are assigned for remotely detecting the subject’s breathing rate via the microwave sensing technique. (bottom) the executed results corresponding to planned two subtasks, i.e., the Alice’s location has been marked with red stars in the reconstructed human skeletons for subtask-1, while a 15 s-length respiration rate is plotted for subtask-2. b ‘my cell phone cannot get a signal, please help me’ : (Top) the graph representation of planned three subtasks. For subtask-1, the router can be easily localized by looking up the knowledge map or visual-semantic map, and thus there is no need to use the SPM; for subtask-2, the user’s cell phone is localized via the beam-scanning strategy, where 3180 focusing coding patterns are involved; for subtask-3, the wireless link between the router and the user’s cell phone is established and enhanced by using a 2.4 GHz SPM. (bottom) the executed results corresponding to planned three subtasks: the microwave localization of the user’s cell phone for subtask-2, and a 0.25 s-length wireless signals at the subject for subtask-3. One can see that the wireless signal of the user has been remarkably enhanced. c ‘robot A, please see me’ : (left) the graph representation of planned five subtasks. For subtask-3 and 5, they share the same semantic coding pattern for the enhanced wireless communication between the router and robot_A. Besides, for subtask-5, it involves the information transfer from metaAgent to robot_A, which the transferred information is about the user’s localization. (right) the executed results corresponding to four subtasks: microwave localization of robot_A for subtask-2, microwave reconstruction of user’s 3D skeleton for subtask-3, a 0.25 s-length wireless signals at the subject for subtask-3, and the constellation of wireless communication between robot_A and router