Coupling perception and action using probabilistic control

Abstract

Although there is a long tradition of separating perception, decision making and action, the theoretical conditions for such a separation are very restrictive and not applicable even to the most basic every day tasks in humans. Instead, all uncertainties relating states of the world to observations, actions to future states and rewards, and actions to observations need to be considered in order to act optimally. In the most general formulation this can be formalized as a partially observable Markov decision process (POMDP).We consider the task of intercepting a moving ball for which a multitude of previous studies has shown that humans use a constant bearing angle strategy. Here we manipulated the observation function in a virtual reality setup so as to change the uncertainty of the ballʼs position in parametric ways. Specifically, the contrast of the ball changes as a function of the heading angle towards the ball along the subjectʼs momentary trajectory. Subjects adjusted their interception strategy within an average of 26 trials and are consistently able to catch these balls.For the classic interception task it has been shown that the optimal strategy for constant velocities and certain noise distributions is indeed the constant bearing angle strategy. The complex observation functions used in the human experiments above require modeling the task as a POMDP. We use an approximate solution technique based on Markov Chain Monte Carlo to solve the control task and compare this to the solution in which the system first finds the most likely position of the target and then acts upon this percept. This computational strategy corresponds to certainty equivalent control and is shown to be suboptimal in this task. We conclude that humans are able to carry out complex control tasks that require computing with uncertainties relating sensory input, control variables, and actions.