Fig. 1: The hard problem of segmentation in visual perception.

a Cartoon illustration of the problem of perceptual segmentation. An observer’s perception of depth in the Necker cube (left) alternates between two plausible interpretations (right). This is because there are no cues in the image that allow the brain to determine unambiguously the three-dimensional orientation of the figure (provided by the monocular cue of occlusion on the right). b When presented with multiple motion signals in close spatial proximity the visual system must determine whether local samples arise from one or multiple objects. The ambiguity inherent in local motion signals, that is, a family of object motions can yield the same local motion, results in multiple equally plausible interpretations of the visual input, i.e., the vector fields here could arise from the coherent motion of a single surface or the transparent motion of overlapping surfaces. c (left) Example of our textured plaid stimuli. Square wave gratings drifting normal to their orientation (“component directions” - white arrows) were superimposed to form plaid patterns. Plaids could be perceived as either coherent motion in a single, pattern, direction (red arrow) or transparent motion in the component directions. Plaid perception was biased through the addition of a random dot texturing cue. (middle) The region highlighted in yellow is expanded and shown for a sequence of frames separately for coherent and transparent cues. Dot motion in each case is represented by the green and red arrows. (right) The (x,y) positions of the highlighted dots are plotted versus frame number. In the coherent case, all texture drifted in a single direction. In the transparent case, texture moved in the component directions. d Cartoon illustration of our motion segmentation task. Monkeys began each trial by fixating on a small point. After a brief delay, a plaid pattern with a particular type (coherent/transparent) and magnitude (e.g., contrast) of texture cue appeared at the location of the MT RF. Plaids could drift in one of two possible pattern directions on each trial. After stimulus offset, choice targets appeared well above and below the MT RF. Monkeys needed to indicate their plaid perception via a saccade to the appropriate choice target.