Fig. 1: Sample stimuli and experimental design.

A Example trials for action and object event stimuli in the video session. For both event types, 2-sec videos were used depicting the movements of a human or a ball. B Experimental design and classification scheme. For both actions and object events, three motion trajectories were used in relation to an animate or inanimate patient (i.e., kick/hit, jump over/bounce over, walk in front/roll in front of an object/person). To test for generalization across animacy within a modality, a classifier was trained on neural activity patterns associated with events from one category (e.g., action videos) and tested on neural activity patterns derived from events of the other category (e.g., object event videos). Training and testing were done in both directions and resulting accuracy maps were averaged. To test for generalization across both animacy and modality, a classifier was trained on events from one category in one modality (e.g., action videos) and tested on the other category in the other modality (e.g., object event sentences). Training and testing were done across combinations of modality and animacy conditions and the resulting accuracies were averaged across training and testing directions.