Table 7 Highlights of the best achieved performance across all data modalities.

From: Understanding action concepts from videos and brain activity through subjects’ consensus

EEG only

Video only

Subjects’ consensus

Fusion

39.78%

TRN: 31.70%

31.70% \(\rightarrow\) 35.67%

46.14%

TSM: 42.33%

42.33% \(\rightarrow\) 43.66%

  1. Subjects’ consensus refers to VIDEO only performance, where models are trained with the additional supervision provided by the consensus predictions inferred from the EEG of the multiple subjects who watched a specific video. Fusion refers to explicit EEG+VIDEO fusion and is thus an upper bound for model performance.
Back to article page