Fig. 5: Closed-loop performance in controlling the inferred images.

a Representative feedback images are shown for E01 during trials 28–30 of the first session with the corresponding target categories shown on the top. Due to the figure size, one of every two images is shown. The images underlined in red are the correct decoding in the context of a three-choice task in each frame; because the evaluation of the three-choice task is based on the inferred vector (v_online), some frames have the same feedback images, but are differently classified. b For E01, the trial-averaged z(R(v_online, v_category)), where (v_category: v_word, v_landscape, or v_face), are shown as green, red and blue lines, respectively, with 95% CIs by the shaded area. The title of each panel indicates the target categories for the averaged trials. c The three-choice accuracy at each frame is shown. Dotted line denotes chance level (33.3%). d Power in the high-γ band during the feedback trials for each target category were averaged to be color-coded for each electrode and frame. The black line on the right side of the plot indicates the electrodes in the higher visual area. For visualization, the powers were z-scored across all trials and frames for each electrode. e For each electrode and frame, F-values of one-way ANOVA across the high-γ powers during the three target categories in (d) were color-coded. The black line on the right side of the plot indicates the electrodes in the higher visual area. f PrjR^k (V_online, V_target) (\({V}_{online}:=\{{v}_{online}^{i,\,j}\}\,{{{{{\rm{and}}}}}}\,{V}_{target}:=\{{v}_{target}^{i,\,j}\}\,{{{{{\rm{where}}}}}}\,i=1,\cdots ,\,120\,({{{{{\rm{trial}}}}}});\,j=1,\,\cdots ,\,32\,({{{{{\rm{frame}}}}}})\)) were Fisher z-transformed and averaged across all four subjects to be shown in order of the principal components. Here, PrjR^k (V_online, V_target) was evaluated only for the 14 principal components whose \(\overline{z(Prj{R}^{k}({V}_{inferred},\,{V}_{true}))}\) was positively significant in the video-watching task. The \(\overline{z(Prj{R}^{k}({V}_{online},\,{V}_{target}))}\) was compared with the corresponding chance distribution for each component (k) (\(\overline{z(Prj{R}^{k}({V}_{online},\,\{{v}_{target}^{i,\,j}\}))}\,{{{{{\rm{where}}}}}}\,i\,{{{{{\rm{is}}}}}}\,{{{{{\rm{shuffled}}}}}}\,(1,\,\cdots ,\,120);\,j=1,\,\cdots ,\,32\)). Individual values are shown with dots. *P < 3.6 × 10⁻³ (Bonferroni-adjusted α-level; 0.05/14), two-sided permutation test.