Supplementary Figure 4: Graphical overview of the procedure to assess the performance of NicheNet in predicting ligand activity from expression data.

First, we collected gene expression datasets in which the transcriptional response of cells was assessed after treatment with a specific ligand or ligand combination (blue: differentially expressed target genes). Secondly, we determined ligand activity prediction accuracy, by testing the ability of the model in predicting with which ligand the cells were treated based on the expression data. In other words, we evaluated how well NicheNet prioritizes ligands according to their potential to regulate a set of affected genes. This procedure is based on the following assumption: the better a ligand predicts the transcriptional response compared to other ligands, the more likely it is that this ligand is active. Therefore, we used the ligand-target matrix (purple palette of regulatory potential scores) to calculate feature importance measures (red palette, example measures: Pearson correlation coefficient, area under the receiver operating characteristic curve) for all ligands in every expression dataset separately. These feature importance measures are proportional to the ability of a ligand to predict the observed differential expression (for example, B -> A: feature importance measures indicate how well ligand B can predict the transcriptional response to ligand A). Then, we assessed how well each of these feature importance scores can predict whether a ligand was truly added to the cells (the activity state; green: active). The final model performance in ligand prediction is then the performance of the most predictive feature importance measure.