Supplementary Figure 2: Comparison of methods on the Plasmodium falciparum example (human-subset).
From: Mass spectrometrists should search for all peptides, but assess only the ones they care about
Histograms of MS-GF+ scores (grey) with estimated number of correct PSMs (#target - #decoys, red) and incorrect PSMs (#decoys, blue), 1% FDR cutoff (dashed line) for the search-all-assess-all (all-all) (a), the search-subset-assess-subset (sub-sub) (b, d) and the search-all-assess-subset strategy (all-sub) (c). In panel (a) and (c), the spectra are searched against a human + Plasmodium database (complete search) and in panel (b) and (d) against a human only database. Panel (a) shows PSM scores for both human and Plasmodium, panels (b) – (d) for the human subset, only. The fraction of incorrect PSMs (π0, first mode in the target distribution) is lower in the human + Plasmodium set (all-all, panel a) than in the human subset (all-sub, panel c) indicating that the all-all FDR is too liberal. The 1% FDR cutoff in the sub-sub strategy (panel b) shifted to higher values and this leads to a decrease in the number of subset PSMs found compared to the all-all and all-sub strategy. It also shows that many PSMs are forced on incorrect subset PSMs in the sub-sub strategy (huge first mode of the distribution). Indeed 13553 (30286-16733) spectra matching to Plasmodium targets/decoys in the human + Plasmodium search switched to a human sequence in the subset-search. 1.3% of the sub-sub PSMs above the 1% FDR cutoff switched peptide sequences (panel d orange) as compared to the complete search (all-all and all-sub strategies). They have lower scores than in a complete search and are questionable at best (black and orange boxplots below histogram in panel d). Sub-sub puts an even higher burden on the target decoy approach for the human-subset than for the Plasmodium-subset (Figure 1 in the main manuscript and Supplementary Figure 3) because more high-quality Plasmodium spectra occur in the sample increasing the problem of forced-PSMs. Note, that the results for the human subset are only included to illustrate that poor FDR control of all-all and sub-sub is not due to a specific choice of the subset. Also note that we do not advocate the use of all-sub on all possible subsets and to combine their results.
