Introduction

Ribosome profiling (Ribo-Seq) studies have demonstrated widespread translation of short open reading frames (sORFs) outside of annotated coding sequences in eukaryotic genomes1,2, suggesting that the proteome may be much larger than currently annotated in databases such as UniProtKB3,4,5,6. Several such individual sORF-encoded microproteins were experimentally found to be implicated in diverse biological processes across the tree of life such as muscle physiology and cancer7,8,9,10,11,12. Yet, these well-characterized cases represent only a small fraction of the microproteins that could be encoded by translated sORFs13. The translation products of many sORFs may be poorly conserved, of low abundance, or rapidly degraded, leading to uncertainty about their biological significance5,14,15. There is a need, therefore, to identify the sORF-encoded microproteins that exist in the cell and have the potential to perform biological activities.

One systematic approach to identify unannotated microproteins predicted by Ribo-Seq is to search for peptide-level evidence in mass spectrometry (MS)-based proteomics or peptidomics datasets16,17. In the typical case, a sequence database is constructed that consists of a curated protein sequence database (e.g. the UniProtKB human reference proteome18) joined together with a list of putative unannotated proteins (e.g. predicted products of translated sORFs cataloged by Ribo-Seq). This protein sequence database may then be used for analyzing conventional “shotgun” MS proteomics datasets, in which protein samples are digested using a protease, or for analyzing datasets generated by immunopeptidomics experiments, which attempt to identify peptides presented by human leukocyte antigens (HLAs) without requiring protease pretreatment19. In both conventional proteomics experiments and immunopeptidomics experiments, the collected spectra will be generated from peptides derived from both annotated and unannotated proteins in the sample. Confident inference of an unannotated protein detection requires that the peptide uniquely supports an unannotated protein; i.e., that one can exclude the possibility that it derives from a protein in a curated protein sequence database. Detection confidence is generally controlled using a target-decoy approach20, which enables the calculation of a false discovery rate (FDR). The FDR can be set at the level of peptide-spectrum matches (PSMs), peptides, or proteins. Peptides and their inferred proteins passing the thresholds, usually 1% FDR at the peptide/protein level, are reported as detected21. Protein-level MS evidence in a conventional proteomics experiment using trypsin or other proteases indicates that the protein existed in the cell. Immunopeptidomics can be used to validate Ribo-Seq predictions by confirming that an sORF was translated and the processed forms of its translation product was presented by HLA molecules, but cannot establish that the protein was stably present in the cell22.

Despite the promise of shotgun proteomics for rapid and large-scale microprotein identification, the small size, low abundance, atypical sequence characteristics and frequent transmembrane localization of microproteins pose major technical challenges for existing MS pipelines23,24,25,26. For example, it can be impossible to observe multiple unique supporting peptides for microproteins whose sequence is too short to hold multiple cleavage sites, or if only one peptide is within the mass-over-charge range of the spectrometer. Therefore, the guidelines established by the Human Proteome Project27 for MS detection of proteins are difficult to apply fully, and researchers use a variety of ad hoc strategies16. As the field develops and the number of reported microprotein detections grows, there is a need to assess which strategies are most effective for identifying genuine microproteins while minimizing false positives.

In this work, we brought together a group of experts to perform a systematic confidence assessment of previously reported unannotated protein MS detections. We find wide variation in the number of unannotated microproteins reported between different proteomic studies, with few microproteins reported in more than one study. Manual evaluation indicates a division between immunopeptidomics studies and studies using conventional tryptic proteomics: most microproteins reported in immunopeptidomics studies are supported by high-quality PSMs, while most microproteins reported in conventional proteomics studies are supported by only low-quality PSMs and may not represent genuine discoveries. Yet, a subset of microproteins is supported by strong evidence in conventional proteomics datasets, suggesting that more remain to be discovered. We outline advice for increasing confidence in proteomic detection of microproteins as this area of investigation continues to grow.

Results

Reported numbers of unannotated proteins vary greatly between studies

To evaluate the extent to which unannotated proteins can be detected in proteomics data, our group of microprotein researchers assembled in 2023 to conduct a literature search for all papers reporting human unannotated protein detections published between 2019 and 2022. We identified 12 such studies that were published in this time window (Table 1). Seven studies searched for unannotated proteins in conventional proteomics data, while two studies searched for peptides derived from unannotated proteins in immunopeptidomics data, and three studies searched both classes of proteomics data. From each study, we obtained a list of the unannotated proteins reported to be detected (of any length), together with the PSMs supporting these detections (Supplementary Data 1, Supplementary Table 1).

Table 1 Properties of reanalyzed studies
Full size table

A key motivation for initiating this community effort was the large variation in the number of validated unannotated proteins reported between studies, ranging from 628 to 490329 (Fig. 1A, Table 1). The peptides reported in support of unannotated proteins in each study were largely distinct: of 9414 total reported peptides across the considered studies, only 326 (3.5%) were reported in more than one study. For 8 of 12 studies, fewer than 10% of the reported peptides were found in any of the other analyzed studies (Fig. 1B, Supplementary Data 2). The low rate of replication is despite some studies analyzing the same collections of mass spectra, albeit with not fully overlapping databases of sORF sequences (Table 1). We do not interpret the high variability between studies as indicating that most reported detections are false: this high variability among reported detected peptides likely reflects in part the high variability in the size and composition of the sORF databases tested (Table 1)16 and the quantity of proteomic data analyzed, as well as the diversity of cell types examined, MS techniques used, HLA allotypes among the immunopeptidomics studies, and search algorithms. Nevertheless, in the absence of robust replicability to establish confidence, a closer assessment of the strength of evidence provided in each study for their reported detected unannotated proteins is needed.

Fig. 1: Broad variation among studies in reports of unannotated microprotein detection.
Fig. 1: Broad variation among studies in reports of unannotated microprotein detection.
Full size image

A The relation between the number of sORFs used to construct the protein database of each study and the number of sORF-encoded proteins reported detected by MS (Spearman correlation = 0.43, p = 0.2). Whether the sORF database was constructed using a curated list of known sORFs, all possible sORFs from three frame translation of a transcriptome, or a list of ORFs found to be translated using Ribo-Seq or RNC-seq data is indicated. B For each study, the proportion of reported peptides supporting an unannotated protein that are also found by another study in our analysis is shown. The numbers of peptides found in other studies out of the total reported in the study are indicated above the bars. C Proportion of peptides mapping to annotated proteins using the ProteoMapper tool, divided into categories depending on the number of common single nucleotide polymorphism (SNP) differences separating the peptide from the peptide present in the reference protein and whether the annotated peptide is tryptic; i.e., could be generated by cleavage after lysine or arginine. Semi-tryptic peptides (where only one peptide end is tryptic) are grouped with non-tryptic. Peptides from immunopeptidomics experiments were not generated by trypsin digestion and therefore are not classified as tryptic or non-tryptic. Peptides matching currently annotated proteins that were not annotated on UniProtKB/Swiss-Prot in 2016 (i.e., recently annotated proteins) are excluded. D For each study, the proportion of reported peptides supporting an unannotated protein that are also found by another study in our analysis, excluding peptides that match to annotated proteins according to the ProteoMapper tool. Note that most studies have focused on different biological systems, which can limit the overlap. Source data are provided as a Source Data file.

Do reported peptides uniquely support an unannotated protein?

We first assessed whether PSMs reported as evidence for the detection of an unannotated protein may also be attributed to an annotated protein. All the studies in our meta-analysis attempted to exclude potential annotated protein-matching peptides, but different analysis pipelines were implemented that might not have equally accounted for the full space of potential proteoforms of annotated proteins16.

To assess whether some peptides reported to derive from an unannotated protein could potentially be attributed to an annotated protein, we used the PeptideAtlas ProteoMapper30 tool. ProteoMapper takes neXtProt31 reported amino acid variants into account; i.e., it will find matches not just to the reference proteome but to proteins that differ from the reference by one or more variant amino acids. We restricted our analysis to peptides that differed from the reference sequence by at most one single amino acid variant. Given this restriction, 12% of peptides reported to support detection of an unannotated protein (1161 of 9732) also had a putative match to an annotated protein on ProteoMapper, with this rate varying from 0% to 96% across individual studies (Supplementary Data 1).

Recent updates in annotation could potentially explain why some reported peptides mapped to annotated proteins when we conducted this ProteoMapper search in 2023. To evaluate this possibility, we checked whether these annotated proteins were annotated in the 2016 version of UniProtKB/Swiss-Prot18, as all studies in our analysis used protein databases published after 2016 to define their annotated set (Table 1). Only eight distinct annotated proteins matching reported unannotated peptides in 2023 were absent from UniProtKB/Swiss-Prot in 2016, indicating that annotation updates are not a major explanation for peptides reported to support unannotated proteins mapping to annotated proteins.

Peptides reported to support unannotated proteins might also map to annotated proteins if the studies did not account for non-tryptic peptides or protein variants. We therefore divided the peptides mapping to annotated proteins by whether they were perfect matches to the UniProtKB/Swiss-Prot reference protein or differed by one single amino acid variant, and by whether they were predicted tryptic (i.e., peptides that could be generated by cleavage after arginine or lysine residues) or non-tryptic (including semi-tryptic) (Fig. 1C). We note that some peptides in Chong et al. 32. map to both unannotated proteins and common variants of annotated proteins, but since this study used customized databases of annotated proteins reflecting each patients’ sequenced genotypes, these common variants were shown to be absent in the patient samples. Without such a customized database, it is difficult to fully rule out an annotated protein source given the possibility of unknown variants of annotated proteins, especially in cell lines or cancer samples.

For two studies, Prensner et al. 33. and Duffy et al. 34., a substantial fraction of reported unannotated peptides (10% or more) were perfect matches to tryptic peptides in reference proteins. The relatively high rate of matching UniProtKB protein references in Prensner et al. 33. might be explained by either the use of the UCSC RefSeq database to define the set of annotated proteins rather than UniProtKB, which was used by most other studies (Table 1), or not preferentially allocating all shared peptides to the annotated set. For Duffy et al. 34., spectra searches were conducted against custom databases of both annotated and unannotated proteins inferred to be expressed in the specific type of brain tissue or cell based on Ribo-Seq data, while all other studies included the full set of human annotated proteins in their protein database. Likely, annotated proteins not detected by Ribo-Seq may still be present in the sample, leading to peptides from annotated proteins potentially being falsely assigned to unannotated proteins. For two other studies6,35, more than half of reported peptides that mapped to both unannotated and annotated proteins were non-tryptic (Fig. 1C). A peptide with a match to an annotated protein does not uniquely support an unannotated protein detection, even if the match is non-tryptic, as trypsin does not have perfect specificity and can vary in grade, cleavage could have been induced by other proteases (e.g. upon lysing cells and tissues), and protein processing in cells can yield non-tryptic peptides.

Overall, these results indicate a need to consider non-tryptic peptides and possible amino acid variants of annotated proteins to ensure that peptides uniquely map to an unannotated protein. Excluding potential hits to annotated proteins can be done with tools such as ProteoMapper30 or the neXtProt peptide uniqueness checker36, as suggested by the HUPO-HPP MS data interpretation guidelines27, or, ideally, using sample-specific customized protein sequence databases based on sequenced genotypes.

After excluding all reported peptides that mapped to annotated proteins according to ProteoMapper, the general trends we observed for the entire set of reported peptides supporting unannotated protein detections remained: for 8 of 12 studies, at least 90% of reported unannotated peptides were only reported in that study (Fig. 1D). Therefore, we next examined the level of support PSMs provided for claimed unannotated protein detections.

Assessing PSM quality by manual evaluation

To assess PSM quality among literature-reported peptides supporting detection of unannotated proteins, a random sample of PSMs from each study was manually evaluated by a panel of six expert evaluators. A total of 406 PSMs from 12 studies were evaluated (1.3% of total), corresponding to 307 peptides from 204 unannotated proteins. These PSMs were sampled after excluding peptides mapping to annotated proteins or proteoforms (Fig. 1C). Of these 406 PSMs, 155 were evaluated by two evaluators each to enable determination of the overall consistency between evaluators. Additionally, a common set of 10 negative control PSMs was included in each sample, consisting of high-scoring decoy-spectrum matches intended to mimic PSMs that perform relatively well according to algorithms. Each PSM was rated on a scale of 1-5. Full evaluation criteria along with example spectra and explanations of their rating are given in Appendix 1. The PSMs assigned to each evaluator were ordered randomly and the evaluators were not informed as to the source publication of each PSM (Supplementary Data 3).

Agreement among evaluators was generally high. For the PSMs rated by two evaluators, ratings were well correlated (r = 0.82, p < 10-10) (Fig. 2A). Only 14 of 155 (9%) PSM scores differed by more than one point. The negative controls scored consistently poorly (average score of 1.5), as expected. Evaluator ratings were also well correlated (r = 0.74, p < 10−10) with the dot product between the observed spectra and the spectra predicted by MS2PIP (Supplementary Fig. 1)37. Among immunopeptidomics studies, PSMs with peptides that were predicted to bind to MHC molecules by NetMHC38 were rated more highly (n = 71, mean rating 3.94) than those with peptides not predicted to bind (n = 14, mean rating 3.29, p = 0.037 by two-sided permutation test, Supplementary Fig. 2, Supplementary Data 4), consistent with manual evaluation discriminating between true and false discoveries. To investigate consistency between manual ratings and machine learning methods for spectral prediction, we generated predicted spectral libraries for all evaluated PSMs under several models using Oktoberfest (see Methods)39. We observed a moderate correlation between the best spectral angle between the model-predicted and experimental spectra (a measure of spectral similarity) and evaluator rating (r = −0.56, p < 10−10, n = 274, Fig. 2B), suggesting both similarities and differences in how expert evaluators and this spectral prediction method assess PSM quality.

Fig. 2: Expert manual evaluation of literature reported unannotated protein detections in mass spectrometry datasets.
Fig. 2: Expert manual evaluation of literature reported unannotated protein detections in mass spectrometry datasets.
Full size image

A Counts of each pair of ratings among the PSMs that were assessed by two evaluators (n = 155). The Pearson correlation between pairs of ratings is indicated. B For a set of manually evaluated PSMs (n = 274), the spectrum was also predicted using several machine learning models (see “Methods”). The spectral angle is an indicator of how different the observed PSM was from the closest predicted spectrum, with larger angles indicating a worse match. The best spectral angles are indicated among PSMs grouped by evaluator rating. The box in each boxplot indicates interquartile range between the first and third quartiles, while the center line indicates the median. The whiskers indicate minima and maxima within 1.5 times the interquartile range. C Mean ± standard error of ratings of PSMs sampled from each study, per each of six evaluators (n = 620 rated PSMs in total). Standard errors were corrected for finite population (total count of reported PSMs supporting unannotated proteins in the study). Ratings were given on a 1–5 scale. D Overall distribution of ratings for unannotated protein PSMs among all studies and evaluators (n = 620 PSMs). Bars indicate proportions ± standard errors. E Log Ribo-Seq read counts for ORFs expressing proteins in PSMs rated highly (rating > 3, n = 65 proteins) or lowly (rating < 3, n = 105 proteins). Reads are from a collection of human Ribo-Seq studies (see Methods). The box in each boxplot indicates interquartile range between the first and third quartiles, while the center line indicates the median. The whiskers indicate minima and maxima within 1.5 times the interquartile range. Differences between group means are tested using a two-sided permutation test. F Predicted lengths of proteins rated highly (>3,n = 65 proteins) or lowly (<3, n = 105 proteins). Box plot meaning is same as above. Differences between group means are tested using a two-sided permutation test. G Evaluated and extrapolated counts (±SEM) of HLA and non-HLA high-rated (rating of 4 or 5) protein detections. Extrapolated counts give the number of high-rated protein detections expected if the entire dataset had been evaluated. Source data are provided as a Source Data file.

There was also a general consistency between evaluators in average rating per study (Fig. 2C). The evaluated PSM quality varied across studies, with average rating ranging from 1.0 to 4.1 (Fig. 2C). Three studies had average PSM ratings that did not exceed the negative controls. For one of these studies, van Heesch et al. 6, the authors recognized the high FDR in their search results, which led them to develop a customized strategy for estimating a microprotein-specific FDR and to favor selected reaction monitoring (SRM) for their downstream analyses. We did not evaluate these SRM results but focused solely on the reported shotgun proteomics hits. For Douka et al. 40., the low ratings are understandable because, rather than using a 1% FDR threshold, this study used a 10% threshold in anticipation of the low abundance of microproteins. For Chothani et al. 4, unannotated protein PSMs were identified by searching hundreds of MS runs individually with a 1% FDR threshold after removing all matches to the annotated proteome, then assembling the hits into a master list. A likely explanation is that, since spectra matching annotated proteins were removed prior to searching for unannotated proteins, there were few genuine detections in the MS runs analyzed. Under conditions of few genuine detections, it is difficult to precisely estimate FDR, leading to potential false positives (Supplementary Fig. 3)41. Chothani et al. highlighted peptides found in multiple datasets; these peptides were not separately evaluated here.

The immunopeptidomics studies (Ouspenskaia et al. 29, Martinez et al. 42, and Chong et al. 32, and some peptides from Prensner et al. 33) reported substantially higher quality PSMs than most of the other studies (mean rating 3.8 vs. 2.3, n = 13, p = 0.024 for difference in mean by two-sided permutation test, Fig. 2C, D). The three studies that focused on HLA data have average scores above three, as do the HLA PSMs (but not non-HLA PSMs) from Prensner et al. 43. The only non-HLA studies with average scores of three or more were Cao et al. 44 and Bogaert et al. 28, which reported only 28 and 8 PSMs derived from unannotated proteins, respectively (Fig. 2C and Table 1). Overall, most (70%) evaluated PSMs supporting unannotated protein detections from HLA studies received a rating of at least 4, the threshold for convincing evidence of detection (See Appendix, Fig. 2D). In contrast, only 15% of ratings for reported matches in non-HLA data were in the 4-5 range. These results are consistent with a recent study, Deutsch et al. 45, where MS searches for peptide-level evidence supporting Ribo-Seq identified sORFs also found higher support in HLA than non-HLA datasets45.

Among 98 high-rated HLA peptides, 33 were reported in multiple studies, and 37 were validated by Deutsch et al. 45 (1 supporting an ORF in Tier 1A, 26 in Tier 1B, and 10 in Tier 2B, Supplementary Fig. 4). Of the 28 high-rated PSMs from non-HLA data, two involved peptides that were reported in multiple studies. Both peptides derive from the same sORF, located in the 5’ UTR of the MKKS locus. The protein encoded by this sORF (UniProt identifier Q9HB66 in UniProtKB/TrEMBL) has now accumulated enough peptide-level evidence to have become annotated as “core canonical” in PeptideAtlas in 2025, though it remains unannotated in UniProtKB/Swiss-Prot so far. Two high-rated non-HLA peptides were also identified as having strong evidence in Deutsch et al45. These peptides mapped to the sORFs c11riboseqorf4 in the Tier 1A class (the highest level of support that an ORF is protein-coding) and c12norep33 in the Tier 2A class (weaker support). These observations illustrate how searching multiple sources of MS data contributes towards a more comprehensive view of sORF-expressed proteins and improves annotations of the human proteome.

Higher rated PSMs are derived from more highly expressed sORFs

To assess whether our PSM ratings were influenced by the expression levels of the corresponding proteins, we compiled a large collection of human Ribo-Seq studies and analyzed translation levels harmoniously, using the iRibo program, for all the sORFs corresponding to evaluated PSMs for which genomic coordinates were provided by the original studies (191 sORFs; see “Methods”, Supplementary Data 5, 6)46. We found that reported unannotated proteins with corresponding PSMs rated 4 or 5 were more highly translated than those with corresponding PSMs rated 1 or 2 (difference in log Ribo-Seq read count per codon by two-sided permutation test, p = 0.005, Fig. 2E). This is consistent with more highly expressed proteins being more readily detectable by MS and thus generating higher quality PSMs47. Unexpectedly, high-rated proteins were also shorter on average by 37 amino acids than low-rated proteins (two-sided permutation test, p = 0.01, Fig. 2F). There was no significant correlation between log iRibo p-value, indicating level of confidence that the ORF is translated, and PSM rating (r = 0.098, p = 0.18).

Discovery of potential unannotated proteins

We next estimated the number of unannotated proteins we would expect to have strong MS support had we evaluated all reported detections. To do this, we extrapolated the number of unannotated protein detections that would be supported by high-scoring PSMs had we evaluated all PSMs among all studies, assuming the frequency of scores for each study would be the same as in the tested set (Fig. 2G). Among unannotated proteins reported in non-HLA data, 27 evaluated proteins were supported by at least one PSM rated 4 or 5. We predict 137 of 1749 (7.8%) would be supported by PSMs of this quality across the whole aggregated dataset. For HLA data, 94 evaluated proteins were supported by at least one PSM rated 4 or 5; we predict 3706 of 5705 (65%) would be found across the entire dataset. Other unannotated proteins are likely detectable in datasets outside our study scope. Thus, there is considerable potential for discovery even in the particularly challenging case of finding unannotated proteins in conventional enzymatically digested samples.

Discussion

Given the growing recognition of the importance of microproteins in human health48, there is an urgent need to prioritize sORF-encoded microproteins that are supported by MS evidence. Here, we reanalyzed twelve published studies that reported detection of unannotated microproteins with MS. While most reported PSMs (70%) in immunopeptidomics studies were of high quality, around 85% of non-HLA PSMs were evaluated by a panel of proteomics experts to be of too low quality to provide evidence of peptide detection. These results point to a need for caution in interpreting claimed unannotated protein detections reported in the literature and motivate technological improvements for the evaluation of microprotein evidence moving forward. Many unannotated protein detections do appear strong, and the microprotein literature has provided great value in expanding the protein universe with real discoveries of likely biological significance45. However, the idea that several hundreds to even thousands of unannotated proteins are genuinely detected in existing mass spectrometry datasets of conventional trypsin digests reflects an unrealistic expectation about the extent to which current shotgun proteomics can validate sORFs identified by Ribo-Seq.

Why do immunopeptidomics studies identify many high-quality PSMs supporting unannotated protein detections while studies using conventional enzymatic digests identify only few? Many unannotated sequences found to be translated by Ribo-Seq lack signatures of evolutionary conservation and may not encode proteins that provide any benefit to the organism5,15,49. It is plausible that many of these poorly conserved proteins are expressed but quickly degraded, and so can be found only as peptides bound to HLAs14,50. However, there are also technical explanations for why HLA-bound peptides derived from unannotated microproteins may be easier to detect. Immunopeptidomics concentrates peptides bound to HLAs, which decreases sample complexity and may thereby enrich for low abundance microproteins. HLA peptides also have physical and chemical properties different from tryptic peptides that may affect detectability. Most immunopeptidomics datasets are from cancer samples, and some proteins may be expressed in some cancers but not in normal physiological conditions. Furthermore, microproteins may preferentially reside in cellular compartments that are hard to sample through non-HLA MS, such as membranes26. Moreover, the laboratories that perform immunopeptidomics are often distinct from those that analyze non-HLA data and may differ in their sample preparation techniques, experimental setup, and analytical choices. Understanding which factors are most important to explaining the difference between immunopeptidomics and conventional shotgun proteomics may require the development of more sensitive proteomic techniques for identifying low-abundance and short-lived microproteins in the cell.

Why do several studies report low-quality spectra despite controlling FDR at 1%? Most of the studies we evaluated control only the proteome-wide FDR instead of controlling FDR for unannotated peptides or proteins specifically (Table 1)17,23,51. Since the proteome-wide FDR does not imply any particular FDR among unannotated proteins17,23, it does not imply high confidence in the unannotated list specifically. In a theoretical example experiment in which 1 million PSMs, 50,000 peptides and 10,000 proteins pass threshold, a 1% FDR corresponds to 10,000 incorrect PSMs, 500 incorrect peptides, or 100 incorrect proteins. If the analysis purports to detect 50 sORFs, the default assumption should be that these are mostly incorrect identifications until very carefully scrutinized. Studies that controlled FDR for unannotated proteins in a class-specific manner, such as Chong et al. 32. and Ouspenskaia et al. 29, scored high in our evaluations. We recommend that studies of the unannotated proteome report local or class-specific unannotated FDRs instead of, or in addition to, whole proteome FDRs, so that confidence in the list of reported unannotated proteins can itself be evaluated. To facilitate future work on the detection of unannotated microproteins by MS-based proteomics, we developed a set of guidelines based on our findings (brief advice in Box 1, detailed guidelines in Appendix 2). The guidelines in Appendix 2 are an extension of the Human Proteome Project Mass Spectrometry Data Interpretation Guidelines 3.027. It is important to note that false positives can occur across the full range of PSM quality; a low-quality spectrum does not prove that a claimed detection is a false positive; nor is a high-quality spectrum conclusive evidence of detection. The gold standard for rigorous MS-based proteomics data validation requires demonstration that a synthetic peptide generates the observed spectrum and is retained on the liquid chromatography column to the same extent as the originally detected peptide, and that the endogenous spectrum is eliminated when the ORF is disabled genetically. Supporting evidence for the biological significance of a protein with inconclusive MS support can also come from outside proteomics, such as by demonstrating the evolutionary conservation of its amino acid sequence or reporting phenotypic impacts upon genetic perturbations23,45.

The thousands of sORFs identified by Ribo-Seq experiments suggest a massive potential for undiscovered microproteins of biomedical relevance, even at low proteomic validation rates. While our community assessment found relatively low proteomic support for these microproteins in the datasets generated by the pioneering studies we analyzed, this finding should not be interpreted to mean that only few sORF-encoded proteins are present in the cell. There are major technical limitations in the ability of proteomic experiments to find short and low-abundance proteins16,23,25, and the microproteins field is still in its infancy. The extent to which sORFs encode stable functional proteins thus remains an open question. To answer it, we will need to expand the limits of protein detectability through further methodological developments, including but not limited to improving the sensitivity of MS instruments. We hope the dataset of 406 manually curated PSMs generated here will prove useful for benchmarking much-needed new data analysis tools and pipelines for unannotated microprotein detection by MS (Supplementary Data 3).

Methods

Study selection

We conducted a search for all studies published in the 2019-2022 period that attempted to detect unannotated proteins using shotgun proteomics. For each study, we obtained information on the PSMs claimed to support each reported unannotated detection (Supplementary Data 1). For each PSM, we collected the information needed to construct a universal spectrum identifier (USI)52 so the PSM could be visualized. Where possible, we obtained the PSM data from the supplementary information provided with the study; otherwise, we attempted to obtain them from the study authors. The sources of data for each study are given in Supplementary Table 1. The authors of one study (Cai et al.)53 were unable to provide the necessary data so this study was not evaluated.

The set of “unannotated” proteins depends on the annotation database used; the proteins included in our analysis followed the definition used in each study. Unannotated proteoforms of annotated proteins were not included.

ProteoMapper analysis

All reported unannotated peptides were submitted to the ProteoMapper online tool30 in July 2023 using default settings. For each peptide, ProteoMapper returns a list of matches to known or predicted proteins, accounting for neXtProt31 amino acid variants. We determined whether each peptide mapped to a human annotated protein according to the 2023 build of the PeptideAtlas database54 and whether each peptide mapped to a protein present in the 2016 version of UniProtKB/Swiss-Prot18. Any peptide that mapped to a core canonical PeptideAtlas protein on ProteoMapper was not passed on for manual evaluation, even if it differed from the reference sequence by multiple neXtProt amino acid variants.

Manual evaluation of PSM quality

PSMs for each study were evaluated by a group of six expert evaluators. Each evaluator rated a random sample of PSMs from each study. A total of 424 PSMs from 12 studies were given for evaluation, out of which 406 were given ratings, as a few PSMs could not be displayed from the input USI. Out of the 406 PSMs evaluated, 155 were evaluated by two evaluators each to enable determination of the overall consistency between evaluators. Evaluations were done by visual inspection of the PSM using the ProteomeCentral USI web application (https://proteomecentral.proteomexchange.org/usi/) in May to June 2023. The evaluators were told to use no other information except the PSM as displayed on the USI application. A common set of 10 negative control PSMs was given to each evaluator; the evaluators were not informed of the existence of these controls. These negative controls consisted of high-scoring decoy-spectrum matches manually selected from among the strongest 30 decoy-spectrum matches in Duffy et al. 34. Each PSM was rated on a scale of 1-5; the rating scale is given in Appendix 1.

Comparing manual evaluations to spectral prediction machine learning methods

Spectra were predicted for each manually evaluated peptide sequence annotated to the set of experimental spectra using the open-source spectral library prediction pipeline Oktoberfest39. Multiple predicted spectra were generated for each peptide at various collision energies (CE = 25, 30, 35 and 40) and using 4 different intensity models (Prosit 2020 intensity HCD55, Prosit 2020 intensity CID, Prosit 2020 intensity TMT, AlphaPept ms2 generic)55,56,57,58. Only methionine oxidation, cysteine carbamidomethylation, and TMT6plex modifications were considered in the spectral predictions; peptides with other modifications were excluded for this analysis. MSP spectral library files output by Oktoberfest were then converted to MGF formatted spectra. Internal python scripts compared the experimental spectra vs. the predicted spectra by calculating spectral angles (SA) between each spectral pair. Similarity was ranked as being high if SA ≤ 20°, moderate if SA between 20°–45°, poor if SA between 45°–70°, and terrible if SA > 70°. The script further generated mirrored plots for each spectral pair and annotated peptide fragment ions. These spectral angles were then compared to the manual ratings for each PSM given by the evaluators.

Predicting HLA binding for immunopeptides supporting unannotated protein detections

For each evaluated immunopeptide from Ouspenskaia et al. 2021, Martinez et al. 42, or Chong et al. 32 used to support an unannotated protein detection, the HLA alleles for the cell type used in the experiment producing the peptide was found in the supplemental data of the study. NetMHC 4.0 was then used to predict binding of the peptide to the HLA-A, HLA-B, and HLA-C allele if the allele was available in NetMHC 4.0. A peptide was classified as being HLA-binding if it met the default criteria for being a weak (% rank <2%) or strong (% rank < 0.5%) binder in NetMHC 4.0.

Relating ORF properties to the probability of detection

The coordinates of each ORF with an evaluated peptide were taken from the supplementary data of each study and the ORF length determined. All ORF coordinates were converted to hg38 coordinates using LiftOver. ORFs from Chen et al. 35, Chong et al. 32., Cao et al. 44., and Lu et al. 59. were not considered because we were not able to identify the ORF coordinates from supplementary data files. To assess translation levels, we aggregated Ribo-Seq data from 109 studies (Supplementary Data 5-6) using the following procedure. Transcriptomes from MiTranscriptome60, FANTOM5 robust set60, CHESS61, RNA Atlas62, and Ensembl version 108 were merged using Stringtie63 version 2.2.1with Ensembl version 108 as the reference annotation (-G parameter). MiTranscriptome and FANTOM5 coordinates were lifted over from hg19 to hg38 prior to merging. Adapters in each ribo-seq run were removed with TrimGalore version 0.6.7 using default options. Trimmed Ribo-seq reads were then mapped to the merged transcriptome using STAR64,65 version STAR-2.7.10b using the parameters--outSAMtype BAM Unsorted --outFilterMismatchNmax 2 --outFilterMultimapNmax 1 --outSAMattributes Standard. The iRibo program46 was then used to aggregate the mapped reads from all studies and assign counts of ribosome P-sites to each position of each analyzed ORF.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.