Fig. 1: Overview of study design.

a Two MAQC samples (A = Universal Human Reference RNA and B = Human Brain Reference RNA), two ERCC synthetic RNA mixtures, and Quartet RNA reference materials were utilized to prepare a set of samples. The M8 and D6 samples were combined with ERCC controls at manufacturer recommended amounts, and then mixed at 3:1 and 1:3 ratios to create samples T1 and T2, respectively. Each sample was prepared with three replicates, and tested by 45 laboratories with distinct protocols and analysis pipelines, resulting in a total of 1,080 libraries and 15.63 Tb of data generated. All 45 laboratories submitted expression data and differential expression calls at gene and transcript levels, while 42 laboratories submitted complete raw sequencing data. DEG, differentially expressed gene. b A comprehensive framework for assessment of real-world RNA-seq data, encompassing assessment of data quality using PCA-based SNR, as well as gene expression profiles and differentially expressed genes by comparing with various ground truths. c A fixed analysis pipeline was applied to all raw data to exclude the influence of the bioinformatic process. Then the relative contributions of experimental factors to inter-laboratory variations were investigated. d High-quality data from 13 laboratories were selected for the benchmarking study, and the performance of 140 differential analysis pipelines composed of two gene annotations, three alignment tools, eight quantification tools following six normalization methods, and five differential analysis tools was compared to explore the sources of variations from the bioinformatics process. SNR Signal-to-Noise Ratio, RMSE Root Mean Square Error, CC Correlation Coefficient, MCC Matthews Correlation Coefficient, TN True Negative, TP True Positive, FN False Negative, FP False Positive, DEG Differentially Expressed Gene. a, c, d included icons created with BioRender.com released under a Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International license.