Fig. 2: Genome understanding performance.

A Line plot showing the relationship between GPU memory usage and input sequence length for OmniReg-GPT and other two latest transformer-based models. The 32GB V100 GPU memory limit is indicated by the dashed line. The 138 million parameter size for OmniReg-GPT refers to the variant with a hidden dimension of 768. B Bar plot depicting the relative computation time ratios of three models across different sequence lengths, with the slowest model at each length set as the baseline (1.0). Note that DNABERT-2 is absent at 50k due to memory exhaustion. C Comparative performance of OmniReg-GPT against DNA foundation models trained on human genome (top panel) and multi-species genome (bottom panel) in predicting histone marks, promoters and enhancers, measured by Matthew’s correlation coefficient (MCC). D, E AUROC scores for fine-tuned multi-classification models (OmniReg-GPT, DNABERT2 and Gena-LM, NT-V2-multispecies(500 M), HyenaDNA-1k) of CpG methylation (n = 7) and histone modification tasks (n = 18) from Bend Benchmark. For each model, individual dots represent AUROC scores for each specific task, while the boxplot height indicates the mean AUROC across all tasks. Specifically, for the CpG methylation tasks, the mean AUROC scores for OmniReg-GPT, NT-V2-multispecies(500 M), DNABERT2, Gena-LM and HyenaDNA-1k were 0.872, 0.858, 0.810, 0.743 and 0.800 respectively; for the histone modification tasks, mean AUROC scores were 0.764, 0.780, 0.739, 0.701 and 0.726, respectively. Statistical significance was assessed by a one-sided paired Wilcoxon test (per-task AUROC pairs, alternative: OmniReg-GPT > baseline; *p < 0.05, **p < 0.01, ***p < 0.001). For the CpG methylation task, the exact p values for the comparisons are: vs DNABERT2 (p = 0.011), vs GenaLM (p = 0.010), vs NT-V2-multispecies (p = 0.011), vs Hyena-1k (p = 0.011). For the CpG methylation task, the exact p values for the comparisons are: vs DNABERT2 (p = 0.0001), vs GenaLM (p = 0.0001), vs NT-V2-multispecies (p = 0.0001), vs Hyena-1k (p = 0.0001). F Bar plot comparing the PR-AUC values of OmniReg-GPT, NT-V2-multispecies (500 M), DNABERT2, Gena-bigbird and HyenaDNA-32k for eQTL variants binary classification tasks across sequence lengths of 2k, 6k, and 10k base pairs. G Precision-Recall Curves plot of OmniReg-GPT, NT-V2-multispecies (500 M), DNABERT2, Gena-bigbird and HyenaDNA-32k for the pathogenic variants binary classification task with length of 10k base pairs. Source data are provided as a Source Data file.