Table 6 Overall performance of DNA foundation models and other genomic models in QTL variant effect quantification tasks

From: Benchmarking DNA foundation models for genomic and genetic tasks

 

Model

eQTL

sQTL

paQTL

ipaQTL

AUC

AlphaGenome, output tracks*

0.8029

0.7147

0.7543

0.8644

Sei, hidden states*

0.7561

0.6534

0.6189

0.6071

Sei, output tracks*

0.7497

0.6276

0.6553

0.606

Enformer, hidden states*

0.7744

0.6662

0.6737

0.6919

Enformer, output tracks*

0.7699

0.6174

0.666

0.6587

DNABERT-2

0.5702

0.5795

0.5066

0.4694

NT-v2

0.6091

0.5047

0.5251

0.6019

HyenaDNA

0.6117

0.5531

0.4699

0.448

HyenaDNA-450K, long sequence

0.6027

0.5262

0.5521

0.5093

Caduceus-Ph

0.6492

0.5666

0.5082

0.5678

Caduceus-Ph, long sequence

0.6265

0.5703

0.4649

0.5203

GROVER

0.5896

0.4742

0.4494

0.4759

Cohen’s d

AlphaGenome, output tracks*

1.287

0.7872

0.9824

1.6347

Sei, hidden states*

1.0335

0.553

0.4538

0.4126

Sei, output tracks*

1.0116

0.4936

0.5503

0.4227

Enformer, hidden states*

1.1102

0.6115

0.6028

0.6576

Enformer, output tracks*

1.1085

0.4129

0.5457

0.5691

DNABERT-2

0.2371

0.2825

0.024

−0.0756

NT-v2

0.3956

−0.004

0.0658

0.3837

HyenaDNA

0.3877

0.2018

−0.0768

−0.2048

HyenaDNA-450K, long sequence

0.3605

0.0757

0.2068

0.0733

Caduceus-Ph

0.5484

0.2278

0.0456

0.2324

Caduceus-Ph, long sequence

0.4913

0.2464

−0.1151

0.075

GROVER

0.319

−0.0978

−0.1393

−0.072

  1. All metrics represent the average AUC and Cohen’s d values calculated across three independent test sets, each defined by a distinct group of chromosomes in our nested cross-validation framework. Non-DNA foundation models are annotated with an asterisk (*). Bolded: the top two highest (absolute) performances for each task.
Back to article page