Extended Data Table 5 Benchmarking cell embeddings using scIB and scGraph with default annotations for 9 cell subtypes of fibroblasts, applied to the fibroblast subset of the Human Fetal Lung Cell Atlas. All methods are re-trained on this subset
From: Limitations of cell embedding metrics assessed using drifting islands
Method | Bio conservation | Batch correction | scIB aggregate score | scGraph - | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
I-label | L-NMI | L-ARI | K-NMI | K-ARI | S-label | cLISI | S-batch | iLISI | KBET | G-Con | PCR | Batch | Bio | Total | ||
PCA | 0.584 | 0.707 | 0.618 | 0.638 | 0.581 | 0.554 | 0.990 | 0.910 | 0.004 | 0.189 | 0.952 | 0.000 | 0.411 | 0.667 | 0.565 | 0.452 |
TSNE | 0.653 | 0.510 | 0.160 | 0.567 | 0.445 | 0.597 | 1.000 | 0.641 | 0.021 | 0.182 | 0.748 | 0.000 | 0.318 | 0.562 | 0.464 | 0.372 |
UMAP | 0.757 | 0.542 | 0.203 | 0.635 | 0.490 | 0.633 | 1.000 | 0.663 | 0.035 | 0.262 | 0.904 | 0.000 | 0.373 | 0.609 | 0.514 | 0.408 |
Harmony | 0.452 | 0.232 | 0.154 | 0.185 | 0.101 | 0.491 | 0.815 | 0.863 | 0.139 | 0.770 | 0.820 | 0.887 | 0.696 | 0.347 | 0.487 | 0.396 |
Scanorama | 0.542 | 0.670 | 0.586 | 0.535 | 0.471 | 0.539 | 0.994 | 0.914 | 0.066 | 0.517 | 0.920 | 0.123 | 0.508 | 0.620 | 0.575 | 0.352 |
BBKNN | 0.659 | 0.470 | 0.178 | 0.527 | 0.408 | 0.580 | 0.994 | 0.669 | 0.076 | 0.388 | 0.897 | 0.000 | 0.406 | 0.545 | 0.489 | 0.075 |
scVI | 0.493 | 0.251 | 0.171 | 0.265 | 0.180 | 0.503 | 0.811 | 0.860 | 0.145 | 0.677 | 0.813 | 0.936 | 0.686 | 0.382 | 0.504 | 0.339 |
scANVI | 0.541 | 0.846 | 0.824 | 0.663 | 0.629 | 0.549 | 0.998 | 0.858 | 0.082 | 0.701 | 0.926 | 0.636 | 0.640 | 0.721 | 0.689 | 0.356 |
scPoli | 0.569 | 0.440 | 0.245 | 0.475 | 0.332 | 0.549 | 0.951 | 0.800 | 0.108 | 0.623 | 0.970 | 0.521 | 0.604 | 0.509 | 0.547 | 0.273 |
Geneformer | 0.486 | 0.340 | 0.149 | 0.121 | 0.082 | 0.504 | 0.940 | 0.892 | 0.081 | 0.542 | 0.677 | 0.558 | 0.550 | 0.375 | 0.445 | 0.378 |
scGPT | 0.455 | 0.147 | 0.046 | 0.110 | 0.051 | 0.473 | 0.773 | 0.764 | 0.161 | 0.551 | 0.710 | 0.589 | 0.555 | 0.293 | 0.398 | 0.259 |
scGPT (FT) | 0.487 | 0.338 | 0.138 | 0.309 | 0.195 | 0.520 | 0.948 | 0.739 | 0.104 | 0.615 | 0.921 | 0.723 | 0.621 | 0.419 | 0.500 | 0.283 |
Islander | 0.822 | 1.000 | 1.000 | 1.000 | 1.000 | 0.803 | 1.000 | 0.896 | 0.073 | 0.811 | 0.943 | 0.000 | 0.545 | 0.946 | 0.786 | -0.036 |
Author’s | 0.753 | 0.569 | 0.220 | 0.723 | 0.615 | 0.647 | 1.000 | 0.683 | 0.069 | 0.474 | 0.946 | 0.000 | 0.435 | 0.647 | 0.562 | 0.421 |