Extended Data Table 2 Benchmarking cell embeddings using scIB with default annotations for 144 cell types on the Human Fetal Lung Cell Atlas, the donor split

From: Limitations of cell embedding metrics assessed using drifting islands

Method		Bio conservation							Batch correction					Aggregate score
Method	HVG	I-label	L-NMI	L-ARI	K-NMI	K-ARI	S-label	cLISI	S-batch	iLISI	KBET	G-Con	PCR	Batch	Bio	Total
PCA		0.571	0.809	0.581	0.723	0.237	0.532	1.000	0.834	0.043	0.620	0.821	0.000	0.464	0.636	0.567
PCA	✓	0.581	0.799	0.619	0.716	0.191	0.535	0.999	0.868	0.059	0.640	0.797	0.000	0.473	0.634	0.570
TSNE		0.583	0.762	0.318	0.720	0.160	0.499	1.000	0.542	0.042	0.480	0.663	0.000	0.345	0.577	0.484
TSNE	✓	0.585	0.767	0.350	0.716	0.157	0.504	1.000	0.568	0.059	0.509	0.693	0.000	0.366	0.582	0.496
UMAP		0.580	0.765	0.387	0.713	0.162	0.489	0.999	0.576	0.068	0.572	0.708	0.000	0.385	0.585	0.505
UMAP	✓	0.548	0.771	0.404	0.713	0.162	0.524	0.999	0.586	0.085	0.560	0.707	0.000	0.388	0.589	0.508
Harmony		0.544	0.720	0.339	0.648	0.203	0.490	0.929	0.831	0.144	0.808	0.774	0.291	0.570	0.553	0.560
Harmony	✓	0.487	0.641	0.334	0.554	0.115	0.488	0.927	0.831	0.150	0.675	0.671	0.597	0.585	0.507	0.538
Scanorama		0.487	0.821	0.692	0.723	0.236	0.538	1.000	0.860	0.096	0.739	0.818	0.000	0.503	0.643	0.587
Scanorama	✓	0.564	0.816	0.703	0.725	0.237	0.536	1.000	0.865	0.091	0.744	0.821	0.000	0.504	0.654	0.594
BBKNN		0.413	0.753	0.340	0.703	0.150	0.541	0.927	0.590	0.160	0.785	0.725	0.000	0.452	0.547	0.509
BBKNN	✓	0.573	0.754	0.392	0.697	0.158	0.487	0.931	0.580	0.133	0.669	0.649	0.000	0.406	0.571	0.505
fastMNN	✓	0.415	0.251	0.060	0.228	0.040	0.397	0.984	0.778	0.162	0.101	0.067	0.616	0.345	0.339	0.341
scVI		0.552	0.709	0.369	0.636	0.136	0.522	0.927	0.834	0.139	0.840	0.860	0.415	0.618	0.550	0.577
scVI	✓	0.606	0.724	0.407	0.663	0.142	0.521	0.923	0.838	0.142	0.818	0.849	0.681	0.666	0.569	0.608
scANVI		0.532	0.785	0.559	0.682	0.174	0.540	1.000	0.818	0.137	0.850	0.862	0.154	0.564	0.610	0.592
scANVI	✓	0.597	0.856	0.738	0.736	0.232	0.554	1.000	0.829	0.121	0.834	0.861	0.521	0.633	0.673	0.657
scGen	✓	0.603	0.902	0.756	0.789	0.285	0.609	0.931	0.695	0.144	0.846	0.906	0.138	0.546	0.697	0.636
scPoli		0.462	0.876	0.663	0.802	0.290	0.624	1.000	0.745	0.145	0.869	0.903	0.000	0.532	0.674	0.617
scPoli	✓	0.661	0.879	0.700	0.802	0.313	0.629	1.000	0.739	0.143	0.867	0.899	0.230	0.575	0.712	0.657
Geneformer		0.492	0.640	0.304	0.520	0.107	0.475	0.996	0.829	0.114	0.672	0.624	0.410	0.530	0.505	0.515
scGPT	✓	0.486	0.583	0.225	0.467	0.063	0.445	0.991	0.770	0.165	0.636	0.549	0.482	0.521	0.466	0.488
scGPT (FT)	✓	0.517	0.717	0.342	0.658	0.156	0.518	0.998	0.762	0.130	0.819	0.826	0.585	0.624	0.558	0.584
Author’s		0.575	0.844	0.561	0.774	0.347	0.567	1.000	0.834	0.070	0.780	0.897	0.000	0.516	0.667	0.607
Islander (Tri)		0.624	0.923	0.932	0.822	0.315	0.724	1.000	0.815	0.114	0.825	0.838	0.000	0.518	0.763	0.665
Islander (SCL)		0.625	0.854	0.380	0.852	0.399	0.785	1.000	0.748	0.145	0.792	0.822	0.000	0.501	0.699	0.620
Islander (Run1)		0.818	0.999	1.000	0.901	0.449	0.793	1.000	0.854	0.124	0.889	0.972	0.240	0.616	0.851	0.757
Islander (Run2)		0.824	0.999	1.000	0.891	0.406	0.793	1.000	0.853	0.123	0.883	0.970	0.217	0.609	0.845	0.751
Islander (Run3)		0.817	0.999	1.000	0.894	0.440	0.794	1.000	0.854	0.123	0.888	0.970	0.249	0.617	0.849	0.756

The highest scores for each metric are highlighted in bold. All subsequent tables adhere to the same annotation scheme.

Back to article page

Extended Data Table 2 Benchmarking cell embeddings using scIB with default annotations for 144 cell types on the Human Fetal Lung Cell Atlas, the donor split

Search

Quick links