Table 3 Zero-shot results on 6 external datasets

Dataset	Anatomy	Modality	Dim	Method	External Data	F1 ↑	MCC ↑	ACC ↑
Brain-Tumor	Head&Neck	MRI	2D	RadIN	Zero-shot	22.29	3.23	63.63
				BiomedCLIP		56.25	32.81	57.21
				Ours		61.09	47.37	73.38
				S.T.	n=571(10%)	61.77	46.69	72.17
POCUS	Chest	US	Video	RadIN	Zero-shot	2.22	0.17	66.67
				BiomedCLIP		50.33	11.73	45.82
				Ours		65.02	47.97	79.84
				S.T.	n=223(1%)	64.20	48.31	80.58
CT-KIDNEY	Abdom&Pelvis	CT	2D	RadIN	Zero-shot	38.1	17.57	64.75
				BiomedCLIP		49.38	29.58	72.21
				Ours		56.50	43.18	69.96
				S.T.	n=995(10%)	55.73	42.28	66.95
MURA	Limb	X-ray	2D	RadIN	Zero-shot	31.26	4.37	52.94
				BiomedCLIP		68.36	14.96	54.70
				Ours		67.64	40.15	77.51
				S.T.	n=3680(10%)	66.71	41.98	78.32
VinDr-Spine	Spine	X-ray	2D	RadIN	Zero-shot	4.34	0.91	87.49
				BiomedCLIP		26.49	12.79	61.3
				Ours		25.93	19.40	83.07
				S.T.	n=84(1%)	24.73	20.61	84.46
TCGA	Chest&Breast	CT&MG	3D	RadIN	Zero-shot	-	-	-
				BiomedCLIP		-	-	-
				Ours		72.18	57.20	83.47
				S.T.	n=173(30%)	70.33	52.26	77.41

*All indicators are presented as percentages, with the % omitted in the table.
For our model, we avoid exposing these external data for zero-shot experiments and use the threshold corresponding to the highest F1 score on the evaluation set. For RadIN and BiomedCLIP, we use the threshold corresponding to the highest F1 score on each individual external dataset. We also report the supervised training results with partially down-stream task external data, denoting as “S.T.”, for demonstrating what diagnosis performance our model can achieve zero-shotly. We have evaluated F1, MCC and ACC, and all the results reported in the table are the averaged values across all classes. In the table, “US" represents ultrasound and “MG" represents mammography.

Quick links

Search