Table 2 Pairwise neuroradiologist interrater agreement and correlation with SHAP-based feature importance

From: Augmenting radiological assessment of imaging evident dementias with radiomic analysis

Question/Field

Mean ρ (95% CI): interrater agreement

ρ: SHAPAD × Med(Rating)

ρ: SHAPOIED × Med(Rating)

Are the hemispheres symmetrical?a

0.06 (−0.042, 0.185)

-

-

Are there prior hemorrhages present?a

0.113 (−0.023, 0.248)

-

-

Are there prior infarcts present?a

0.184 (0.021, 0.398)

-

-

Are there hyperintensities present in the T2 FLAIR sequence?

0.549 (0.497, 0.606)

0.605***

−0.057

Is there asymmetry?a

0.113 (−0.009, 0.211)

-

-

Atrophy/volume loss in the frontal lobea

0.319 (0.186, 0.452)

−0.281*

0.311**

Frontal lobe atrophy

0.515 (0.401, 0.609)

−0.273*

0.298*

Left precentral gyrus

0.263 (0.098, 0.501)

-

-

Right precentral gyrus

0.304 (0.113, 0.513)

-

-

Left anterior insula

0.409 (0.306, 0.527)

−0.248*

0.055

Right anterior insula

0.458 (0.362, 0.569)

−0.287*

0.248*

Left anterior cingulate gyrus

0.444 (0.373, 0.515)

−0.251*

0.166

Right Anterior cingulate gyrus

0.472 (0.39, 0.564)

−0.189

0.155

Left caudate nucleus

0.45 (0.374, 0.55)

−0.295*

−0.084

Right caudate nucleus

0.456 (0.365, 0.579)

−0.281*

−0.093

Atrophy of the substantia nigra

Unable to assessb

-

-

Left hippocampus

0.661 (0.582, 0.732)

0.338**

−0.415***

Right hippocampus

0.685 (0.615, 0.75)

0.596***

−0.555***

Atrophy of the left parahippocampus?

0.118 (0.031, 0.246)

0.072

0.027

Atrophy of the right parahippocampus?

0.111 (0.014, 0.221)

0.101

0.024

Left amygdala

0.604 (0.521, 0.694)

0.539***

−0.337**

Right amygdala

0.703 (0.617, 0.781)

0.496***

−0.589***

Is there mesial temporal lobe atrophy?a

0.507 (0.415, 0.605)

-

-

Mesial Temporal lobe atrophy (rating)

0.622 (0.507, 0.732)

-

-

Left mesial temporal lobe

0.662 (0.609, 0.737)

0.391***

−0.028

Right mesial temporal lobe

0.689 (0.612, 0.767)

0.565***

−0.584***

Is there non-mesial temporal lobe atrophy?a

0.49 (0.367, 0.635)

−0.143

0.215

Non-mesial temporal lobe atrophy (rating)

0.591 (0.426, 0.745)

−0.177

0.229

Left temporal lobe

0.517 (0.368, 0.698)

0.249*

0.105

Right temporal lobe

0.626 (0.468, 0.777)

0.307**

−0.331**

Left lateral temporal lobe

0.476 (0.318, 0.683)

−0.013

0.293*

Right lateral temporal lobe

0.587 (0.414, 0.752)

−0.253*

0.334**

Anterior temporal lobe

0.62 (0.491, 0.754)

-

-

Posterior temporal lobe

0.387 (0.25, 0.516)

-

-

Left middle and inferior temporal gyrus

0.484 (0.304, 0.678)

0.261*

−0.177

Right middle and inferior temporal gyrus

0.574 (0.377, 0.759)

0.384**

0.363**

Left fusiform gyrus

0.451 (0.259, 0.636)

0.075

0.14

Right fusiform gyrus

0.524 (0.347, 0.727)

0.078

0.024

Is there parietal lobe atrophy?a

0.356 (0.287, 0.428)

0.238*

−0.197

Left parietal lobe atrophy (general)

0.498 (0.417, 0.601)

0.192

−0.057

Right parietal lobe atrophy (general)

0.484 (0.397, 0.572)

0.196

−0.196

Is there occipital lobe atrophy?a

0.236 (0.114, 0.377)

0.17

−0.038

Atrophy in the occipital lobe

0.34 (0.158, 0.602)

0.169

−0.038

Atrophy in the brainstem

0.045 (−0.031, 0.159)

Unable to assessc

Unable to assessc

Atrophy in the cerebellum

0.668 (0.565, 0.788)

0.087

−0.053

  1. We analyzed interrater agreement among seven neuroradiologists who independently evaluated specific imaging findings on a test set of 70 cases, excluding any cases marked as “Cannot Assess” by a neuroradiologist. The mean pairwise Spearman correlation coefficient between neuroradiologists, along with its 95% confidence intervals, was estimated using a bootstrapping procedure. These results are also presented graphically in Supplementary Figs. 3 and 4. Additionally, we conducted a Spearman correlation analysis between expert ratings of imaging findings and SHAP-based feature importance. For each case, the median expert rating of each imaging finding, Med (Rating), was correlated with SHAP values for Alzheimer’s disease (SHAPAD) and other imaging evidence of disease (SHAPOIED). These SHAP values represent feature contributions to the model’s probabilities, serving as proxies for model-identified abnormal imaging findings.
  2. * Indicates p < 0.05, ** indicates p < 0.01, *** indicates p < 0.001.
  3. - Indicates fields without mappings to ROIs in the DKT atlas (see Supplementary Table 1).
  4. aRatings were provided as yes/no rather than on a 4-label ordinal scale.
  5. bUnable to assess by raters on the provided sequences.
  6. cUnable to assess correlation due to zero variance.
Back to article page