Introduction

The number of patients with chronic kidney disease (CKD) increased to 3.73 million worldwide in 2016. With an annual growth rate of 5–6%, this figure may reach 5.4 million by 20301,2. In Japan, approximately 340,000 individuals are maintained on renal replacement therapy for end-stage kidney disease. Diabetes mellitus (DM) is a leading cause of CKD, with significant albuminuria and proteinuria being strong predictors of progression to end-stage kidney disease3,4. Consequently, early diagnosis, understanding of the disease pathogenesis, and timely therapeutic interventions are imperative to address this major health challenge. Several studies have been conducted to reveal early pathological changes in diabetic kidney disease5,6,7,8,9. Our previous study showed that even before the onset of albuminuria or at the stage of microalbuminuria, characteristic lesions of diabetic kidney disease, such as nodular lesions and mesangiolysis, can already be detected and are strong prognostic factors for end-stage kidney disease5. Although the early stages of diabetic nephropathy should involve certain pathological changes, these have not been fully elucidated. Therefore, detecting subtle, novel, and early pathological changes on light microscopy in the kidneys of patients with diabetes without clinically overt diabetic nephropathy is important.

To detect the early pathological changes in the kidneys of patients with diabetes without diabetic nephropathy, numerous glomeruli need to be collected and analyzed; however, this is difficult. A kidney biopsy specimen usually obtains only 10–30 glomeruli. However, nephrectomized kidneys due to kidney cancer or other reasons contain several hundred glomeruli11. Numerous glomeruli need to be compared and analyzed to determine the changes in diabetic kidneys. Although this is time consuming if manually performed, machine learning can be employed for this task. Machine learning techniques can compare thousands of glomerular images at once12,13 and involve both supervised and unsupervised learning. Preparing the correct labels is essential for supervised learning, whereas unsupervised learning does not require correct labels14. As the subtle, novel, and early pathological changes on light microscopy are unknown, unsupervised learning techniques are suitable for this task. Invariant information clustering (IIC) is an unsupervised learning clustering technique that uses two convolutional neural networks (CNNs)15. The features underlying this clustering can be visualized using gradient-weighted class activation mapping (Grad-CAM)16 and generative adversarial networks (GAN)17. These visualization techniques provide cues to identify the subtle pathological changes on light microscopy in the kidneys of patients with diabetes.

In this study, we performed IIC clustering on glomerular images obtained from nephrectomized kidneys.

Results

Clinical background of nephrectomized kidneys

Of the 45 nephrectomized kidneys, 24 and 21 were from patients with and without diabetes, respectively. The clinical background of each patient group is shown in Table 1. The median age of the diabetes and non-diabetes groups was 67 and 71 years, respectively. The proportion of men was 66.7% in the diabetes group and 47.6% in the non-diabetes group. The median duration of diabetes was 6.5 years. HbA1c levels of the diabetes and non-diabetes groups were 6.9% and 5.7%, respectively. The prevalence of hypertension was 75% in the diabetes group and 76.1% in the non-diabetes group. Median serum creatinine (Cr) levels were 0.87 mg/dL in the diabetes group and 0.80 mg/dL in the non-diabetes group. The median eGFR levels were 66.0 ml/min/1.73m2 and 58.0 ml/min/1.73m2, respectively (Supplementary Table S2). Overall, 13,251 glomerular images were used in this study. Advance pathological check confirmed that few lesions of advanced diabetic nephropathy, such as nodular lesions and mesangiolysis, were seen in specimens of these 45 cases. The maximum number of glomerular images obtained from one case was 1,166, and the minimum was 30 (Supplementary Table S2).

Table 1 Clinical background of patients whose kidneys were used in this study.
Full size table

Image clustering by IIC

All glomerular images were used for clustering with IIC. Clustering was performed by sequentially increasing the number of clusters in a step-by-step manner. When the number of clusters was 9 or 11, no image was detected in some clusters (Fig. 1). However, when the number of clusters was 10, all clusters contained ≥ 16 images.

Fig. 1
figure 1

The number of glomerular images per cluster as the number of clusters increased step-by-step manner. Blue bars represent the number of images from diabetic cases, and orange bars represent the number of images from non-diabetic cases. When the number of clusters was nine or eleven, no image was detected in some clusters. But, when the number of clusters was ten, all clusters contained more than sixteen images.

Full size image

When the 13,251 glomerular images were clustered into 10, each cluster contained between 16 and 3,969 glomerular images. The proportion of images from diabetes cases in each cluster ranged from 32 to 100% (Fig. 2A). Each cluster was named according to the percentage of images from diabetes cases (from Cluster 0 to Cluster 9). The images in Cluster 0 were all from diabetes cases; however, this cluster contained only 16 images. Cluster 1 contained 67 images, and 99% were from diabetes cases. Cluster 2 contained 221 images, and 97% were from diabetes cases. Cluster 9 contained 2,639 images, and 32% were from diabetes cases. As shown in Fig. 2B, diabetic clusters that mainly contained glomerular images from diabetes cases (Clusters 0, 1, and 2) and non-diabetic clusters that mainly contained glomerular images from non-diabetes cases (Clusters 8 and 9) were distinguished in the t-SNE analysis.　In Cluster 1, a cluster mainly contained glomerular images from diabetes cases, cortical and juxtamedullary glomeruli were nearly equal in number (cortical glomeruli: juxtamedullary glomeruli = 47%: 53%). Similar to the HE-stained images, the PAS-stained images also showed several clusters with a majority of glomeruli from diabetes cases and the other clusters with a majority of glomeruli from non-diabetes cases. The percentage of images of diabetic cases in each cluster (PAS) ranged from 30 to 89% (Supplementary Figure S2A).

Fig. 2
figure 2

Visualization of clustered glomerular images by IIC and t-SNE. (A) shows the percentage of glomeruli obtained from diabetes cases in each cluster. The 13,251 glomerular images were clustered into 10 clusters by IIC. Blue and orange bars represent the percentage of glomeruli obtained from diabetes and non-diabetes cases, respectively. Numbers above the bars represent the number of glomerular images included in each cluster. (B) shows the results of dimensionality reduction which was performed using t-SNE. Each cluster was color coded as follows: cluster 0, light green; cluster 1, light blue; cluster 2, yellowish green; cluster 3, pink; cluster 4, green; cluster 5, orange; cluster 6, black; cluster 7, red; cluster 8, purple; cluster 9, blue. Clusters 0, 1, and 2 mostly contained glomerular images from diabetes cases, and clusters 8 and 9 mostly contained glomerular images from non-diabetes cases. The number of glomerular images from each patient in each cluster is shown in (C) and (D). Correlations between glomerular images from patients and clusters are illustrated by a Sankey diagram. (C) shows diabetes cases, and (D) shows non-diabetes cases. The numbers on the left and right sides of each diagram represent the case number and cluster number, respectively. The thickness of the line connecting the cases and clusters represents the number of glomeruli in each case and cluster. The glomeruli from diabetes and non-diabetes cases were highly shuffled and clustered.

Full size image

Correlations between glomerular images from patients and clusters are presented in a Sankey diagram. The thickness of the line connecting the cases and clusters represents the number of glomeruli in each case and cluster. As shown in Fig. 2C and D, the glomeruli from diabetes and non-diabetes cases were highly shuffled and clustered.

Depiction of feature areas using Grad-CAM in representative clusters

Grad-CAM was used to visualize where a CNN model was looking. Figure 3 shows images of representative clusters in pairs with the Grad-CAM and original images. Grad-CAM revealed that essential regions for clustering were highlighted in red or orange. In images from Cluster 1, the outer portions of the glomerular capillary were colored red (Fig. 3A). High-magnification original images were placed in the lower right corner of each panel to indicate that podocytes on the outer portions of the glomerular capillaries were highlighted by Grad-CAM. Cluster 2 images also showed that glomerular capillaries were highlighted in the same areas as those in Cluster 1. Additionally, as shown in Fig. 3B, Cluster 2 images showed that some mesangial areas or capillaries inside the glomeruli were highlighted by Grad-CAM. These portions were characteristic regions of Cluster 1 and 2 images. Moreover, images from Cluster 1 (Fig. 3A, Supplementary Figure S1B) and Cluster 2 (Fig. 3B, Supplementary Figure S1C) had larger Bowman’s spaces and tubular lumens than those in Cluster 9 (Fig. 3C, Supplementary Figure S1D). In both training and validation, the loss function curve decreased, and the accuracy curve increased as progressed the learning epochs of supervised learning conducted to apply Grad-CAM (Supplementary Figure S3B). In Cluster (PAS) 1, which contained the second most images of diabetic cases in the clustering of PAS-stained images, the outer portion of the glomeruli were highlighted (Supplementary Figure S2B).

Fig. 3
figure 3

Visualizations of regions of interest by Grad-CAM for clustering. Clusters 1 and 2 mostly contained glomerular images from diabetes cases. Representative images of clusters 1 and 2 are in pairs with the Grad-CAM and original images (A and B, respectively). Grad-CAM revealed that regions for clustering are highlighted in red or orange. Glomerular images from cluster 9 are also shown (C). As shown in images of cluster 1 (A), the outer portion of the glomerular capillary was red on Grad-CAM. High-magnification original images at the lower right corner of each panel shows that podocytes on the outer portion of the glomerular capillary were positive for Grad-CAM. In cluster 2 (B), in addition to areas positive for Grad-CAM were similar to those in cluster 1, there were also areas in the mesangium and/or capillaries inside the glomeruli that were positive for Grad-CAM. Moreover, images in clusters 1 and 2 showed larger Bowman’s spaces and tubular lumens than those in cluster 9 (C).

Full size image

Comparative study using Cycle-GAN

To further determine the differences in the images between clusters, Cycle-GAN generated images similar to those in Cluster 1 or 2 from Cluster 9 images (Fig. 4A). Conversely, Cycle-GAN generated images similar to those in Cluster 9 from Cluster 1 or 2 images (Fig. 4B). In the generated images similar to Cluster 1 or 2, the outer glomerular capillaries were effaced, and Bowman’s space was enlarged (Fig. 4A). In contrast, glomerular capillary structures or debris-like structures in Bowman’s space were added in the generated images similar to Cluster 9. These changes were emphasized by pseudo-colored images with overlay (Fig. 4C).

Fig. 4
figure 4

Results of Cycle-GAN analysis. (A) shows that the outer portion of the glomerular capillary in cluster 1 images was erased, and Bowman’s spaces were enlarged. Generated images from cluster 9 revealed glomerular capillary structures or debris-like images in Bowman’s spaces (B). To emphasize the differences between the original and generated images, pseudo-colored images were created and overlayed. As shown in panel C, pseudo-colored images in green (left) represent the original images, and those pseudo-colored in red (middle) represent the generated images by Cycle-GAN. The images on the right represent the overlayed images. In the overlays of the original images from cluster 1 or 2 and generated images from cluster 9, the additional glomerular capillary structures are highlighted in red. In contrast, in the overlays of the original images from cluster 9 and generated images from cluster 1 or 2, the erased glomerular capillary structures are highlighted in green.

Full size image

Discussion

This study identified the subtle pathological changes on light microscopy in kidneys of patients with diabetes without overt diabetic nephropathy. Grad-CAM highlighted characteristic regions in representative clusters, revealing differences in glomerular structures. Cycle-GAN further illustrated disparities between clusters by generating similar or contrasting images.

Clustering with deep learning made it possible to identify the subtle pathological changes on light microscopy in kidneys of patients with diabetes. Previous pathological analyses of the kidneys using deep learning have mainly employed supervised learning by mimicking diagnoses by pathologists to expedite12,13 or identify findings similar to known pathological findings such as mesangial expansion12 or interstitial expansion13. Additionally, deep learning has been employed to measure and evaluate morphological metrics such as the shape, diameter of structures, and cell density18,19,20. Assessments based on these measurements are one of the strengths of image analyses. However, revealing novel lesions has not been achieved despite utilizing these morphological metrics. Deep learning may potentially allow the discovery of subtle features that are not recognizable with previous interpretations. Unsupervised learning, which does not require correct labels, may potentially reveal unknown features. For example, an analysis of prostate biopsies, including prostate cancer, using unsupervised learning revealed morphological changes in the stroma of nonmalignant areas that pathologists had not previously focused on, which was found to improve the diagnostic accuracy of cancer recurrence21. Similarly, the pathological findings that we discovered in this study have not been previously noted, but these may indicate novel lesions in diabetic nephropathy.

Although the lesions shown in this study have been discussed with pathologists, the tissue changes are so subtle that it has not been possible for us to define them pathologically at this time. So, further analysis is needed to clarify whether the lesions present represent a distinct glomerular type or glomerular characteristics. In addition, further studies using kidney samples from mouse models would be useful to determine the relationship between the lesions shown in this study and known lesions of diabetic nephropathy. Analyses of mouse kidney samples at longitudinal time points with gene expression, protein expression, and/or detailed observation using electron microscopy would be necessary.

Grad-CAM revealed that podocytes on the outer portions of glomerular capillaries are characteristic lesions of diabetic clusters. Flattening or effacement of foot processes and detachment of podocytes are known pathological changes in early diabetic nephropathy10. Mitochondrial damage22, inflammation23, and autophagy24,25 in podocytes are reportedly involved in early pathological changes in diabetes. Transient receptor potential canonical channels 6 (TRPC6), which is involved in autophagy of podocytes, participates in the induction of early renal pathological changes in diabetes24. The expression of TRPC6 was elevated in podocytes on the outer portions of glomeruli. Additionally, autophagy in podocytes is upregulated in the early stages of diabetic nephropathy25. Disturbance of autophagy in podocytes alters the cytoskeletal structure of glomerular capillaries and causes albuminuria24,25. This may explain why damage in podocytes in the outer portions of glomerular capillaries, which is a subtle pathological finding in diabetic nephropathy, was visualized through machine learning. However, to establish the association between the characteristic pathological changes identified in this study with known molecular findings, further analyses with immunostaining or animal studies are required.

Cycle-GAN revealed that smaller glomerular tufts compared to Bowman’s space, suggesting glomerular collapse, was a characteristic feature of the diabetic clusters. Diabetic nephropathy and early-stage obesity-related nephropathy reportedly enlarge Bowman’s space and the tubular lumen26,27,28. It is speculated that the enlargement of Bowman’s space reflects an increase in glomerular capillary pressure26,27. Similarly, enlargement of the tubular lumen is also speculated to occur with hyperfiltration27. Oxidative stress due to diabetes is reportedly involved in the enlargement of the tubular lumen28. Diabetic model mice lacking the enzyme Nrf2, which is involved in the production of the antioxidant glutathione, showed enlargement of the tubular lumen28. Therefore, the enlarged Bowman’s space and tubular lumen observed in this study may reflect hyperfiltration or oxidative stress in diabetic nephropathy.

The volume of glomeruli was thought to have affected the results of clustering.　In this clustering analysis, the size of the glomeruli was not measured, but the 13,251 glomeruli evaluated in this study were of a certain size with some degree of variability. It was reported that juxtamedullary glomeruli were larger than those in the outer cortical zones29. Certainly, the glomerular images were collected both the cortical and juxta medullary zones in this study. However, cortical and juxta medullary glomeruli were nearly equal in number in cluster1, a cluster mainly contained glomerular images from diabetes cases. So, at least in Cluster 1, location of glomeruli from the cortex or juxta medulla would not be affected to the result. Further detailed studies on cortical and juxtamedullary glomerular differences are necessary in the future.

This study has several limitations. The first limitation is the small sample size. The 13,251 glomerular images in this study are few compared to other datasets, such as ImageNet30, and may have led to overestimation. The second limitation is that the sample used in this study was limited to one point data of nephrectomy. Most of the cases used in this study are specimens with short diabetes duration and very mild proteinuria. In addition, advance pathological check confirmed that few lesions of advanced diabetic nephropathy, such as nodular lesions and mesangiolysis, were seen in specimens of these 45 cases. However, we were unable to provide evidence as to whether the lesions indicated in this study were “early” lesions, so we described these lesions as “subtle” lesions. In the future, it is necessary to verify the results over time using mouse models to minimize bias. The third limitation is the issue of staining. In this study, hematoxylin-eosin (HE) stained images and periodic acid-Schiff (PAS) stained images were analyzed. These two stained images are completely different objects. Although there are some similarities in the results, it is hard to discuss these two results at once. The fourth limitation is the clustering method. Invariant information clustering performs clustering based on the mutual information content of two CNNs15. As it is difficult to directly apply Grad-CAM, supervised learning was employed using the clustering results as the supervised labels. Thus, the rationale for clustering may not have been accurately described. However, in the learning curves of the supervised learning, the curves of loss function for both train and validation were steadily decreasing. So, overfitting was unlikely to be occurring in the supervised learning.

Conclusion

Research on novel features of diabetic kidney disease using deep learning is scarce. We established an approach to extract glomerular images from nephrectomy specimens, perform clustering, and visualize differences between clusters. The findings in this study may demonstrate the subtle and novel pathological changes on light microscopy in the kidneys of patients with diabetes without diabetic nephropathy. Our results may be an initial step of milestone for the detection of subtle pathological changes in diabetic nephropathy.

Methods

Collection of kidney specimens

Nephrectomized kidneys (partial/total) from Kanazawa Medical University from 1998 to 2019 were used in this study. Diabetes cases were defined as the following: (1) previously diagnosed with diabetes mellitus (DM) or (2) with an HbA1c level of ≥ 6.5% at the time nephrectomy. Non-diabetes cases were defined as those who had never been diagnosed with DM and whose HbA1c level was < 6.5% at the time of nephrectomy. Patients who did not meet the above criteria were excluded from the study. The following patients were also excluded: (1) those with kidneys that contained little or no glomeruli, and (2) those with kidney that exhibited significant compression due to carcinoma. All nephrectomized cases during the period were selected as candidate cases. Then, cases with interstitial fibrosis and tubular atrophy (IFTA) accounting for more than 50% of the whole specimen were then excluded. Cases with interstitial fibrosis and tubular atrophy (IFTA) accounting for more than 50% of the whole specimen were then excluded. Also excluded were cases with serum Cr > 2.0 mg/dL and eGFR < 20 ml/min/1.73 m2. As a result, 24 patients were selected as diabetes cases. Non-diabetes cases were also selected, in the same way as the diabetic cases. But the following cases were considered ineligible and excluded: (1) cases with previously diagnosed diabetes and (2) cases with significant lack of clinical data. As a result, 21 non-diabetes cases were selected.　Overall, 45 patients were included in this study. Because this study was a retrospective study, informed consent was obtained using the opt-out approach which had been approved by the ethical review board of Kanazawa Medical University. This study was approved by the ethical review board of Kanazawa Medical University (Kanazawa Medical University Hospital Medical Research Ethics Committee; approved No. I493). This study was also conducted in compliance with the Declaration of Helsinki (2013).

Glomerular detection and image analysis

All hematoxylin–eosin (HE) and periodic acid-Schiff (PAS) stained nephrectomized kidney specimens were captured and stored as virtual slides (NDP.View2, Hamamatsu Photonics, Hamamatsu, Japan). The renal cortical area was converted to a jpeg image with a size of 60,000 × 30,000 pixels (4.5 × 10 10 pixels / inch2). Basically, three 60,000 × 30,000-pixel images were collected from each kidney. In some cases, only two 60,000 × 30,000-pixel images were collected because the kidney was not large enough. Areas containing or close to malignancies and areas with obvious inflammatory cell infiltration were manually removed. Then, these 60,000 × 30,000-pixel images were scanned by the sliding windows31 and cropped and generated several 20,000 × 20,000-pixel images. All images were annotated for glomerular detection using LabelImg32. An object detection algorithm YOLOv5 (Ultralytics, Frederick, MD)33,34 was used to detect glomeruli. In the glomerular image selection step, glomeruli with global sclerosis, and glomeruli with extremely small cross-sections were excluded from training datasets. Therefore, global sclerosis, and glomeruli with extremely small cross-sections were not selected as glomeruli in this analysis. After detection of glomeruli images, when a glomerulus was located in the overlapping region of consequence window, bounding boxes overlapping by 35% or more were merged into one to avoid duplicate detection31. Each of the detected glomerular areas was saved as a jpeg image. As the detected images included non-glomerular areas and/or incompletely trimmed glomeruli, images were re-evaluated via supervised learning using EfficientNet-b035. Finally, 13,251 glomerular images were included in the dataset of HE stained. On the other hands, 20,645 glomerular images were included in the dataset of PAS stained.

Clustering of glomerular images

Clustering of glomerular images was performed using IIC15, which is an unsupervised learning system. In generating the paired images, we only performed rotations of up to 30 degrees without changing the contrast and color saturation, left–right and up–down horizontal flipping, or skewing. In this study, all glomerular images were converted to grayscale for clustering. Residual neural networks50 (ResNet50)36 was used as the CNN. IIC was trained with an initial learning rate of 1 × 10− 3, an epoch count of 50, and a batch size of 16. Adaptive moment estimation optimizer (Adam)37 was used as the optimization function for learning. To detect the suitable number of samples for clustering, the number of clusters was changed in a step-by-step manner. The PAS-stained data sets were also clustered in the same way as the HE-stained. However, the two datasets were clustered separately, as they are completely different subjects of analysis. The features of each cluster obtained from the IIC training were reduced in dimensionality by t-distributed stochastic neighbor embedding (t-SNE)38 and represented and visualized in a two-dimensional map.

Region of interest visualization using Grad-CAM

The region of interest (ROI) of the CNN during clustering was visualized using Grad-CAM. Grad-CAM uses the gradients of any target concept flowing into the final conventional layer to produce a coarse localization map that highlights important regions in the image for predicting the concept and visualizing it16. EfficientNet-b0 was retrained using supervised learning, employing clustering results from IIC as pseudo-labels, and was used as the CNN for Grad-CAM. Supervised learning was trained with an initial learning rate of 5 × 10− 4, an epoch count of 100, and an early stopping with patience of 10 epochs. All glomerular images were randomly assigned to train and validation sets. The ratio of training to validation was 8:2. Learning curves for train and validation were drawn to show the performance of the supervised learning.

Visualization with GAN

Cycle-GAN, which is another visualization technique to identify the reason of clustering results17, was used. Cycle-GAN visualized the differences between the diabetic and non-diabetic clusters. Using datasets from both clusters, Cycle-GAN can transform an original image from the diabetic cluster to a converted virtual image similar to an image in the non-diabetic cluster. Cycle-GAN was trained with a batch size of 4, an initial weight value (λ) of 10, and an epoch number of 200. To visualize the differences between the original image and the converted virtual image, pseudocolored images were created and overlayed. Pseudo-colored representative paired images were created by Photoshop (Adobe Inc, San Jose, CA). The original image was colored green, and the converted virtual image was colored red; the two images were then overlayed. The general overview of the workflow in this study is shown in Supplementary Figures S4.

System environment for image analysis

The PC used was Windows 11 (CPU Intel Core) with a GPU NVIDIA RTX3070 (8 GB) graphics board and CUDA version 12.1. All programming languages were coded in Python, and glomerular detection, clustering, and visualization of ROIs were implemented in Pytorch39. The CNN used for glomerular anomaly detection and clustering was ResNet5036. Efficinetnet-b035 was used for supervised learning for visualization of ROIs. Conversely, the adversarial generative network was implemented with TensorFlow40.