Introduction

In recent years, an increase in the incidence of cancer in the world has risen sharply, and the number of patients is growing year by year. The amount of cancer patients worldwide has rocketed from 17.2 million in 2016 to 19.3 million in 2020. It is estimated that the amount will rise to 20.2 million in 20221. Of the cancers, leukaemia is the most common blood cancer and is thought to contribute to about 60,000 deaths every year2.

Early diagnosis of leukaemia can remarkably improve the survival rate. In medicine, the early diagnosis of leukaemia is usually made by clinicians observing the morphology and structure of bone marrow and blood cells, such as microscopic examination of bone marrow aspiration and blood smears3. The microscopic examination of blood smears usually requires clinical doctors to observe and measure manually under a microscope, which is tedious and time-consuming, and the diagnostic accuracy depends upon prior knowledge, contextual information and operator’s experience, and so on. Thus, accurate, automated and repeatable methods for white blood cell classification are urgently needed in clinical practice.

Accurate white blood cell classification from cytopathological images depends upon the effective extraction of characteristics of white blood cells. However, this is difficult for several reasons. First, different cells vary greatly in shape, size, edge and position. Second, different staining methods and colorants can give rise to color differences among similar cells in different images. Finally, digital acquisition from different instruments for the same blood smear will also produce large differences in color. Nowadays, a multitude of cell classification methods have been proposed, including traditional methods4,5,6,7,8,9, machine learning methods10,11,12 and deep learning methods13,14,15,16,17,18,19,20,21,22,23. However, the traditional methods and the machine learning methods require hand-crafted features, the extraction of which is cumbersome and time-consuming, and the results are usually unsatisfactory.

Recently, with the rapid advancement of artificial intelligence, deep learning technology has swiftly permeated the medical field. Convolutional neural networks (CNNs) possess stable and reliable modelling capabilities. They can automatically extract image features and integrate them to generate higher-level semantic features based on global information, thereby enhancing the efficiency of image processing.

In 2017, Hung et al.13 used the Faster R-CNN network for the first time on malaria-infected blood smear images to identify infected cells and distinguish different stages of infection according to different characteristics of the cells. In the same year, Duggal et al.14 carried out research on normal and malignant cell classification of white blood cells in blood smears. They transformed microscopic images from RGB color space to optical density space. According to the principle of cell staining, the optical density space reveals a significant number of stained cells, which can provide detailed the characteristics of the cells. In 2019, Shahin et al.15 presented a new identification system for five types of white blood cells based on CNN, including transfer learning and fine-tuning of existing deep networks. They also proposed an end-to-end WBCsNet. In 2020, Huang et al.16 proposed an attention-aware residual network based manifold learning model (ARML) to classify white blood cells. Their aim is to enhance the network’s learning ability for classifying six types of white blood cells. In 2021, Shi et al.17 proposed to use Graph Convolutional Network (GCN) for cervical cell classification. First, features of all cervical cell images were extracted and clustered by CNN, and then a graph structure was used to further capture the underlying correlations among clusters. Second, GCN was applied to propagate the node dependency, thus producing the relation-aware feature representation. Finally, GCN features were added to strengthen the discriminative ability of CNN for features. In 2022, Ha et al.18 proposed a semi-supervised white blood cell classification method, which contained an SSTS module and a FGIA mechanism. In 2022, Sharma et al.19 proposed a deep learning method, which ultilizes the DenseNet121 to classify the white blood cells. In 2023, Zheng et al.20 proposed a two-stage method, which treated WBC detection as a small salient object detection task. In the saliency detection stage, Itti’s visual attention model was used to locate the ROIs. In the WBC detection stage, a more precise WBC classification was obtained by performing the modified CenterNet model on ROI sub-images. In the same year, Tummala et al.21 proposed a Siamese twin network (STN) model based on contrastive learning for the classification of normal peripheral blood cells, which was trained on relatively few images and used a EfficientNet-B3 as the backbone network. In 2024, Dhalla et al.22 proposed a model that simulated the real human brain, which consisted of many convolutional layers, each having its own set of additional skip connections. Useage of vector made them rotationally invariant. Eventually, the network was simplified by 20%, facilitating its deployment in resource-constrained environments. Gao et al.23 proposed a generative adversarial network (GAN) assisted framework, which can enhance images and boost the performance of peripheral leukocyte recognition and classification. However, many of the exiting methods are not sufficiently tailored to the specific characteristics of white blood cell images. The current network's ability to extract features is insufficient for distinguishing between normal and tumor cells and needs to be further improved.

Although deep learning based cell classification methods have made far more achievements than other methods, there are still some challenges in cell classification research as follows:

  1. 1.

    Current classification models for white blood cell discrimination are usually designed to implement cell classification directly. However, as there are various types of cells in blood smear images, white blood cell classification on blood smear images is easily disturbed by the background, resulting in unsatisfactory classification results.

  2. 2.

    The morphological characteristics of normal and tumor cells are very similar, and only local characteristics are slightly different. This is a fine-grained classification task, which is more difficult than the common classification tasks.

To address the above problems, motivated by the ResNeXt classification network24, we propose a ResNeXt-Cell classification (ResNeXt-CC) network for white blood cell classification from cytopathological images. Our study makes the following contributions:

  1. 1.

    Our study reveals that a suitable color space model can highlight the dynamic contrast and texture details of white blood cells. That is to say, the transformation of the color space can suppress or enhance the expression of some characteristics of images, thus promoting the classification performance.

  2. 2.

    Given that convolutions on different levels contain different discriminative information, and for the fine-grained classification task, local features play an important role in distinguishing different classes.We propose a cross-layer deep-feature fusion module (CFFM), which aims to fuse the discriminative feature information from different layers to improve the network performance.

  3. 3.

    To avoid the negative effects of feature compression and enhance cross-channel information extraction. ECANet is introduced into our network to enhance discriminative information extraction. ECANet 0 is introduced into our network, aiming at enhancing the extraction ability of discriminative information, thereby improving the classification results.

  4. 4.

    Since the cells morphology is highly similar, only the local characteristics differ, and there is a class imbalance between normal and tumor cells, we improve the softmax loss function and combine it with the center loss function to maximize the inter-class separation and enhance the intra-class compactness.

Methods

We propose a ResNeXt-CC network, as shown in Fig. 1. First, the efficient attention mechanism of ECANet module is used to strengthen feature extraction ability of cell details; second, a CFFM is proposed to enhance the discriminative information extraction; finally, we improve the softmax loss function and introduce the center loss function to train the network, which effectively addresses the problem of class imbalance and improves the discriminative information extraction.

Figure 1
figure 1

Schematic diagram of ResNeXt-CC cell classification network.

Full size image

ResNeXt classification model

Due to the small size of cytopathological image datasets, only increasing the network depth makes the network too complex, thus leading to the network over-fitting. The ResNeXt architecture can better extract features and achieve higher classification accuracy with fewer network parameters by combining ResNet-5026 and Inception27.

In this paper, the ResNeXt-50 realizes feature extraction through multiple ResNeXt modules and down-sampling layers, and then classifies images through the fully connected layer. Considering that it is relatively simple to extract features in the pooling layer, which may miss some effective information, and using a convolution method to down-sample can retain more information. Therefore, the convolution layer is used for down-sampling in our classification method instead of the pooling layer. The detailed architecture of the ResNeXt-50 model in this paper is shown in Table 1, where C represents the number of groups, x of Conv5_x represents the x-th ResNeXt module.

Table 1 Detailed architecture of the ResNeXt-50 model.
Full size table

Data enhancement for cytopathological images

In the field of medical imaging, due to the diversity of detection methods and equipment, different image features differ greatly. Therefore, image enhancement methods are designed according to the characteristics of different medical images. Particularly, color space transformation is an image enhancement method, that is, some images with low contrast and unclear meaning are transformed into other color spaces. This transformation can suppress or enhance the expression of some characteristics of images. In this paper, we use color space transformation to preprocess cytopathological images, with the aim of making it easier for the network to extract important information, to improve the network performance.

The RGB color model is widely used in medical imaging. The superposition of the red, green and blue light components in different proportions generates different colors. Moreover, image brightness is also closely related to these three components. In contrast, the HSV color model comprises three distinct color bands: hue (H), saturation (S), and lightness (V), where H represents color information, and its value ranges from 0° to 360°. S value ranges from 0 to 1: the higher the value, the darker and brighter the color. V value ranges from 0 to 1: the higher the value, the higher the brightness. In our experiment, prior to model training, the values within each channel of the HSV image representation were normalized into the range of [0, 1].

The HSV color space has more advantages over the RGB color space in terms of expressing hue, brightness and so on. In cytopathological images, different components of stained cells present different colors. Compared with the RGB color space, the HSV color space can better represent the hue, saturation, brightness and so on in cytopathological images. In this paper, the RGB color space is transformed into the HSV color space, which is more conducive to extracting cell details.

The image comparison before and after the transformation is shown in Fig. 2. We can see that the dynamic contrast and texture details of the latter are more prominent. Subsequent experiments show that the classification accuracy of white blood cells is significantly improved after the cytopathological images are transformed from the RGB to the HSV color spaces.

Figure 2
figure 2

Image comparison chart of different color models. (a) RGB cytopathological cell image; (b) HSV cytopathological cell image.

Full size image

ResNeXt-CC cell classification method

Cross-layer deep-feature fusion module

Although normal white blood cells and tumor white blood cells have high morphological similarity, their local characteristics are slightly different. Therefore, it is necessary to strengthen the network to identify the difference of local characteristics between classes to improve classification accuracy.

It is worth noting that different semantic information can be extracted from convolutional layers with different receptive fields, so there is a problem of information loss in the down-sampling process of CNN. Compared with common classification tasks, for the fine-grained classification tasks such as cell classification, using only one convolutional layer at the same network depth to obtain the characteristic information of cells often loses too much useful information. In recent years, many researchers have explored multi-level feature fusion. In Bilinear CNN28, two pooling operations were designed to fuse features from two branches to better obtain the spatial relationship between regions and improve classification accuracy. In addition, many feature fusion strategies have emerged in semantic segmentation and target detection fields. The U-net29 fused the feature information from different convolutional layers through the code-decode structure to achieve better segmentation results. The feature pyramid network30 used multi-scale feature fusion to further improve the detection accuracy. To sum up, a multitude of methods based on feature fusion have achieved good results in various fields of deep learning.

In this section, to explore the different discriminative feature information contained in different convolutional layers, the Grad-CAM31 algorithm is used to visualize the last layer of the Conv5_1, Conv5_2 and Conv5_3 modules from the backbone network, and the visualization results are shown in Fig. 3. The redder areas in the heatmaps indicate that the network pays more attention to them. From Fig. 3, the different convolutional layers from the classification network pay attention to different areas of cells and contain different discriminative feature information.

Figure 3
figure 3

Comparison of heatmaps of different convolutional layers. (a) Input image; (b) Conv5_1; (c) Conv5_2; (d) Conv5_3.

Full size image

Based on the above analyses, to better obtain the discriminative feature information from the different convolutional layers, we propose a CFFM, the structure of which is shown in Fig. 4. Note that the advanced semantic information from the deep layers can help improve the performance of the classification network, the last layers from Conv5_1 and Conv5_3 are fused.

Figure 4
figure 4

Schematic diagram of CFFM.

Full size image

The specific process of CFFM is as follows:

  1. 1.

    Generation of feature maps Z1 and Z2: First, a 1 × 1 convolution operation is used for the last feature layers of the Conv5_1 and Conv5_3 modules from the backbone network respectively, resulting in two feature maps Z1 ∈ RC×W×H and Z2 ∈ RC×W×H, where C is the number of channels, W and H are the width and height respectively, followed by a 1 × 1 convolution operation for increasing information flow among channels. In this paper, C is set to 2048, the dimensions of Z1 and Z2 are the same as those of the outputs of the last layers from Conv5_1 and Conv5_3, respectively.

  2. 2.

    Multi-level feature space fusion: the feature map Z3 is obtained by the cross-layer fusion of feature maps Z1 and Z2, and its calculation formula is as follows (1); then the Z3 is fed into the global average pooling (GAP) layer to attain FP ∈ RC×1×1; finally, it is fed into the fully connected layer to obtain a classification result.

    $$Z_{3} = Z_{1} \otimes Z_{2}$$
    (1)

    where Z1 ∈ RC×W×H, Z2 ∈ RC×W×H, Z3 ∈ RC×W×H, \(\otimes\) stands for pixel-wise multiplication.

Feature enhancement module based on down-sampling

As we know, CNN can extract rich feature information from feature maps, and then this feature information can be used to classify objects by the classification network. For fine-grained image classification tasks such as classification of normal and tumor cells, the ability of using CNN to obtain discriminative features is limited due to the great similarity among input images. Note that, ECANet25 is an improvement upon SENet32, designed to avoid the negative effects of feature compression and enhance cross-channel information extraction. ECANet is introduced into our network to enhance discriminative information extraction. Given the fact that different channels contain different semantic information in a feature map, the importance of extracted features from each channel is different, and even some extracted features may interfere with classification results. As a result, we use ECANet to screen out the discriminative features of cells more effectively and make the classification results more accurate.

The specific process of the ECANet algorithm is as follows: the original feature map F ∈ RC×W×H is fed into the GAP layer to extract the global feature Fp; followed by a k × k convolution for extracting cross-channel information Fe from Fp, where k is calculated by (2), γ and b are 2 and 1 respectively, and odd represents the nearest odd number; finally, Fe is fed into an activation function to obtain a result that is multiplied by the original feature map, thereby adjusting the channel relationship.

$$k = \left| {\frac{{\log_{2} \left( C \right)}}{\gamma } + \frac{b}{\gamma }} \right|_{odd}$$
(2)

With the deepening of the network, the receptive fields of the deep layers become larger, but multiple down-sampling operations also give rise to the loss of the detailed features of cells. Therefore, in order to improve the extraction performance of the network, we enhance the features after each down-sampling. The structure of the feature enhancement module based on down-sampling is shown in Fig. 5, where Convk represents the k-th stage of the classification network.

Figure 5
figure 5

Schematic diagram of the feature enhancement module based on down-sampling.

Full size image

Loss function optimization

The softmax loss function, which is defined as follows, is often used in training a deep learning model and makes it achieve good classification accuracy.

$$\delta \left( {Z^{(i)} } \right)_{j} = \frac{{e^{{z_{j}^{(i)} }} }}{{\sum\nolimits_{m = 0}^{n - 1} {e^{{z_{m}^{(i)} }} } }}$$
(3)

However, it is noting that there exists a class imbalance between normal and tumor cells for the classification of normal and tumor cells. Besides, the morphological characteristics of normal and tumor cells are similar, and only the local characteristics are slightly different, which leads to low classification accuracy. To solve the above problems, we improve the softmax loss function, called w-softmax loss function. As we all know, the softmax loss function consists of the cross-entropy loss function and the softmax function. Next, the cross-entropy loss function is first introduced and defined as:

$$L = - \sum\limits_{c = 1}^{M} {y_{ic} } \log \left( {p_{ic} } \right)$$
(4)

where M is the number of classes in the dataset, i expresses the i-th sample, pic represents the probability of i being class c, and yic is used to quantify the similarity between the predicted label and the true label c. Concretely, yic = 1 means exactly the same, while yic = 0 indicates completely different.

In this paper, only normal and tumor cells of white blood cells are classified. Therefore, with M = 2 and yi1 + yi2 = 1, Eq. (4) can be transform into Eq. (5), which represents the binary cross-entropy loss (BCELoss) function.

$$L_{b} = - \left( {y_{i} \log \left( {p_{i} } \right) + \left( {1 - y_{i} } \right)\log \left( {1 - p_{i} } \right)} \right)$$
(5)

The number of normal cells is usually different from the number of tumor cells, leading to class imbalance. Therefore, in this paper, we improve the BCELoss by weighting, namely the losses of both classes are multiplied by weights to make the network model pay more attention to the smaller class in the training set. The improved BCELoss is given by:

$$L_{w} = - \left( {\alpha_{1} y_{i} \log \left( {p_{i} } \right) + \alpha_{2} \left( {1 - y_{i} } \right)\log \left( {1 - p_{i} } \right)} \right)$$
(6)

where \(\alpha_{1}\) is the proportion of normal cells to all white blood cells in the training set. Similarly, \(\alpha_{2}\) is the proportion of tumor cells to all white blood cells in the training set. Finally, (3) and (6) are combined to attain the improved BCELoss as follows:

$${\mathcal{L}}_{S} = - \left( {\alpha_{1} y_{i} \log \frac{{e^{{\theta_{1}^{T} x^{(i)} + b_{1} }} }}{{\sum\nolimits_{j = 1}^{2} {e^{{\theta^{T}_{j} x^{(i)} + b_{j} }} } }} + \alpha_{2} \left( {1 - y_{i} } \right)\log \frac{{e^{{\theta_{2}^{T} x^{(i)} + b_{2} }} }}{{\sum\nolimits_{j = 1}^{2} {e^{{\theta^{T}_{j} x^{(i)} + b_{j} }} } }}} \right)$$
(7)

where \(\theta_{i}\) corresponds to the i-th column of the output matrix of the fully connected layer. \(x^{(i)}\) represents the characteristics of the input of the last layer of the fully connected layer, and bj represents the bias.

For the classification of normal and tumor cells, there are not only great similarities between them, but also differences in local characteristics. Owing to the above problems, for cell classification, it is highly recommended to maximize the inter-class separation and enhance the intra-class compactness to improve the classification accuracy. However, using the softmax loss in the classification network can only enforce the inter-class separation, but cannot enhance the intra-class clustering. To address this issue, we incorporate a center loss function33 in the training of the network. In other words, a constraint is added to the softmax loss function. The center loss function helps learn the feature centers of different cell classes and enhance the intra-class compactness. The center loss function is defined as:

$${\mathcal{L}}_{C} = \frac{1}{2}\sum\limits_{i = 1}^{m} {\left\| {x_{i} - {\mathbf{c}}_{{y_{i} }} } \right\|_{2}^{2} }$$
(8)

where xi is the extracted feature map of the i-th image. \(c_{{y_{i} }}\) represents the center of the \(y_{i}\)-th cluster, and m is the number of images in a mini-batch. Ideally, the center of each cluster should be updated every epoch, but this is too inefficient. Therefore, in the training process, the center of each cluster is updated based on the mini-batch m, and the partial derivative of \({\mathcal{L}}_{C}\) with respect to xi and the update gradient of \(c_{{y_{i} }}\) are shown as.

$$\frac{{\partial {\mathcal{L}}_{C} }}{{\partial x_{i} }} = x_{i} - c_{{y_{i} }}$$
(9)
$$\Delta c_{j} = \frac{{\sum\nolimits_{i = 1}^{m} \delta \left( {y_{i} = j} \right) \cdot \left( {c_{j} - x_{i} } \right)}}{{1 + \sum\nolimits_{i = 1}^{m} \delta \left( {y_{i} = j} \right)}}$$
(10)

where \(\delta (condition)\) = 1 if the condition is satisfied, otherwise \(\delta (condition)\) = 0.

In this paper, the improved BCELoss function and the center loss function are combined. The former can ensure the feature separation between classes, meanwhile also effectively improve the decline of classification accuracy caused by class imbalance. The latter can make images from the same class cluster. In other words, this approach can enhance the intra-class compactness.

The combination of these two loss functions (7) and (8) improves network performance. Which is defined as:

$$\begin{aligned} {\mathcal{L}}_{S} & = - \lambda \sum\limits_{i = 1}^{m} {\left( {\alpha_{1} y_{i} \log \frac{{e^{{\theta_{1}^{T} x^{(i)} + b_{1} }} }}{{\sum\nolimits_{j = 1}^{2} {e^{{\theta^{T}_{j} x^{(i)} + b_{j} }} } }} + \alpha_{2} \left( {1 - y_{i} } \right)\log \frac{{e^{{\theta_{2}^{T} x^{(i)} + b_{2} }} }}{{\sum\nolimits_{j = 1}^{2} {e^{{\theta^{T}_{j} x^{(i)} + b_{j} }} } }}} \right)} + \beta \frac{1}{2}\sum\limits_{i = 1}^{m} {\left\| {x_{i} - {\mathbf{c}}_{{y_{i} }} } \right\|_{2}^{2} } \\ & = \lambda {\mathcal{L}}_{wd} + \beta {\mathcal{L}}_{C} \\ \end{aligned}$$
(11)

where m is the number of images in a mini-batch, \(\lambda\) and \(\beta\) are the weight coefficients, which are used to adjust the proportion of the two parts and set to 1 and 0.05, respectively.

Ethics approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Experiments and results

Dataset

ALL-IDB34, which consists of 108 images and contains about 39,000 blood elements, and each image resolution is 2592 × 1944, is used to study the segmentation and classification of white blood cells. The BCCD dataset35 includes 364 microscopic images of various white blood cells. In ALL-IDB, however, only 260 cytopathological cells can be used for classification. For classification tasks, the number of cytopathological images in the dataset is too small, which will seriously affect the network performance. Therefore, in this paper, we use a dataset with more cytopathological images, namely the C-NMC 2019 dataset36. This was released in the Medical Imaging Challenge of the International Symposium on Biomedical Imaging (ISBI). We use a subset of the C-NMC 2019 dataset, which contains white blood cell images from 118 patients. There are 12,528 white blood cells in the dataset, including 8491 cancer cells and 4037 normal cells, each cell of which is annotated by pathologists. Each image resolution is 2560 × 1920, and each blood smear contains multiple cells. To better classify white blood cells, the researchers segmented each white blood cell from the blood smear images to construct a dataset, in which each image resolution is 300 × 300. The dataset is classified into three subsets: a training set, a validation set and a testing set, and these three subsets are split with the ratio 7:1:2.

The white blood cells from the blood smears are shown in Fig. 6. Except for white blood cells, a blood smear contains other similar cells that may affect the classification performance. Therefore, the separated white blood cell images are adopted to train the classification network, which is expected to achieve better classification accuracy.

Figure 6
figure 6

Comparison of tumor cells and normal cells. (a) Tumor cells; (b) Normal cells.

Full size image

Configuration

In this paper, we implemented the classification method in Python on a computer with two NVIDIA 1080Ti GPUs, each with 11 GB of memory, and an Intel Xeon E5-2630 CPU. The pytorch library served as a high-level framework, and the experimental platform is the Ubuntu 18.04 system. The hyper-parameter details of the classification model are shown in Table 2. Note that we adopted the vanilla cross-entropy loss function to guide the training of both ResNeXt-50 and ResNeXt-101 (section "Effect of different depths on network performance"), while we employed the proposed w-softmax loss function for our model in other experiments (section "Effect of color models on network performance" to "Comparison with CNN-style models").

Table 2 Hyper-parameters of the ResNeXt-CC model.
Full size table

Performance evaluation metrics

In this paper, tumor cells are positive samples and normal cells are negative samples. The performance evaluation metrics can be described by four values:

  1. 1.

    TP: true positives, the number of correctly classified tumor cells.

  2. 2.

    FP: false positives, the number of normal cells classified as tumor cells.

  3. 3.

    TN: true negatives, the number of normal cells correctly classified.

  4. 4.

    FN: false positives, the number of tumor cells classified as normal cells.

Currently, the most widely used metrics for the quantitative evaluation of image classification results are the following: Accuracy, Precision, Recall, F1-score, Specificity and AUROC. Five of these are defined by:

$$Accuracy = \frac{TP + TN}{{TP + TN + FP + FN}}$$
(12)
$$Precision = \frac{TP}{{TP + FP}}$$
(13)
$$Recall = \frac{TP}{{TP + FN}}$$
(14)
$$F1 - score = 2 \times \frac{{\Pr ecision \times {\text{Re}} call}}{Precision + {Recall}}$$
(15)
$$Specificity = \frac{TN}{{FP + TN}}$$
(16)

The AUROC metric refers to the area under the receiver operating characteristic (ROC) curve.

Ablation experiment

Effect of different depths on network performance

In CNN, simply increasing the network depth does not necessarily improve its performance. To validate the effect of the network depth on the classification performance, we conducted experimental research on ResNeXt network with different depths using the HSV color space. The experimental results are shown in Table 3.

Table 3 The effect of different depths on model’s performance.
Full size table

From Table 3, the deeper network layers don’t mean the better performance in medical image domain. When the network deepens to a certain depth, the network performance will decline, although the Specificity of ResNeXt-101 is higher. Figure 7 illustrates a ROC comparison between ResNeXt-50 and ResNeXt-101. It is evident that ResNeXt-50 outperforms ResNeXt-101 in cell classification, so we have chosen ResNeXt-50 as the backbone network for this paper.

Figure 7
figure 7

Comparison of ROC between ResNeXt-50 and ResNeXt-101.

Full size image

Effect of color models on network performance

In this section, we explore the effect of color space transformation on cell classification. As a common color space model, the RGB color space is widely used in medical imaging domain. However, for different applications we should select the most suitable color space. Specifically, we investigated RGB and HSV color spaces in terms of classification accuracy.

It can be seen from the experimental results in Table 4 that the classification accuracy is improved after transforming images from the RGB color space into the HSV color space, which proves that cytopathological images in the HSV color space can express image features better than those in the RGB color space. Cytopathological cell images in the HSV color space can better highlight local features and increase the contrast between cell classes. In contrast to the RGB color space, the three metrics of Accuracy, F1-score, AUROC and Specificity of the HSV color space are 0.16%, 0.42%, 0.07% and 1.24% higher, respectively. For the above reasons, we choose HSV as our color space model for white blood cell classification.

Table 4 Performance comparison using different color spaces based on the ResNeXt-CC model.
Full size table

Effect of feature fusion on network performance

In Table 5, we show the effect of CFFM on our network. Notably, our experiments are based on the HSV color model. When the last layer of Conv5_3 fuses with the last layers of Conv5_1 and Conv5_2, respectively, all metrics of the classification network are improved. The accuracy, F1-score, AUROC, and specificity of the Conv5_1 + Conv5_3 method is 0.59%, 0.43%, 0.53%, and 1.08% higher, respectively, than those of the Conv5_2 + Conv5_3 method.

Table 5 The effect of CFFM on the performance of the ResNeXt-CC model.
Full size table

It proves that convolutions on different levels contain different discriminative information, and for the fine-grained classification task, local features play an important role in distinguishing different classes. Thus, the CFFM proposed in this paper can better obtain the discriminative information of cells by fusing different level features. In contrast, we choose the feature fusion of Conv5_1 and Conv5_3.

Effect of composite loss function on network performance

In this section, our experiments are based on both the HSV transformation and CFFM. To further boost the cell classification performance, we improve the softmax loss function (w-softmax loss function) and combine the w-softmax loss function with the center loss function to achieve better results. In Table 6, we compare the results of softmax loss, w-softmax loss and w-softmax loss + center loss. From the comparison of Table 6, it is easily observed that, when using our loss function, the Accuracy, F1-score, AUROC and Specificity reach optimal performance, 87.46%, 90.62%, 93.87% and 77.72%, respectively. Compared with the original softmax loss, our network performance is slightly increased by the w-softmax loss, and our loss function is the most competitive since it can maxmize the inter-class separation and improve the intra-class clustering.

Table 6 The effect of different loss functions on the performance of the ResNeXt-CC model.
Full size table

Effect of feature enhancement module based on down-sampling on network performance

In this section, our experiments are based on the HSV transformation, CFFM and composite loss function. For fine-grained classification of normal and tumor cells, it is important to enhance the discriminative ability of cell details. However, not all extracted cell features are useful for the classification task. Meanwhile, some important information may be lost in the down-sampling process. Therefore, we constructed a down-sampling feature enhancement module that introduces ECANet into the down-sampling process to enhance the network's discrimination for useful features.

In Table 7, we compared the white blood cell classification performance between using down-sampling (ECANet) and pooling. The experiment shows that by placing the attention module in the down-sampling layer, the loss of useful information of images is mitigated and the discriminative information of images is extracted more efficiently, and thus dramatically improving the network performance. In contrast to not using ECANet, the four metrics of Accuracy, F1-score, AUROC and Specificity are 1.99%, 1.46%, 0.86% and 3.39%, respectively. Figure 8 visualizes the feature heatmaps from the last convolutional layer of the network, which are drawn using the Grad-CAM algorithm31. In the heatmap, the redder area represents the greater response of the network to that area. That is to say, the redder area represents more attention being rendered. On the contrary, the bluer area represents less attention being rendered. Clearly, with the addition of the attention module (ECANet), the network responds more to cells, particularly emphasizing their edges and content.

Table 7 The effect of feature layer attention module on the performance of the ResNeXt-CC model.
Full size table
Figure 8
figure 8

Heatmaps before and after feature enhancement. (a) Images of cell; (b) Before feature enhancement; (c) After feature enhancement.

Full size image

Comparison with CNN-style models

To validate our network performance, in this section, we compare our algorithm with CNN-style models. The classification results are shown in Table 8.

Table 8 Comparison of performance of CNN-style models with the proposed method.
Full size table

From the results in Table 8, ResNet-50 has achieved good results in cell classification. Furthermore, both ResNet-50 and our method have adopted the residual connection, which further proves that the residual connection method used in our algorithm is helpful to improve the classification accuracy. Densenet121 uses dense blocks to extract image features, because the network is too complex, thereby leading to over-fitting. That is to say, the classification results are not very good. In a word, the Accuracy, F1-score, AUROC and Specificity of our method are 8.57–4.66%, 6.86–3.21% and 7.26–3.19% and 11.68–6.12% higher than those of other models, respectively. The comparison of the ROC of the above classification algorithms is shown in Fig. 9, from which we can see that our method has a clear advantage over the other methods. To sum up, our method is superior to these common classification methods in various classification measures, which fully proves the superiority of our method.

Figure 9
figure 9

ROC comparison of CNN-style models and the proposed method.

Full size image

Comparison with ViT-style models

To further validate our network performance, in this section, we compare our algorithm with ViT models. From Table. 9 we can observe the classification results. Compared with other ViT-style models, the Accuracy, F1-score, AUROC and Specificity of our algorithm are 10.39–20.43%, 13.7–23.56%, 13.1–25.71%, and 22.78–36.98% higher than those of the other models. These above results confirm that our method is significantly superior to the other five ViT-style models.

Table 9 Comparison of performance of ViT-style models with the proposed method.
Full size table

Besides, to more intuitively evaluate classification performance between the four ViT-style models and our method, we have constructed a bar chart. Figure 10 shows our method has an overall advantage.

Fig.10
figure 10

Comparison of bar chart for ViT-style models and our method.

Full size image

Discussion

Regarding the diversity of training data, deep learning-based classification models necessitate a substantial corpus of high-quality annotated data to achieve robust performance. Nevertheless, acquiring such a comprehensive dataset can be challenging, particularly in scenarios where data availability is limited. Therefore, data augmentation techniques are potentially effective approaches. In this study, we employ traditional data augmentation methods, including horizontal flips, vertical flips, random scaling, and random rotation transformations, during the model training. However, conventional data augmentation technologies are limited in their ability to simulate real data diversity. To address this, more effective data augmentation and utilization methods should be introduced into our network. Specifically, future research could explore the application of semi-supervised learning and distillation techniques to mitigate the reliance on extensive annotated datasets. Additionally, incorporating generative adversarial networks (GANs) offers a promising avenue for further enriching our dataset.

In terms of attention mechanism in network architecture, the self-attention mechanism has demonstrated significant potential. The self-attention technique allows the model to focus on the most informative regions within the input, prioritizing the relevant features that contribute to the classification task. By effectively combining CNN and the self-attention mechanism, we can capture multi-scale contextual features from different network layers, thereby improving the accuracy of white blood cell classification. Moreover, building upon the self-attention technique, the image-content-based biomarkers of white blood cells can be extracted through interpretable visualized analysis of the white blood cell classification process. This enables a deeper understanding of the underlying features and patterns that the model has learned, potentially leading to new insights and the development of more robust and clinically relevant classification systems.

Conclusion

In summary, while the morphological characteristics of normal and tumor white blood cells are quite similar, they exhibit subtle differences in localized areas. Drawing inspiration from the ResNeXt classification network, we proposed a ResNeXt-CC classification network for white blood cells. Our method offers several advantages: (1) the proposed CFFM enhances the ability of discriminative information extraction; (2) a feature enhancement module, based on down-sampling, strengthens cell detail extraction; (3) the combination of w-softmax loss and center loss improves intra-class compactness and ensures inter-class separation; (4) color space transformation is applied to better represent cell texture information. The experimental results on the C-NMC 2019 dataset show that our proposed method manifests obvious advantages over the existing classification methods, including ResNet-50, Inception-V3, Densenet121, VGG16, Cross ViT, Token-to-Token ViT, Deep ViT, and simple ViT about 5.5–20.43% accuracy, 3.6–23.56% F1-score, 3.5–25.71% AUROC and 8.1–36.98% specificity, respectively. Ultimately, we hope that our work will inspire other related research on white blood cell classification.