Introduction

Concrete is an important building material that is widely used in all kinds of civil engineering due to its wide availability, low price, convenient construction, etc.1 However, concrete has compressive but not tensile load-bearing characteristics, and is very susceptible to cracks during use due to the effects of applied loads, temperature changes, heat of hydration, etc.2. The appearance of cracks weakens the safety, stability, and durability of the concrete structure and cracks may even cause engineering disasters3, such as the I-35W highway bridge collapse in Minnesota, USA in 20074 and the major infiltration accident in the Shijingshan Tunnel in Guangdong, China in 20215. Therefore, detecting cracks in concrete is essential to ensure the safety of concrete structures.

Traditional concrete crack detection is mainly manual detection, low efficiency, low accuracy, poor safety, belongs to the subjective detection method, and needs to invest a lot of money6. With the development of machine vision technology in recent years, significant progress has been made in automated detection based on digital image processing algorithms and traditional machine learning algorithms. Digital image processing algorithms rely on thresholds, edges, regions, and other features of the images. Traditional machine learning algorithms often require manual setting of width, area, and other features to recognize cracks. However, compared to other object recognition, cracks are usually very subtle and small in size, and the morphology and orientation of cracks do not have a fixed pattern, making it difficult to accurately recognize them. In addition, image backgrounds in engineering environments are often contaminated with dust, stains, and other structural defects, which can interfere with the accurate identification of cracks. As a result, these classifiers, based on “image features” have poor performance and robustness when dealing with complex and varied crack images7,8,9,10,11,12,13.

Compared with digital image processing algorithms, deep learning algorithms can extract features directly from images, perform end-to-end learning, and effectively guarantee the identification of concrete cracks after extensive training, showing good robustness. For example, Cha et al.14 established a deep architecture convolutional neural network (CNN) concrete crack classification model, which outperformed the Canny operator and Sobel operator in the recognition of images with strong spots and shadows; the recognition accuracy reached 98%. Furthermore, Lu Deng et al.15 established a deformable modular CNN (R-CNN) crack detector, which employed deformable convolution and deformable pooling operation to effectively improve the extraction of crack features. Additionally, Zhou and Song16 developed a crack classification method based on heterogeneous image fusion based on the temperature difference between cracked and non-cracked regions, yielding a recognition accuracy of 99.2%. Moreover, Guo et al.17 established a deep-width network (DWN) model to classify concrete cracks, displaying an accuracy rate of up to 98.55%. Darragh O 'Brien et al.18 classified and recognized tunnel lining cracks based on VGG16, which effectively overcame the unfavorable tunnel environment, and the recognition accuracy rate reached 98.3%. Majdi Flah et al.19 combined an improved Otsu image processing technique with a deep learning classifier to classify the crack types according to the direction of the cracks in the concrete with 96.17%; Yamaguchi et al.20 proposed an accurate classification method for pavement crack images based on a deep learning architecture, and the improved VGG16-SOM achieved an accuracy of 96%. Meanwhile, Dong et al.21 combined the channel attention mechanism and multi-head attention for crack recognition in complex scenes; Yang et al.22 proposed a transfer learning-based multi-classification crack detection method for DCNN with 99.83%, 99.72%, and 97.07% test results on three kinds of datasets, respectively. The concrete crack recognition model based on a deep learning algorithm exhibited good robustness and high accuracy. However, most of the existing crack recognition models based on deep learning algorithms were improved by using large CNNs, such as VGG, ResNet, Densnet (Dense Convolutional Network), etc.18,20,22,23,24. These result in a large number of computational parameters and high deployment requirements for applications.

Therefore, establishing a lightweight recognition model with high accuracy and easy deployment will provide more practical application value and potential. Among the existing lightweight neural networks, MobileNet is often used as a basic model25,26. In order to improve the concrete crack recognition accuracy, the model must be enhanced by selecting an appropriate attention mechanism for engineering features27,28,29. Therefore, based on the characteristics of concrete crack image recognition, this paper employed the C//Sim (Coordinate, Simple, and Parameter-Free Attention Module) attention mechanism to extract spatial features and channel features embedded with position information in the image, and introduced it into MobileNetV3. The MobileNetV3-C//Sim modeling was completed. The effectiveness, high accuracy, and light weight of the model for concrete crack recognition were verified experimentally in the concrete crack binary classification task (cracks, non-cracks) and multi classification task (horizontal, vertical, diagonal, and irregular cracks).

Lightweight model based on MobileNetV3-C//Sim

MobileNetV3

MobileNetV3 is a lightweight convolutional neural network25, which is characterized in comparison with traditional convolutional neural networks.

  1. (1)

    Depth separable convolution greatly reduces the number of parameters of the model through pointwise and deep convolutions. The ratio of depth separable convolution to ordinary convolution computation is shown in Eq. (1).

    $$\frac{{D_{K}^{2} MD_{F} + MND_{F}^{2} }}{{D_{K}^{2} MND_{F}^{2} }} = \frac{1}{N} + \frac{1}{{D_{K}^{2} }}$$
    (1)

    where Dk denotes the size of the convolution kernel, DF represents the size of the input feature map, M indicates the number of channels, and N refers to the number of channels of the output feature map.

  2. (2)

    Figure 1 displays the inverted residual structure. The accuracy is ensured by an hourglass-type structure with two small ends and a large middle, which further reduces the number of parameters of the model and improves its running speed.

    Fig. 1
    figure 1

    Bottleneck block structure.

    Full size image
  3. (3)

    The SE (Squeeze and Excitation) attention mechanism is used. Firstly, the feature map U of size H × W × C is globally average pooled to obtain the feature vector z by Squeeze operation. Subsequently, the obtained feature vector z is inserted into two fully connected layers containing the ReLu activation function using the Excitation operation, and the weights of each feature channel are generated using the Sigmoid activation function. Finally, the normalized weights are applied to the feature maps of each channel by the Scale operation.

Improved attention mechanisms

Considering that the SE attention mechanism in MobileNetV3 is limited to evaluating the importance of information between encoded channels, the location details that are crucial in recognizing the structure of the object cannot be fully captured, limiting the model recognition rate. Concrete cracks have different morphologies, directions, and widths, and their distribution is often inhomogeneous. Therefore, the features at different locations in the spatial dimension should be weighed to focus on the critical regions that likely contain cracks. Therefore, additional feature information needs to be introduced to improve the recognition accuracy of the model and enhance its sensitivity to the crack location information.

This paper combines the CA (Coordinate Attention) attention mechanism30 and SimAm (A Simple, Parameter-Free Attention Module) attention mechanism31, taking advantage of their strengths. Consequently, this study proposes the C//Sim attention mechanism, the structure of which is shown in Fig. 2. The CA attention mechanism obtains the feature maps in two directions by dividing the input feature maps into two directions, namely width and height, and performing global average pooling. In the detection task, this attention mechanism can improve the model accuracy by embedding the position information into the channel attention32. The SimAm attention mechanism is based on the biologically active principle of neuron inhibition of peripheral neurons and obtains a higher priority attention allocation mechanism, which can automatically assign different weights to the channel features and spatial features of the target without increasing the parameters. It can effectively extract the crack features, improve the model accuracy, and reduce the amount of computation in the detection of concrete cracks33.

Fig. 2
figure 2

Attention characterization maps for SE and C//Sim.

Full size image

As shown in Fig. 2, the feature maps obtained from the C//Sim attention mechanism and the SE attention mechanism, respectively, are demonstrated. In the feature map of the C//Sim attention mechanism, more crack regions are highlighted, showing a higher level of attention, which indicates that the C//Sim attention mechanism is more effective in capturing and emphasizing the feature of cracks. Therefore, compared with the SE attention mechanism, the C//Sim attention mechanism can reduce the number of computational parameters and storage requirements while improving the feature expression ability. The mechanism is suitable for lightweight convolutional neural networks. The specific implementation steps were as follows:

  1. (1)

    In the CA module, the input feature maps in the vertical and horizontal directions were first divided into two independent direction-aware feature maps using two 1D global pooling operations. Then, the feature information in these two directions was spliced and downscaled to obtain an intermediate feature map of C/r × 1 × (W + H) (r is the scaling factor for the number of scaled-down channels). Subsequently, the intermediate feature maps from the vertical and horizontal directions were respectively, which was upscaled to the original dimension by 1 × 1 convolution, and the final feature maps in the two directions were obtained by the Sigmoid activation function. Finally, the input feature maps and the amount of final directional feature maps were converted to the final output feature maps by multiplication.

  2. (2)

    In the SimAm module, the importance of different neurons was first divided by calculating the energy function of different neurons, as shown in Eq. (2), where λ is the canonical term, t was the target neuron of the input feature map on a single channel, and μ and σ are the mean and the variance of all neurons on a single channel, respectively. A lower minimum energy e indicated that the target neuron t was more different from the other neurons x, with a higher importance. Finally, the importance of the neurons was weighted by the Sigmoid activation function, as shown in Eq. (3).

    $$e_{t} = \frac{{4\left( {\sigma^{2} + \lambda } \right)}}{{\left( {t - \mu } \right)^{2} + 2\sigma^{2} + 2\lambda }}$$
    (2)
    $$S = sigmoid\left( \frac{1}{E} \right) \cdot X$$
    (3)
  3. (3)

    As shown in Fig. 3, the final feature map was obtained by adding the feature maps obtained in the first two steps.

    Fig. 3
    figure 3

    C//Sim structure.

    Full size image

Modeling MobileNetV3-C//Sim

The MobileNetV3-C//Sim network model based on MobileNetV3 was obtained by replacing the SE module in the MobileNetV3 structure with the C//Sim module. The overall structure of the model is shown in Fig. 4 below.

  1. (1)

    Input layer: the original crack image is received as input.

  2. (2)

    Feature extraction layer: the input image is first passed through a 3 × 3 initial convolutional layer with a step size of 2 to initially extract the crack features; the feature map obtained from the initial convolutional layer is then used as an input to the bottleneck layer, which further extracts the crack features by upgrading and then downgrading the feature map through the depth-separable convolution, residual connection, and the C//Sim attention mechanism while keeping the model lightweight.

  3. (3)

    Output Layer: Global Average Pooling is used to convert the feature map into a one-dimensional feature vector, then a 1 × 1 convolutional layer is used instead of the fully connected layer, and the SoftMax activation function is used to generate probability distributions for each category, yielding the final classification results.

Fig. 4
figure 4

MobileNetV3-C//Sim structure diagram.

Full size image

Experimental design

Test environment

Python 3.9 was used as the experimental environment in this study, with the deep learning framework of Pytorch 2.0.0. The CPU was AMD Ryzen 5 5600H with Radeon Graphics 3.30 GHz, GPU NVIDIA GeForce RTX3050Ti with 4 GB of video memory, and the operating system was Windows 11.

Dataset construction

We incorporated structures from different environments and designed two datasets. The datasets are specified below:

Dataset 1: In this paper, the datasets for binary and multi classification are selected from the public dataset of concrete cracks in roads and bridges34,35, and were divided into the training set, the validation set, and the test set according to a ratio of 6:2:2. Among them, the binary classification dataset contained 12,000 crack images and 8000 background images; the multi classification dataset contained 2000 horizontal cracks, 2000 vertical cracks, 2000 diagonal cracks, and 1500 irregular cracks, as displayed in Fig. 5a.

Fig. 5
figure 5

Concrete cracks data set.

Full size image

Dataset 2:This dataset is from the Changlinggang Tunnel on the Chengde South Expressway in China. We took 425 original images with a resolution of 3072 × 4096, In order for the model to extract more features to accommodate images with different brightness and contrast. The dataset is expanded using operations such as cropping, luminance change, and rotation as in Fig. 5c. Cropping simulates different viewpoints or local attention and helps the model to learn different parts of the image and contextual information. Luminance change enables the model to adapt to images under different lighting conditions, enhancing the model’s generalization ability in real scenes. Rotation enables the model to learn to deal with cracks or scenes with different orientations.

Divided into a training set, validation set, and test set in the ratio of 6:2:2. The binary classification dataset contains 4000 crack images and 4000 background images; the multi classification dataset contains 1000 crack images of each type, as in Fig. 5b.

Experimental procedure

To improve the performance of the concrete crack classification model, the crack dataset was randomly shuffled, the cross-entropy loss function was chosen as the loss function during training, Epoch was set to 50, the initial learning rate was set to 0.001, and the batch size was set to 8. The training set was input into the model and repeated iterative calculations were conducted to optimize the model parameters. The model that demonstrated the highest accuracy rate on the validation set was determined as the best model for subsequent testing. Finally, the test set is fed into the trained model, and the accuracy, recall, precision, and F1 values are evaluated to verify the generalization ability of the model. The implementation process is shown in Fig. 6.

Fig. 6
figure 6

MobileNetV3-C//Sim implementation flow.

Full size image

Test results and analysis

Evaluation index

Correctly choosing appropriate evaluation metrics can improve the accuracy, robustness, and discriminative ability of the model. In this paper, accuracy, recall, precision, F1 values, and Confusion Matrix are chosen as evaluation metrics. For the classification evaluation indexes, the closer the value is to 1 the better the prediction effect of the model. The formulas of Accuracy, Recall, Precision, and F1 values are shown in Eqs. (4)–(7).

$$Accuracy = \frac{TP + TN}{{TP + TN + FP + FN}}$$
(4)
$$Precision = \frac{TP}{{TP + FP}}$$
(5)
$$Recall = \frac{TP}{{TP + FN}}$$
(6)
$$F1 = \frac{2Precision*Recall}{{Precision + Recall}}$$
(7)

Classification task test results

Figure 7 shows the validation set accuracy curves of MobileNetV3-C//Sim and MobileNetV3 during the training process of the binary classification task. The training on the two datasets shows that the accuracy curve of MobileNetV3-C//Sim rises quickly and smoothly, oscillates after a rapid rise in rounds 1 to 12, and finally reaches the convergence state. The initial accuracy curve of the MobileNetV3 model was lower than that of MobileNetV3-C//Sim, and the subsequent curves were almost always lower than that of MobileNetV3-C//Sim. The above two best-performing models in the validation set were used as test models. The test results are shown in Table 1, the improved model performs higher accuracy, recall, precision, and F1 values than Mobilenetv3 in both Dataset 1 and Dataset 2. In dataset 1, accuracy increased by 0.6%, recall increased by 0.6%, precision increased by 0.48%, and F1 values increased by 0.84%. In dataset 2, accuracy increased by 0.69%, recall increased by 0.69%, precision increased by 0.66%, and F1 values increased by 0.68%.

Fig. 7
figure 7

Accuracy curve of the validation set of the dichotomous model.

Full size image
Table 1 Dichotomous model test results.
Full size table

For the training of the multi classification task, Fig. 8 shows the validation set accuracy profiles of MobileNetV3-C//Sim and MobileNetV3. The accuracy curves of the two models went through three phases of rapid increase, slow increase, and continuous oscillation, and finally reached the convergence state around the 30th round. Similar to the binary classification task, the initial accuracy curve of the MobileNetV3 model was lower than that of this paper’s method, and the subsequent accuracy was always lower than that of MobileNetV3-C//Sim. The test results are shown in Table 2, the improved model also performs higher accuracy, recall, precision, and F1 values than Mobilenetv3 in both dataset 1 and dataset 2. In dataset 1, accuracy increased by 0.46%, recall increased by 0.65%, precision increased by 0.37%, and F1 values increased by 0.53%. In dataset 2, accuracy increased by 0.87%, recall increased by 0.88%, precision increased by 0.76%, and F1 values increased by 0.89%. The experimental results of binary and multi classification show that the improved model performs better performance in data adaptation and feature learning, which enhances the classification performance and robustness of the model.

Fig. 8
figure 8

Multi classification model validation set accuracy curve.

Full size image
Table 2 Multi classification model test results.
Full size table

As shown in Fig. 9, the test results of the MobileNetV3-C//Sim model on two multi classification datasets are visualized by the confusion matrix, where the diagonal lines represent the number of correctly classified, the rows represent the predicted labels, and the columns are the true labels, which gives a clear picture of the model’s misidentification. In both datasets, it is the irregular cracks that have higher misrecognition, which may be due to the inclusion of features of other cracks in the irregular cracks at the same time. But overall, the recognition rate is still high.

Fig. 9
figure 9

Confusion matrix (A) horizontal, (B) Vertical, (C) Diagonal, (D) Irregular.

Full size image

In summary, the improved MobileNetV3-C//Sim model achieved a higher recognition accuracy compared to MobileNetV3, indicating that the C//Sim attentional mechanism better captures the feature information of the cracks, while the higher F1 values reflect that the model works well in balancing precision and recall. In addition, the robustness of the model is verified by comparing the self-constructed dataset with the public dataset.

Comparative tests of different models

We selected some representative convolutional neural networks for comparison. As shown in Table 3, the parameter for MobileNetV3, ResNet50, DenseNet, EfficientNetV2, ShuffleNetV2, and MobileNetV3-C//Sim are 4.21 M, 5.48 M, 9.85 M, 21.45 M, 5.34 M, and 2.90 M, respectively. Compared with the benchmark network, MobileNetV3-Large, the method in this paper effectively reduces the number of parameters by 30.17%. On mobile device deployment, it only requires 11.5 MB, which is lower than other models, indicating that the model occupies very little storage and is suitable for resource-limited environments. In addition, during runtime, the model occupies a certain amount of memory for inference computation, usually within the range of tens of MB, which is acceptable for low-resource mobile devices.

Table 3 Model parameters and memory footprint on mobile devices.
Full size table

To verify the practicality of this paper’s method, we conducted a comparison using the same experimental data and model parameter settings, as shown in Table 4. Among the two classification tasks, only EfficientNetV2 has a slightly higher recall of 0.01% for the binary classification task in dataset 1, but its 21.45 M computational parameter count is 7.4 times higher than that of 2.90 M, whereas a larger number of parameters will consume a large amount of memory space and computational resources. In the rest of the metrics, the method proposed in this paper is equal to or surpasses other models. In conclusion, the improved model proposed in this paper strikes a good balance between accuracy and lightweight recognition.

Table 4 Experimental results of different models on the test set.
Full size table

Ablation test

To test the rationality of the C//Sim attention mechanism, ablation tests were conducted on different combination approaches between the CA attention mechanism and the SimAm attention mechanism to explore the effects of different combination approaches on recognition performance. The combination approaches are displayed in Table 5, which contains parallel connection, front-back tandem combination, and single type.

Table 5 Ways of combining CA and SimAm.
Full size table

The test results are shown in Table 6. In both datasets, the parallel combination is higher than the front and back tandem combination and the single type in all metrics. The accuracy exceeds 0.07–0.8%, recall exceeds 0.04–0.88%, precision exceeds 0.02–0.71%, and F1 values exceed 0.08–0.75%. The reason for this improvement is information complementarity, where different attention mechanisms may focus on different aspects or features of the input data. When combining in parallel, the SimAm attention mechanism can dynamically adjust the weights of the feature maps, making the model more focused on important features. When combined in parallel with the CA attention mechanism, the enhancement of feature extraction can be more significant. In sequential combination, the final attention mechanism loses some of the feature information of the original input image. In addition, the parallel combination can also reduce the computation time. In sequential configuration, one module needs to wait for the completion of another module to continue processing. Parallel combination can make more effective use of hardware resources, reduce training time by about 8%, and improve training efficiency.

Table 6 Ablation test results.
Full size table

During the acquisition process, image data is often affected by such things as noise during shooting and distortion caused by image compression. To verify the reliability and robustness of the C//Sim attention mechanism, we build a new binary classification test set and a multi classification test set, in which the binary classification test set has 400 images with the ratio of the number of cracks to the background of 1:1. the multi classification test set has 400 images with the ratio of the number of transverse, longitudinal, diagonal, and composite cracks of 1:1:1: 1:1. We choose Gaussian noise and salt&pepper noise for robustness testing mainly because these two types of noise are very typical in digital image processing, in addition, during image acquisition, these two types of noise may be introduced due to sensor material properties, operating environment, electronic components, and circuit structure. By adding Gaussian noise and salt&pepper noise, possible scenarios are simulated to evaluate the performance of the model in the presence of noise as shown in Fig. 10.

Fig. 10
figure 10

Test set with different brightness and noise.

Full size image

We compare the C//Sim attention mechanism with other excellent attention mechanisms for noise stress tests using models trained on Dataset2. As shown in Table 7, when the models embedded with different attention mechanisms encounter completely unfamiliar noise, the C//Sim attention mechanism performs the smallest degradation in all the performances and exceeds the other attention mechanisms by 0.65 to 4.28%. This stability indicates that the C//Sim attention mechanism is robust and able to resist noise interference to a certain extent. In the C//Sim attention mechanism, features are extracted and the influence of noise is minimized mainly through the CA module and the SimAm module, and the following mainly explains from two aspects. Firstly, the CA module enables the model to understand the input data more comprehensively and capture the association between features more accurately by considering the global context information and spatial location information. It helps the model maintain accuracy in recognizing key features despite noise interference. SimAm module adopts a parameter-free design to generate attention weights by calculating the similarity between elements within the feature map. This parameter-free design reduces the risk of overfitting to noise while suppressing irrelevant features introduced by noise, improving the generalization ability and robustness to noise. The generated attention weights are able to weight the feature map to emphasize important features.

Table 7 Noise stress tests with different attentional mechanisms.
Full size table

Summary and prospect

Summary

In this paper, the MobileNetV3-C//Sim model, an improved concrete crack recognition model based on MobileNetV3, was developed. The C//Sim attention mechanism introduced in this model extracts spatial features and channel features embedded with position information in the image, which improves the recognition performance of the model while reducing the computational complexity and storage requirements. In order to verify the practicality of the model, this paper conducts tests on self-constructed datasets and public datasets. The improved model performs higher accuracy, recall, precision, and F1 values than mobilenetv3 for dataset 1 and dataset 2, with increases of 0.44–0.69% and 0.46–0.89% for binary and multi classification tasks, respectively. The structural soundness of the C//Sim attention mechanism is verified by ablation tests on different combinations of the CA attention mechanism and SimAm attention mechanism. In the noise test with CA, CBAM, and SE attention mechanism, C//Sim performs the smallest degradation, which verifies that it performs better noise resistance and robustness. In terms of the number of model parameters, the method proposed in this paper involved only 2.90 M, representing a 30.17% reduction compared with MobileNetV3. In summary, the MobileNetV3-C//Sim network model proposed in this paper can accurately recognize crack and non-crack images and can provide a model reference for the research of lightweight recognition of concrete cracks.

Prospect

The method in this paper optimizes recognition performance, but there is still room for improving the recognition accuracy of composite cracks. For example, in previous research based on image processing methods, scholars have specially processed the color and morphology of the crack texture to improve the recognition accuracy; however, in many current studies based on deep learning, few scholars have analyzed the texture features of the crack itself. Therefore, in future research, excellent feature processing algorithms in the past can be combined with deep learning to explore the attention mechanism related to crack texture to improve the recognition accuracy of composite cracks.