Artificial intelligence assists the identification and application of rural heritage wall surface damage in Zhejiang

Abstract

Rural cultural heritage is a valuable testament to history and culture, with traditional dwelling walls serving as key research objects. However, in Zhejiang Province, humid conditions and weathering have led to damage such as cracks, holes, stains, and yellowing, posing protection challenges. This study employs the YOLOv8 deep learning model, leveraging multi-layer feature extraction and feature fusion to enhance automated damage detection. Field experiments in Hangzhou and Lishui involved data collection, annotation, model training, and testing. Results indicate high accuracy in yellowing (0.8) and stain detection (F1-score 0.71), with improved performance under high-resolution conditions. Additionally, a Raspberry Pi 5-based mobile detection system enables real-time monitoring, providing a cost-effective and efficient solution for cultural heritage protection.

Introduction

Rural heritage refers to material and intangible heritage with historical, cultural, artistic, scientific, and other values within rural areas^1,2. It is not only a testament to the history and culture of the countryside but also an important foundation for the development of rural society³. Traditional dwellings are an important part of rural heritage because they have extremely high historical, cultural, and artistic value and are highly important for the study of local history, culture, folk customs, and other aspects^4,5. Different villages have different characteristics of rural heritage due to their geographical location, natural environment, ethnic composition, and other factors. In addition, rural heritage can enhance the sense of identity and belonging among rural communities. For example, local residents have a deep emotional connection with ancient buildings and traditional customs in the village. This is an aspect that warrants consideration in the ongoing preservation of rural heritage. Zhejiang Province is located in the Jiangnan area, and the climate is humid, which easily leads to a variety of damage to the walls of traditional houses, such as wall mold, deterioration, wall cracks, aging, corrosion and other problems. This damage needs to be repaired in time to maintain wall integrity and aesthetics and to extend the service life of the building⁶.

Digital technology is developing rapidly in the new era, and information technology, “Internet +”, and artificial intelligence are emerging and constantly changing. The digital protection of traditional village cultural heritage requires the integration of big data, the internet, cloud computing, digital museums, the Internet of Things, and sensor networks; this highlights the scientific thinking of traditional village protection in the new era. China’s “Key Points for Digital Rural Development in 2022” underscores the importance of fostering the growth and development of rural digital culture and advancing the digitization of exceptional rural cultural assets⁷. These key points provide policy support for and promote the combination and application of artificial intelligence technology in rural cultural heritage, which is also needed for current social development. The rapid development of artificial intelligence is playing an increasingly important role in the protection of rural heritage. For example, artificial intelligence can be used to digitally model and repair traditional dwellings and thus provide new ideas and methods for the protection and inheritance of rural heritage^8,9,10; it can also effectively identify and classify cultural heritage buildings. Moreover, artificial intelligence can be used for rural heritage monitoring and management to improve the efficiency and quality of protection work¹¹.

In recent decades, the concept of the digital twin has emerged as a transformative tool for the preservation and utilization of cultural heritage. By enabling the collection, processing, and integration of graphical, spatial, and attribute data, digital tools have demonstrated significant potential across a range of applications, including safety diagnostics, virtual reproduction and exhibition, and heritage management¹². The digitalization of cultural heritage uses digital technologies such as digital collection, digital storage, digital processing, digital display and digital communication to transform, reproduce and restore cultural heritage into a digital form that can be shared and is renewable¹³. Moreover, with appropriate structural design, digitization can be used as a data resource for retrieval, learning, dissemination and utilization. The digitization of cultural heritage is a method of cultural heritage protection that arose from the development of modern science and technology¹⁴. However, with the integration of culture and technology, such digitization is no longer a simple technology of cultural heritage records and preservation but a system of cultural heritage protection, utilization and inheritance^15,16,17,18. SarahYounan et al. suggested that integrating cultural heritage into popular culture through three-dimensional (3D) models, innovating forms of cultural heritage participation, and displaying cultural heritage to people can trigger their thinking on memory and identity¹⁹. Furthermore, the research on digital heritage protection promotes the protection, inheritance, and dissemination of cultural heritage and provides theoretical support for the digital protection of cultural heritage. Many companies and institutions have undertaken projects in the research and practice of heritage digitization. Google’s Cultural Heritage Center, for example, works with museums and cultural institutions around the world to transform artifacts and artwork into digital form through digital technology and provides online display and access²⁰. Another example is the World Digital Library, which is a collaboration between UNESCO and the Library of Congress that aims to digitize cultural heritage materials from around the world to promote cross-cultural exchange and understanding^21,22. Such research plays an important role in promoting the innovation and development of cultural heritage.

The protection of rural heritage has become an increasingly important topic, and the application of current technologies has played a crucial role in this domain. Academic research on digital heritage protection has advanced the digital protection and inheritance of intangible cultural heritage by deeply exploring the application of digital technology in cultural heritage protection. Current work details how digital technology can not only effectively prevent the loss of cultural heritage due to natural disasters²³, man-made destruction or the passage of time²⁴ but also achieve the permanent preservation of cultural relics and buildings through high-definition 3D scanning and digital archiving technology^25,26,27. For example, scholars Hu et al. used high-resolution 3D scanning and photogrammetry to create detailed digital models of rural heritage sites²⁸. These digital replicas aid in preservation efforts and facilitate both virtual access and educational purposes. Remote sensing techniques have also emerged as valuable tools. Scholars have noted that satellite imagery and LiDAR can be employed to monitor the conditions of large rural landscapes and detect changes over time^29,30,31. This aids in the early detection of potential threats, such as land erosion or illegal construction. This technology not only provides an accurate basis for future generations to recover but also enables intangible cultural heritage to be recorded and passed on in digital form by breaking geographical and time constraints. Although digital protection platforms for traditional villages have been established, most of them focus only on collecting and classifying information on cultural heritage and are based mainly on pictures, texts, and videos, with less involvement of 3D simulation models, 3D virtual technology, remote sensing technology, digital restoration, and digital detection.

The use of big data and artificial intelligence in heritage protection has gained momentum in recent years. Yu et al. (2016) asserted that when historical data and environmental factors are used, predictive analytics can identify the maintenance needs and potential risks³². In China, rural heritage protection has emerged as a crucial area of study, reflecting the growing recognition of the significance of rural cultural and historical assets. However, challenges remain. The existing wall inspection tools for Chinese rural heritage have some limitations in terms of technology, cost, and practical application. The acquisition and operating costs of 3D laser scanners, ultrasonic detection and other high-tech measurement technologies are high. This may be difficult for heritage wall detection work in rural areas. In addition, there may be certain errors in the detection accuracy and depth. To overcome these limitations, more suitable testing technologies in rural areas need to be explored and low-cost, easy-to-operate testing equipment needs to be developed to improve scientific and normative testing work. The cost and complexity of implementing some of these technologies in rural areas with limited resources can be prohibitive³³. Additionally, there is a need for standardized protocols and guidelines to ensure the accuracy and comparability of the data collected using different technologies. Accordingly, current technologies offer promising opportunities for rural heritage protection, but their effective application requires addressing issues related to cost, standardization, and integration with local communities and traditional conservation practices.

The YOLO (You Only Look Once) series of models have become core tools in many fields, including cultural heritage protection, due to their rapid and efficient real-time object detection capabilities³⁴. In recent years, cultural heritage protection has been developing in the direction of digitalization and intelligence. YOLO has widely implemented object detection technology in digital archiving, monitoring, and restoration of cultural relics. For example, the YOLO model can quickly detect cracks, peeling, and other damage locations on the surface of ancient buildings, thereby providing accurate positioning support for subsequent restoration work^35,36,37,38. In cultural heritage monitoring, YOLO’s efficient object detection capability can identify threatening behaviors in video surveillance in real time, such as the detection of theft and vandalism at heritage sites³⁹. Additionally, the YOLO model performs automatic classification and annotation of cultural heritage scenes⁴⁰. For instance, the YOLO model identifies the style and color of opera costumes in cultural heritage⁴¹, and it also identifies the human figures on the Han Dynasty portrait stone of ancient Chinese art heritage⁴². Archaeologists use the YOLO model to automatically identify archaeological features in aerial images of heritage sites^43,44. Combined with high-resolution images taken by drones, YOLO can quickly screen out possible archaeological targets on the surface of the site, saving the archaeological team a lot of time⁴⁵.

With the continuous iteration of the YOLO model, the YOLOv8 model has gradually begun to inject new vitality into the protection of cultural heritage. Ultralytics developed YOLOv8, a target detection model. YOLOv8 significantly improves on previous YOLO versions in terms of model architecture, detection accuracy, inference speed, and hardware support. It is especially suitable for devices with limited resources (such as NVIDIA Jetson, GPU, macOS, and other platforms)⁴⁶. The high efficiency and flexibility of YOLOv8 make it a multi-domain task, such as crack detection in ancient buildings and fire detection in historical buildings. The YOLOv8 model provides a solid technical foundation for the protection of cultural heritage with its high efficiency and flexibility^47,48. The YOLO model with artificial intelligence can detect and identify damage features in rural heritage buildings, meet the requirements of rapid analysis and make it possible to expand the protection of rural heritage buildings in terms of reducing costs and improving efficiency.

From a macro perspective, the content of rural heritage protection includes environmental and ecological protection; social, cultural, and structural protection; material cultural protection (architectural heritage); intangible cultural heritage protection; sustainable use of heritage; and management and policy. In the protection of material culture, special emphasis is placed on the protection of architectural heritage, and a protection framework is built around the multidimensional content of traditional rural heritage. This study focuses specifically on identifying surface damage to the brick walls of rural heritage areas via machine learning, establishing a link for artificial intelligence technology intervention in rural heritage protection, and implementing the subsequent four steps (Fig. 1). First, deterioration identification and information storage are needed. The researchers obtained damage information on the surface of brick walls through deterioration identification technology and stored and recorded the deterioration. Next, they built a digital database to annotate the deterioration. They uploaded, encoded, and segmented the surface damage image information to form a standardized deterioration annotation database, which provides systematic data support for subsequent protection work. Third, the dataset was trained using machine learning. To improve the accuracy of deterioration identification and the scalability of the model, artificial intelligence performed machine learning training on the surface damage image data. Fourth, daily inspections and equipment maintenance were performed. Finally, mobile devices for onsite inspections to evaluate deterioration in real time were introduced, and daily maintenance data were recorded through the equipment to ensure the continuous monitoring of deterioration. This study emphasizes improving the speed and efficiency of identification through technical improvement and method integration and is widely applicable to the protection of regional buildings. This multidimensional and systematic protection measure not only achieves the efficient protection and management of brick walls in rural architectural heritage but also enhances the scientific validity and sustainability of architectural heritage protection.

Fig. 1

Artificial intelligence intervention in rural heritage protection.

The innovation of this article is that the researchers proposed and verified an automatic detection method for wall brick damage based on the YOLOv8 object detection model. The application of YOLOv8 improves the degree of automation of the detection process. By accurately and quickly identifying and locating damage, researchers can protect and repair cultural relics faster and reduce further damage to cultural relics caused by environmental factors or human negligence. In this way, solid scientific and technological support for the long-term preservation of regional traditional dwellings in Hangzhou City and Lishui City in Zhejiang Province can be provided. This paper focuses primarily on the following three key questions. (1) According to onsite surveys and photographs, what types of damage are present in the brick walls of traditional folk houses in Zhejiang? (2) How effective is the trained machine learning model? (3) How accurate is automatic detection recognition?

Methods

Study area and image collection source

The research area includes mainly the traditional dwellings in Hangzhou City and Lishui City. Tropical marine air masses and polar continental air masses alternately control Hangzhou’s subtropical monsoon climate⁴⁹. In summer, it is hot and rainy, and in winter, it is mild and rainy, forming a subtropical evergreen broad-leaved forest. The average annual temperature in Hangzhou is 17.5 °C, and it is warming. It has been above 17 °C since the 21st century, and it reached 18.8 °C in 2021 and 2023^50,51. The average annual precipitation is 1483.3 mm, and the average annual rainy days are 146.9 days (approximately 40% of the days in the whole year have rainfall); rainfall is mainly concentrated in the flood season from May to September, and it fluctuates greatly every year, with the highest year being 2.5 times the lowest year. The average annual sunshine hours are 1644.8 h, with a significant annual difference and unstable solar energy resources. Lishui City is situated in a mountainous and hilly area of southern Zhejiang Province, with high terrain in the southwest and low terrain in the northeast. Xianxiang is in the northwest, Donggong Mountain is in the south, and Kuacang Mountain is in the northeast. The main rivers are the Daxi, Xiaoxi, and Longquanxi Rivers, which belong to the upper reaches of the Ou River, and there are large reservoirs, such as Jinshui Beach and Tankeng. Lishui City has a subtropical monsoon climate, with high temperatures and heavy rains in the summer and mild temperatures and little rain in the winter (Fig. 2). The average temperature in January is 6.8 °C, the average temperature in July is 29.4 °C, the annual average temperature is 18.4 °C, and the annual precipitation is 1400 mm⁵². The extreme maximum temperature is 43.2 °C (July 31, 2003), and the extreme minimum temperature is −7.7 °C (January 16, 1967; January 16, 1970). The long-term climate has affected the way that local houses are built, but severe changes in the climate have also affected the surface of the brick walls of houses, causing damage.

Fig. 2

The climate of Lishui City.

Zhejiang Province is a typical eastern coastal region that is affected mainly by monsoons. Heat and rainfall decrease from south to north. The soil zone essentially shifts with latitude and roughly follows a pattern of “red soil–yellow soil–yellow brown soil–brown soil”; the saturation of the soil color decreases in line with these changes⁵³. Some areas have slight differences due to the influence of topography. For example, the Wenzhou City and Lishui City areas in southern Zhejiang Province still preserve simple rammed earth buildings in some remote ancient villages, and the wall color essentially matches the local soil, displaying a highly saturated yellow tone. Therefore, in this study, the handmade water-polished bricks used in rural houses also appear darker yellow. In Hangzhou City, Longmen Ancient Town is the main sampling point. In Lishui City, No. 19 Gaojing Lane, No. 34 Gaojing Lane, and No. 1 Qinglin Village, the top ten war resistance sites, Tan’s House and the former residence of Wuling are the main sampling points (Fig. 3). These are all important local cultural relic protection units, and their architectural features are relatively well preserved. Longmen Town is located in Fuyang District, Hangzhou City, Zhejiang Province, China. Its jurisdiction includes Longmen Village No. 1, Longmen Village No. 3, Longmen Village No. 5, and Longmen Village No. 7. Longmen Village has also been rated as a “Chinese Traditional Village”. In 2008, it received the designation of a Chinese Historical and Cultural Town. It has a typical landscape of a rural ancient town that embodies the traditional Chinese clan inheritance system. Longmen Ancient Town is the birthplace of Sun Quan, the Eastern Wu emperor during the Three Kingdoms period (A.D. 220–A.D. 280). It has a long history and a profound cultural heritage. It retains a strong clan atmosphere and unique folk customs. A large group of ancient buildings from the Ming and Qing dynasties (A.D. 1368–A.D. 1912), which are rare in China, is well preserved. The types of ancient buildings include ancestral halls, halls, houses, ancient towers, stone bridges, and archways. The construction of Tan’s House occurred in the Bingshen and Dingyou years of the Daoguang reign of the Qing Dynasty (A.D. 1836–A.D. 1837)^54,55. It has a reasonable architectural layout and exquisite craftsmanship. It covers 530 square meters, with a total area of 20.20 meters and a total depth of 24.20 meters. A platform of small stones supports the house. It is a through-beam-style structure. It consists of two parts: the main courtyard and the side courtyard. The main courtyard is a two-story building divided into two parts: the first part is the door wing, the patio, the main hall, and the wing rooms; the second part is the patio, the back hall, and the wing rooms. The main hall has three bays, with a niche on the back side of the main hall and plank doors on both sides to enter the back hall. The main hall features a through-beam-style beam with seven purlins and five columns in the center and a front sub-step with a back corridor. On each side, there is a one-wing room with a through-beam-style beam, five purlins, and three columns in the center. The original historical appearance of Tan’s House is well preserved, and it has certain cultural connotations and historical information. It is Lishui City’s largest ancient building and provides a typical example for the study of Qing-style residences in southwestern Zhejiang. The Zhejiang Provincial People’s Government announced Tan’s House as a provincial cultural relic protection unit on March 16, 2005.

Fig. 3

Scope of the study area.

The researchers collected brick wall samples from the side facades of traditional residential buildings with visible damage features and adequate lighting and from the facades of the archways close to the village entrance. The researchers used high-definition cameras to ensure the clarity of the samples. During the field investigation, photographs of many brick wall samples were collected, which totaled 196 large high-definition images (Table 1). Following screening and additional cropping, the researchers obtained 187 valid samples with a smaller size of 512 × 512 pixels. The 187 valid samples covered four different types of damage, namely, holes, white stains, yellowing (or browning), and cracking, and there are 42, 45, 75, and 25 of these images, respectively. After the brick wall samples were collected, the researchers manually classified the damage features, resulting in a summary of the four types of damage. The researchers made it easier for the YOLOv8 model to learn and determine the differences among these four types of damage by dividing them into smaller pieces. This made it possible to accurately find and analyze the damage to wall bricks in real life, which helps to protect and restore rural architectural heritage.

Table 1 Wall brick image collection locations

Wall brick feature analysis and damage types

Traditional Chinese culture is an outstanding representation of ancient Chinese architectural art. Traditional Chinese culture exhibits unique cultural traits in architecture, and people cherish it for its role in safeguarding architectural heritage. Among them, hollow walls have the advantages of less material, a light weight, good heat and sound insulation and are suitable as load-bearing walls of 1–3-story civil buildings or filling the walls of frame buildings⁵⁶. In China, hollow walls are a traditional type of wall. This is also the case for traditional folk houses in Zhejiang Province and the Wuzhou region. In fact, according to traditional construction methods, the hollow wall is not completely hollow. The inside is filled with broken bricks, broken tiles, and broken stones, and a thick slurry mixed with lime slag and sticky loess mud is then poured to combine the filler and bricks. Since the Ming Dynasty (A.D. 1488–A.D. 1605), hollow walls have been widely used to build houses and temples. They are also commonly used in the Yangtze River Basin (including Zhejiang Province) and Southwest China. Figures 4 and 5 illustrate the primary distribution of wall bricks in various residential house locations^57,58,59. According to the researchers’ on-site investigation and interviews, the production of this traditional building hollow wall material and the later installation and construction process will cause damage to the hollow wall due to problems. In the early installation and construction of the hollow wall (Fig. 6), due to the lack of standardized technology and scientific basis, many projects relied on the experience of the master, and there would be errors in the horizontal height of the wall, resulting in cracks in the wall. The problems of holes and discoloration in the bricks themselves can also be attributed to the relatively primitive early firing technology. The raw materials for brick making are mainly clay and sand (Fig. 6a, b). If the clay particles are excessively fine or coarse, it can impact the molding and firing process of the bricks, leading to holes, cracks, and other issues after prolonged use. The bricks are fired in primitive kilns (Fig. 6c, d). The technology for controlling temperature and time is not mature. Cracks, discoloration, and other damages will occur due to too high or too low temperature. In addition, the inaccurate firing time will also lead to an uneven internal structure of the bricks, insufficient strength, and damage.

Fig. 4

Large Hehua and small Hehua wall construction methods in Lishui City.

Fig. 5

Masonry method at the corner of traditional rural houses in Lishui City.

Fig. 6: The production process of hollow brick walls in traditional buildings.

a Illustration of ancient craftsmen panning for sand; b The craftsmen transporting the sand after panning; c The craftsmen transport the raw materials suitable for firing into the kiln; d The fired bricks are used to build buildings.

The damage to wall bricks is closely related to climatic conditions, and different climatic conditions cause different degrees of damage. Moreover, the environmental conditions to which they are subjected and the sizes of the damaged areas that they show differ. After field surveys and image collection analysis, the researchers classified the damage into four main types: holes, white stains, yellowing (or browning), and cracking.

• (1) Holes: Long-term exposure to the natural environment (wind, sun, rain, etc.) may cause bricks to gradually weather and erode, thus forming holes. Furthermore, the formation of holes may result from the collision of heavy objects, human damage, or animal accidents. For example, accidentally colliding with a wall when carrying heavy objects may cause local damage and hole formation.

• (2) White stains: During the construction or renovation process, if certain materials that may contain soluble mineral components are used, then over time and under the influence of environmental factors, these components may gradually seep from the inside of the wall to the surface where they form white stains. For example, the lime component in bricks or mortar dissolves and migrates to the surface in a humid environment, leaving white lime deposits after the water evaporates. The efflorescence backflow phenomenon is another example where the alkaline substances contained in the brick materials in the wall migrate to the wall surface with the water, and the alkaline substances crystallize and precipitate after the water evaporates to form white powder or stains. Alternatively, the wall may have been repaired or protected in the past via paint or grout containing white material, such as limewash. Over time, these materials may have partially peeled or faded, resulting in white stains.

• (3) Yellowing (or browning): Damage can also occur from natural factors. Zhejiang is a coastal area with a high soil salt content, a humid and rainy climate, and frequent typhoons. Groundwater severely erodes the bricks in the buried part of the wall, often in a humid or even water-logged environment. Through capillary action, the saline‒alkali components in the groundwater reach the upper part of the wall and the wall surface and evaporate to form “wall nitrate”. Organic microorganisms and low-level plant spores adhere to the brick wall surface and then interact with gases such as CO₂, NO₂, and SO₂ in the atmosphere, which causes discoloration.

• (4) Cracking: The temperature in Zhejiang Province varies greatly, and thermal expansion and contraction cause stress in brick walls. When this stress exceeds the wall material’s bearing capacity, cracks may appear, especially on longer walls, where vertical cracks are more likely to appear. For example, brick walls expand during a hot summer because of heat and contract during a cold winter. Repeated temperature changes may cause walls to crack. In addition, nonstandard wall construction techniques, such as loose bonding between bricks and insufficient mortar, weaken the strength of the wall and lead to cracks later.

Research process

Based on the YOLOv8 object detection model, the researchers automatically detected wall brick damage in traditional residential buildings in Zhejiang Province. The main processes involved wall brick image data collection, data processing, data labeling, model training, model testing, and result analysis (Fig. 7).

• (1) Data collection: The diversity and representativeness of the data significantly correlate with the stability and generalizability of the model performance. The researchers primarily collected and annotated the image data of wall bricks during their field investigations. Special attention was given to the side facades and village archway locations during the image data collection stage; these structures could represent traditional rural dwellings in Zhejiang. Since this type of wall brick is made of red clay and tends to be orange‒yellow, during the image collection process, the researchers collected images in mainly sunny and cloudy conditions during the day to avoid color differences caused by artificial light sources at night to ensure that as many real scene changes as possible were captured.

• (2) Data processing: In the data processing step, the researchers used a variety of image preprocessing methods to optimize the image quality and ensure the consistency of data input, thereby improving the effectiveness and efficiency of model training. Two examples of such techniques are histogram equalization and noise-filtering techniques. The purpose of these methods is to minimize the impact of environmental factors such as light and shadow on the image, reduce random noise in the image, and improve the clarity of damaged features. In addition, the researchers standardized the image size to ensure that all images input into the model had the same resolution and size. This entails altering all images to a resolution of 512 × 512 pixels, a horizontal and vertical resolution of 96 dpi, and a depth of 24 bits. This guarantees the consistency of the model training and mitigates the computational complexity that originates from inconsistent image sizes and ratios. Finally, codes were introduced for data improvement operations such as rotating and flipping photos in the program part of model training to expand the dataset, which enhance the model’s generalizability and increase its robustness.

• (3) Data labeling: The data labeling stage is the key to ensuring that the YOLOv8 model learns accurate features, so the research team endeavored to provide accurate and consistent labeling information for each image. This study used the professional image labeling tool LabelImg to draw accurate bounding boxes for the damage phenomena in each image and to assign a unique code to each damage type according to its name to establish an accurate correspondence (Fig. 8). The researchers were divided into two groups during the research process. One group classified the wall brick images one-by-one after consulting rural architectural heritage professionals and relevant craftsmen in traditional residential construction. One group subsequently classified and labeled the preprocessed wall brick images, while the other group reviewed the first batch of labels to ensure data accuracy. Two rounds of annotation review and calibration were conducted to ensure the accuracy of each annotation box and category label. The researchers also included the different stages and degrees of various types of damage during the annotation process to enhance the model’s generalizability and guarantee its accuracy in predicting varying degrees of damage.

• (4) Model training: In this study, the YOLOv8 object detection model framework was used to train the model to accurately recognize wall brick damage images. The researchers divided the processed and annotated image dataset into a training set, a validation set, and a test set, with ratios of 70%, 20%, and 10%, respectively, to ensure the adequacy of model training and the effectiveness of evaluation. To optimize the damage detection accuracy, the model training adopted the cross-entropy loss function. Moreover, to increase the convergence speed and improve the stability of the model, the researchers adopted the learning rate warm-up and gradual decay strategies. During the training process, the YOLOv8 model needed not only to learn to identify different types of wall brick damage (such as cracks and holes) but also to accurately predict the size, shape, and location of the damage. As a result, the model training included optimization of the backbone, neck, and head. The backbone uses an advanced convolutional neural network that can extract key features effectively from wall brick images. The neck uses multiscale fusion technology to enhance the model’s ability to recognize small or occluded damage. The head is directly responsible for generating the predicted bounding boxes and classification probabilities.

• (5) Model testing: In the model testing phase, to evaluate the performance of the YOLOv8 model in identifying the types of damage to traditional building wall tiles in Zhejiang Province, this study used a variety of indicators, including the average precision (AP), F1 score, recall rate, and accuracy. These indicators help quantify the model’s detection effect on different types of damage (stains, holes, yellowing, and cracks). In addition, a confusion matrix analysis was used to intuitively display the model’s classification accuracy and misclassification, which facilitates a more in-depth analysis of the model’s recognition effect. Furthermore, the model testing included both a quantitative analysis of performance indicators and the generation of confusion matrices to further analyze the model’s ability to identify various types of damage. The confusion matrix is generated by comparing the true labels (such as stains, holes, yellowing, and cracks) of each wall tile damage image in the test set with the model’s prediction results one by one. The rows and columns of the matrix correspond to the actual categories and the model’s predicted categories, respectively. The values in the diagonal position represent the number of accurate recognitions of that type, and the values in other positions represent the situation where the model misclassifies a certain real damage type as another type. The confusion matrix helps this study determine in detail the true hit rate, false alarm rate, and missed alarm rate of the model in the identification of various types of damage, thereby providing a direction for further tuning the model. Finally, through the feature maps generated during the model prediction process, a deep understanding of the model’s learning and recognition mechanism for damage features was obtained. These feature maps show the effects of feature extraction and fusion when the model handles different types of damage, which verifies its applicability and reliability in practical applications.

• (6) Model application: After the model testing phase, this study developed a mobile detection device based on Raspberry Pi 5 for outdoor real-time damage detection. The device includes a Raspberry Pi 5 development board, a CSI camera module, a servo system, and a power module. It can be connected to a smartphone via WiFi for remote operation and data transmission. The rotation and movement servo system of the device enables the CSI camera to flexibly adjust the angle to adapt to various detection environments to obtain high-quality images of building surface damage. The purpose of this step was to verify the usability and accuracy of the model in a real environment, especially under different lighting conditions and higher resolution images (4096 × 3072 pixels) than the lower resolution (512 × 512 pixels) used in the training phase. During the deployment process, the researchers first optimized the model to adapt to the processing power of the mobile device to ensure the smoothness and responsiveness of the model operation. Additionally, rural heritage buildings such as Longmen Ancient Town, No. 19 Gaojing Lane, No. 34 Gaojing Lane, No. 1 Qinglin Village, the Ten Major Anti-Japanese War Sites, Tan’s House, and the Former Residence of Wuling were selected as test points for field testing. These locations have rich historical value and are typical examples of architectural styles in the traditional villages of Zhejiang Province. Field testing on specific historical buildings allowed the model to more accurately reflect its level of accuracy and reliability and the differences in identifying different types of wall brick damage.

Fig. 7

Research methods and process.

Fig. 8

Label-making process.

Accordingly, this study demonstrated a possible method to automatically detect and identify wall brick damage in the traditional villages of Zhejiang Province by introducing the advanced YOLOv8 model. In particular, the application test on mobile devices aimed to demonstrate the potential efficiency and practicality of the model in a real environment. The researchers expected the trial deployment of this technology to significantly increase the degree of technical support for the long-term maintenance of traditional buildings, offering a fresh perspective on cultural heritage protection. This study aims to extend this technology to a broader range of historical and cultural protection projects and transform it into a crucial tool for safeguarding and enhancing traditional buildings and rural heritage through future practical applications and verification. Moreover, the experience and data accumulated in this study will provide valuable references for the further application of artificial intelligence in related fields.

Model setting and YOLOv8 design

This study uses the YOLOv8 deep learning model to achieve efficient and accurate wall brick damage detection. The model is based on the latest deep learning progress and has a highly optimized network architecture. The advantage of YOLOv8 lies in its fast detection speed and high accuracy, which makes it highly promising for cultural relic protection and building damage assessment.

As previously stated, the network architecture of YOLOv8 consists of three key parts: the backbone, neck, and head (see Fig. 9). The functions of each part are as follows. (1) Feature extractor (backbone): The backbone extracts basic features of the image through a multilayer convolutional network. In this study, ResNet is used as the backbone to capture hierarchical features from basic textures to complex objects. To address the multiscale damage features in complex scenes, the backbone is adaptively configured so that it can process input images of different scales. (2) Feature fusion module (neck): The main function of the neck is to fuse multilevel features from the backbone. This study uses multiscale feature fusion technologies such as the feature pyramid network (FPN) or path aggregation network (PAN) to enhance the model’s recognition ability for small damage or occluded areas. (3) Detection head (head): The head is responsible for generating the predicted bounding boxes and classification probabilities. The multiscale head design of YOLOv8 can simultaneously predict damage types at different scales and improves the coverage and accuracy of detection. This design greatly enhances the adaptability of the model in various damage detection tasks.

Fig. 9

The YOLOv8 model architecture and feature graph used in this study.

Although YOLOv8 has excellent performance, in this study, because of equipment and time limitations, no further hyperparameter adjustment, data enhancement, or cross-validation was performed on the model. Nevertheless, after multiple rounds of training and optimization, the model showed stable accuracy in the identification of damage types. Future work will further explore operations such as hyperparameter optimization, data enhancement, and cross-validation to improve the generalizability of the model in multiple scenarios.

Results

Model training process

In this study, the training process of the YOLOv8 model is a crucial step, and it involves the initial parameter adaptation to the final performance stability. The researchers iterate the entire process through 200 training epochs to ensure that the model can effectively learn and accurately identify different types of wall brick surface damage (Fig. 10). In the first epoch of initial training, the model’s training loss and validation loss were as high as 275.70 and 282.83, respectively. This high loss value reflects the initial adaptability and unoptimized state of the model parameters. Subsequently, in the 25th epoch, the loss value decreased rapidly, indicating that the model quickly adjusted its internal parameters to reduce prediction errors and gradually adapted to the training data. By the 37th epoch, the validation loss value declined to 4.36, which signals that the model began to enter a stage of relatively stable performance.

Fig. 10

The loss statistics during model training.

As training progressed, the training and validation loss values gradually stabilized and remained at a low level. In particular, after the 100th training epoch, the model’s mean average precision (mAP) reached 0.44, reflecting the stability and efficiency of the model after multiple epochs. The model’s prediction accuracy and effectiveness in practical applications directly correlated with its high mAP value at this stage. The training loss finally reached a minimum of 2.16 at the 169th epoch, at which point the model showed a high degree of accuracy and generalizability after optimization throughout the entire training epoch.

Therefore, in this study’s YOLOv8 model training process, the researchers specifically focused on models that demonstrated exceptional performance during the training epochs. The models from these key epochs were selected for further testing to guarantee their optimal performance and highest accuracy in practical applications. This practice’s professional role is to be able to systematically evaluate and compare the strengths and weaknesses of models at different training stages to more accurately determine which model is most suitable for specific research tasks and practical application scenarios. This includes the following. (1) The model at the 37th epoch showed a significant decrease in validation loss. The model at this stage may have adapted well to the training data but may not have been completely stable. It is suitable for a preliminary assessment of the model’s response to the data and the loss reduction trend. (2) The model at the 100th epoch achieved the highest accuracy in the test, indicating that the detection accuracy of the model at this time reached its peak, which shows that the model’s ability to detect damage is already very strong and is suitable for a detailed performance evaluation and fine-grained damage identification. (3) The model at the 169th epoch reached its lowest point in terms of its training loss. At this stage, the model may have performed very well in terms of parameter optimization and overfitting control, which makes it suitable for generalizability testing in complex scenarios. (4) The model at the 200th epoch is the final state of the preset training epoch. It integrates the optimization results of all previous training epochs and is generally considered to be the most stable and mature version. It is suitable for final application deployment and comprehensive performance evaluation.

Comparison of the Model Indicators

The researchers subsequently tested the performance of the YOLOv8 model at the 37th, 100th, 169th, and 200th epochs and evaluated its performance on four types of damage (stains, holes, yellowing, and cracks) via metrics such as the average precision (AP), logarithmic mean error rate, F1 score, recall, and precision (Fig. 11).

Fig. 11

Index test statistics for the model at different iteration epochs.

The test results show that the 100th epoch model performed best in terms of yellowing and stain detection, with an average accuracy of 0.8 for yellowing and 0.46 for stains, while the detection accuracy of holes slightly decreased to an average accuracy of 0.59. Although the model made significant progress in the identification of yellowing and stains, the detection accuracy of cracks never reached an effective level, probably because crack features are relatively rare and difficult to distinguish in the dataset.

In the 200th epoch, the detection accuracy of yellowing reached the highest value of 0.82, and the F1 score also increased to 0.74, indicating that the model gradually optimized the recognition of yellowing during continuous training. However, the detection accuracy of holes and stains fluctuated, suggesting that when the model improves the detection accuracy of certain damage types, this may affect the recognition of other types.

Therefore, the model of the 100th epoch was relatively balanced across multiple damage types and was determined to be suitable as the final model version of this study. The detection of cracks is still a weak link of the model, and its recognition ability needs to be further improved through more data or specialized optimization methods in the future.

Confusion matrix analysis

The study used the confusion matrix to deeply evaluate the model’s classification ability on various damage types, reveal the relationship between the model prediction and the actual label through quantitative indicators, and intuitively display the classification performance and error distribution. The analysis results are as follows:

• (1) In the 37th training epoch’s confusion matrix (Fig. 12), the model had a pretty good recognition rate (77%) for yellowing damage. However, it was clearly not good enough at recognizing other types of damage, like cracks, stains, and holes; cracks in particular could not be correctly classified at all. The fact that crack characteristics are relatively subtle and challenging to distinguish from the complex background may contribute to this limitation. Simultaneously, the misclassification rate for holes and stains is relatively high (48% and 59%, respectively), suggesting a need for further improvement in the model’s recognition ability under background interference.

  Fig. 12

  

  The confusion matrix of the 37th epoch model.

• (2) After the 100th epoch (Fig. 13), the confusion matrix shows that the model has made significant progress in some aspects, especially the correct classification rate of yellowing damage, which increased from 77% to 85%, and the proportion of background misclassification was reduced to 15%. The recognition accuracy of stains has also improved, from 41% to 56%, indicating that the model has gradually adapted to their characteristics. However, the recognition rate of holes dropped from 52% to 48%, indicating that the model may have sacrificed some of its sensitivity to hole characteristics when trying to optimize other categories. Furthermore, the model’s identification of cracks remains at zero, indicating either a lack of sufficient crack samples in the training data or significant challenges in extracting crack features.

  Fig. 13

  

  The confusion matrix of the 100th epoch model.

• (3) By the 169th epoch (Fig. 14), the model’s performance in yellowing damage recognition fluctuated slightly (the recognition rate dropped to 77%), but it still maintained a high degree of stability. However, the accuracy of identifying stains and holes dropped to 46% and 41%, respectively, and the percentage of wrong classifications in the background went up even more. This suggests that the model may have caused performance changes during the long-term training process because of complex feature weighting adjustments. The recognition of cracks has not improved, indicating that the model’s feature learning and generalization capabilities in this category still have fundamental problems.

  Fig. 14

  

  The confusion matrix of the 169th epoch model.

• (4) At the 200th epoch (Fig. 15), the confusion matrix shows that the model maintains a stable performance in the recognition of yellowing damage (77% accuracy), and the recognition rate of stains has rebounded (49%). However, the recognition accuracy of holes remains stagnant at 41%, and the recognition ability of cracks has not seen a breakthrough. This phenomenon mirrors the prevalent issue of class imbalance in multi-class classification tasks, whereby the model’s focus on optimizing certain easily identifiable classes may lead to a decrease in attention to features of other classes. In addition, the long-term difficulty in crack recognition further exposes the limitations of the model in dealing with subtle features and complex backgrounds.

  Fig. 15

  

  The confusion matrix of the 200th epoch model.

Comparative Analysis of the Test Results

To test the model’s performance in actual use, in this section, common types of wall brick damage in the field are selected, including cracks, holes, stains, and yellowing, and the model it tested at four key epochs, namely, the 37th, 100th, 169th, and 200th epochs. Figure 16 shows a detailed comparison and analysis of the performance of various epoch models in identifying different types of damage. By combining this analysis with the previous numerical test results, we can determine the most appropriate model for this study. (1) From A2 to A5 in Fig. 16, each epoch model effectively detects both obvious and subtle cracks. The confusion matrix does not reflect this, primarily because the model’s ability to capture small features is relatively accurate in actual image testing, and the data in the confusion matrix may reflect the limitations of specific samples in the dataset. Despite the consistency of each epoch model in crack detection, previous numerical analyses reveal a low average crack accuracy, necessitating adjustments to the model to accommodate various crack types across a broader test set. (2) In terms of holes, the performance of each epoch model in hole recognition is relatively consistent, but a small number of holes are missed (as shown in B2 to B5), which is in line with the high misjudgment rate in the confusion matrix. In particular, the B2 model misjudged a hole, showing that the model may have specific challenges in dealing with hole recognition. These omissions and misjudgments suggest that there is still room for improvement in the stability of the model’s hole detection ability. (3) In terms of stains, from the results of C2 to C5, the models of each epoch show high consistency in stain recognition. Although there are a few omissions, C2 is slightly more accurate in terms of stain recognition than the other epochs are. The confusion matrix also confirms this, demonstrating that the model’s average precision and recall for stains remains stable across multiple epochs. (4) The recognition of yellowing damage is relatively accurate in the epochs shown by D2, D4, and D5, while D3 indicates some misjudgments. This fits with the high average precision and recall of yellowing in the numerical analysis. This means that, in addition to D3, the models in the other epochs are good at identifying yellowing damage.

Fig. 16

The model test results for different types of damage.

Based on the above analysis and numerical test results, the model of the 100th epoch (especially when dealing with stains and yellowing) presents high average precision and stable recall. Although the model from the 200th epoch continues to perform well in detecting yellowing damage, it shows slight fluctuations in the processing of stains and holes. Therefore, based on the comprehensive performance and stable detection capabilities of each damage type, the model of the 100th epoch is considered the most suitable model in this study. This model provides balanced recognition effects for multiple damage types and especially exhibits high accuracy and reliability when identifying complex and subtle damage features.

Analysis of the Model Feature Layer

This study selects the 100th epoch model for feature layer analysis because of its relatively good performance in various indicators from the above test. As shown in Fig. 17, the feature extraction ability of the YOLOv8 model when processing 512 × 512 pixel images is obvious. By analyzing each feature layer (cv2 and cv3) and its fusion results, a deeper understanding can be gained of how the model processes and identifies different types of damage in the image.

• (1) Feature extraction layer: The cv2 layer usually focuses on lower-level image features, such as edges, textures, and primary shapes. The analysis highlights the red areas (large values) in the cv2 layer image. These are usually preliminary signs of damaged areas, indicating that the pixels at these locations have large changes and are potential damage points. The cv3 layer focuses on capturing more complex attributes, such as the details of specific damage shapes and depth features. The increase in red intensity in the cv3 layer image indicates that the model has detected more obvious feature changes in these areas, which helps to distinguish different types of damage or more accurately determine damage boundaries.

• (2) Feature fusion: The fusion layer integrates the information from cv2 and cv3 to form a comprehensive feature map, which provides more comprehensive damage identification information. In the fusion map, the hot spots (high-value areas) of the damage area are more concentrated and significant, which helps to improve the final recognition accuracy and reliability. The fused feature map shows the significance of the damage area in the feature layer and provides strong support for the location and classification of damage.

• (3) Object detection results: In the object detection results graph, the model successfully marks the damage location using the fused feature map and marks it with different colors (the colors here are used only to distinguish different damage types). This demonstrates that the model effectively converts the image information after feature extraction and fusion into accurate damage identification, thereby confirming its effectiveness in practical applications.

Fig. 17

Model feature layer testing (based on the image size during training).

This experiment was conducted using high-resolution images (4096 × 3072 pixels) taken by a mobile device to further test the model’s usability. Compared with the previous test using 512 × 512 pixel images, the results revealed a difference in the performance of the model at different image resolutions (Fig. 18). In this test, when the model processed high-resolution images, the output of the feature layer clearly showed more details, especially in the cv2 and cv3 layers, where the model captured more local features in more detail; this was in stark contrast to the results of the previous 512 × 512 pixel test. In low-resolution images, the model tends to rely more on large-scale feature extraction, while in high-resolution images, the model can recognize more detailed features.

Fig. 18

Model feature layer testing (based on the image size captured by the device in actual use).

The feature map analysis shows that the hotspot distribution of the cv2 layer is more refined in high-resolution images. This means that the model is better at capturing details, especially in the stain area. This contrasts with the previous test on 512 × 512 pixel images, which primarily concentrated feature extraction from low-resolution images on a wider range of obvious damage areas, resulting in a weaker ability to identify subtle damage. Additionally, in the cv3 layer, the model’s response becomes more focused. This is because it can better recognize complex features in high-resolution images. This allows the model to tell the difference between different types of damage without relying on information about a single feature.

In the feature fusion and final detection results, the model’s performance on high-resolution images further demonstrates its advantage in processing complex images. Compared with the previous low-resolution test results, the model clearly detects more damaged areas in high-resolution images and particularly has a more complete recognition of stains and yellowing. In the 512 × 512 pixel test, the model occasionally misses some stains and yellowing areas, whereas in the high-resolution images, these areas are more accurately labeled, which indicates that the model has better spatial resolution under high-resolution conditions. This also implies that the model might lose some features at low resolution but retains more details of high-resolution images, leading to more accurate detection results.

Accordingly, the YOLOv8 model at the 100th epoch demonstrated its capabilities in feature extraction and processing and proved its practicality and reliability in building damage detection applications by effectively identifying and accurately locating the surface damage features of wall bricks. In particular, when dealing with complex damage scenes, the model’s multilayer feature processing strategy ensured high-precision and high-efficiency detection results, which makes it an ideal choice for wall brick damage assessment.

Discussion

To conduct outdoor experiments, this study developed an autonomous mobile detection device, as shown in Fig. 19. The Raspberry Pi 5 development board is the device’s core control unit. It interacts with a smartphone by connecting to a WiFi network, allowing remote operation and efficient image and data transmission. The device is powered by a 5 V/5 A mobile power bank, which ensures the continuous operation of the device in outdoor environments. The Raspberry Pi directly connects the CSI camera module to the image acquisition part. The rotation servo and translation servo specifically coordinate the servo system, which enables the CSI camera to adjust the angle and position. The servo driver board controls these servo motors to achieve precise camera rotation and movement, permitting the camera to flexibly adjust the viewing angle and shooting position to suit various outdoor test environments. This structural design allows the device to flexibly adjust the camera’s position and angle and results in more comprehensive building surface damage data. Additionally, the T-shaped design of the device base provides for the stable installation of various sensors and driver boards and integration with the development board and power module. This system is highly integrated and portable, which allows it to perform effective mobile damage detection tasks in outdoor environments. It is particularly suitable for refined detection and data collection on architectural heritage surfaces. A smartphone controls the device and enables users to monitor and adjust detection parameters in real time, thereby providing high-quality image input for subsequent data analysis and model detection.

Fig. 19

The testing equipment of this study.

Three representative traditional buildings in Zhejiang Province were selected for this outdoor test: No. 1 Qinglin Village, Tan’s House, and the former residence of Wuling. Figure 20 shows the original image of each site and the damage detection results based on the YOLOv8 model. These results show the following. (1) The test results for No. 1 Qinglin Village primarily identified yellowing damage (A1 and A2 in Fig. 20). The model was able to accurately capture the yellowing area on the brick wall surface and correctly locate and mark most of the yellowing damage. Consistent with the tests in the laboratory, the detection of yellowing damage showed extremely high accuracy at high resolution. The model was also able to identify areas with a wide distribution of yellowing and effectively distinguish between the background and yellow parts. This result shows that the model still has excellent adaptability under complex lighting conditions. (2) In the Tan’s House test, the model primarily detected stains and hole damage (B1 and B2 in Fig. 20). The results show that the model has high accuracy in stain recognition, and it successfully detected many stained areas, especially areas with severe weathering on the wall surface. Although the recognition of holes was relatively successful, there were a few missed detections, particularly some small holes that remained incompletely captured. This is consistent with the test results of the previous experiment. Although the model’s performance in handling small amounts of damage still requires further improvement, its overall detection accuracy already meets the requirements of practical applications. (3) In the test conducted on the former residence of Wuling, the model primarily identified holes and yellowing (C1 and C2 in Fig. 20). The model could accurately identify the locations of the holes and accurately mark them, especially the larger holes, and the model recognition effect was obvious. However, the model still had a small number of missed detections for the smaller holes, which is consistent with the previous indoor test. In addition, the recognition performance of the yellowing area was good. The model could distinguish the yellowing damage area well from the background, especially in the area of the brick wall where the color difference was not large. The model still had high recognition accuracy.

Fig. 20

Field testing the model.

The results of this outdoor test show that the YOLOv8 model has strong adaptability and robustness in identifying different types of building surface damage, especially under high-resolution conditions. The model can accurately recognize a variety of damage types, including cracks, stains, holes, and yellowing. Practical application tests demonstrate that the model is most stable in identifying yellowing and stains, and it effectively and accurately covers the majority of damaged areas. However, there are still a few missed detections in the identification of holes, especially small holes, which shows that the model is insufficient in extracting some feature details. The model demonstrates strong robustness by maintaining high accuracy and reliability in outdoor environments with complex lighting and diverse wall materials, particularly when dealing with brick wall surfaces that exhibit complex textures and color changes. The findings show that the model has broad prospects for application in actual cultural relic protection and the nondestructive testing of building surfaces. Moreover, future research can further optimize the model to detect small-scale damage and complex backgrounds, reduce the occurrence of missed detection and false detection and improve the overall performance of the model.

The humid subtropical monsoon climate characteristics of Zhejiang Province significantly affect the damage to traditional building wall bricks. These environmental factors cause the physical and chemical properties of bricks to change with the seasons and directly affect their durability. Studies have shown that Zhejiang’s hot and rainy summers and cold and humid winters are very likely to cause damage to brick walls, especially yellowing, white stains, and cracks. In a humid environment, wall bricks absorb water through capillary action. The water rises to the surface along the capillary channels in the bricks, carrying minerals in the groundwater, especially salts. After the water evaporates, these minerals remain on the surface of the bricks and result in yellowing or white stains. Such a brick wall near the surface prominently displays a yellowing area, which aligns with the model’s detection of yellowing in the near-surface damage of wall bricks. Through this humidity-driven mineral migration mechanism, the yellowing and white stains on the wall surface are not only a manifestation of material aging but also an important sign of surface damage to traditional brick walls in humid climates. The model closely links the yellowing and white stain areas to the long-term effects of the humid environment, particularly during seasonal changes with significant humidity fluctuations.

Temperature changes also produce stress effects on traditional bricks in Zhejiang. The thermal expansion and contraction caused by temperature differences lead to the accumulation of stress inside the brick, and microcracks are more likely to form in brick walls with fragile structures. Under climatic conditions with large temperature differences between winter and summer, brick walls are affected by the impact of long-term thermal expansion and contraction, especially the significant temperature difference between night and day, which accelerates the expansion of these microcracks. The crack damage detected by the model is mainly concentrated in the parts of the brick body that are subjected to greater stress, which indicates that the formation of cracks may be closely related to the stress accumulation caused by temperature differences and humidity changes. The generation of cracks is not only the result of the impact of temperature fluctuations on building materials but also a manifestation of the damage to the durability of bricks caused by the climate in Zhejiang Province. Combined with existing research results, the physical damage mechanism of bricks caused by a humid climate, temperature fluctuations and alternating temperature and humidity has been confirmed in the test results of this study.

In addition, traditional buildings in Zhejiang mostly use local materials, such as red bricks and mortar, and the properties of these materials also affect the damage pattern. Red bricks, as the main material of the wall, have high hygroscopicity. When the water that they absorb in the humid environment evaporates on the surface, red bricks easily form mineral deposits, resulting in yellowing or white stains on the surface of the brick. The model detection results in this study also show that the distribution of yellowing and stains is closely related to the high humidity environment and the hygroscopicity of the brick material. The mortar material used in traditional buildings becomes loose over time, which weakens the bonding strength between bricks and makes the wall more prone to physical damage, such as holes or cracks, when stress is concentrated. The holes and cracks detected by the model are mainly concentrated in the key positions affected by stress, indicating that these types of damage are related to the aging and material properties of the mortar. Combined with the long-term effects of climatic factors, these characteristics make traditional brick walls in Zhejiang more prone to hole and crack damage in natural environments, demonstrating significant regional characteristics.

In this study, the YOLOv8 model was used to automatically detect wall brick damage, and combined with the climate and material characteristics of traditional Zhejiang buildings, a comprehensive analysis of wall brick damage types was conducted. The identification of typical damage, such as yellowing, stains, and cracks, verified the adaptability of the YOLOv8 model in complex outdoor environments and further revealed the impact mechanism of Zhejiang Province’s climate and material characteristics on the damage to traditional building wall bricks. This analysis method not only detects wall brick damage but also deeply explores the causes of damage and their relationship with the environment, providing an important reference for more effective cultural relic protection and restoration strategies in the future and highlighting the innovative contribution of this study in the academic and practical aspects.

Overall, Zhejiang is located in the central subtropical region and has a humid monsoon climate with moderate temperatures, four distinct seasons, sufficient sunshine, and abundant rainfall. Traditional rural dwellings are situated in the typical Zhejiang climate influencing zone. Long-term wind and rain, inorganic salts, and other environmental factors often erode the rural architectural heritage, necessitating regular monitoring and maintenance measures. However, the scattered villages are not concentrated in the city center. There are many wall bricks, and it is very difficult to manually detect a single damaged piece. In addition to the inconvenience of transportation, the time cost of manual detection is relatively high. To achieve better monitoring, this study proposed and verified an automatic detection method for building damage in the traditional villages of Zhejiang based on computer vision technology. The YOLOv8 object detection model is at the heart of the study. It uses multilevel feature extraction and fusion strategies to address the problems that traditional building damage detection has, such as being slow and expensive. This study introduces a novel technical approach to the field of architectural heritage protection by collecting image data onsite and combining it with an efficient deep learning model. This approach is particularly suitable for the automatic detection and identification of surface damage types such as wall tile cracks, holes, stains, and yellowing in rural architectural heritage protection.

This study used the YOLOv8 deep learning model and evaluated its performance in detecting different damage types by training and testing it for multiple key epochs (37th, 100th, 169th, and 200th). The researchers conducted multiple tests on laboratory images with 512-pixel resolution and outdoor high-resolution images with 4096-pixel resolution to confirm the model’s adaptability and robustness under various resolution conditions. Moreover, a set of outdoor detection equipment based on Raspberry Pi 5 was developed, which integrates a CSI camera, servo system, and remote control module, that can realize real-time data acquisition and surface damage detection in practical applications. Through a series of experiments and practical application tests, this study shows that the following. (1) The model in this study performs very well in the detection of surface damage to rural architectural heritage objects, especially in the detection of yellowing, stains, and large holes. The 100th epoch model achieved the best balance of multiple indicators. The average accuracy of yellowing was 0.8, the F1 score was 0.71, and the recognition accuracy of stains was close to 0.56. Although the recognition of holes revealed some missed detections, the overall performance remained at 0.48. (2) The model performed stably in the detection of yellowing and stains and could handle damage recognition in complex backgrounds well. (3) Outdoor high-resolution tests further verified the practicality of the model. The model significantly improved the detection effect of various types of damage in high-resolution images, especially the ability to capture details.

Although the model performs well in detecting various types of damage, it still has certain limitations in identifying fine cracks and small holes. These limitations include the following. (1) In both laboratory tests and actual outdoor tests, the crack detection performance is not ideal, and the average accuracy of crack damage is almost 0 in multiple tests, which shows that the model still lacks sufficient ability to identify such subtle and difficult-to-distinguish features. (2) Although high-resolution images improve the overall detection accuracy, the large computing resource requirements pose a challenge to field applications, especially when processing high-resolution images in real time. The computing power of the equipment becomes one of the limiting factors.

Future research will focus on solving the problem of subtle damage identification. This study presents the following prospects. (1) The model’s performance in complex environments can be improved by increasing the number of fine crack samples in the training data or introducing new feature extraction techniques. (2) The operating efficiency of the model can be enhanced in low-computational-resource environments, and the delay in high-resolution image processing can be reduced by lightweighting the model and optimizing the algorithm. (3) This research can be further extended to other types of architectural heritage protection projects, including wooden structures and stone buildings, to verify the applicability of this method to other materials and structures and to develop more intelligent detection tools suitable for cultural relic protection.