Introduction

Amidst the rising utilization of computed tomography (CT) for lung cancer screening, a noteworthy surge in the identification of pulmonary nodules has occurred1. Small pulmonary nodules with a diameter ≤ 10 mm (sub-centimeter pulmonary nodules, SCPNs) are observed in approximately 14.5% of the screened population, of which 55.5% of the detected lung cancers were SCPNs2. The utilization of CT-guided percutaneous transthoracic needle biopsies (PCNB) and CT-guided percutaneous transthoracic thermal ablation has become widespread for the diagnostic and therapeutic management of pulmonary nodules3,4,5. Nonetheless, the percutaneous transthoracic puncturing of small pulmonary nodules remains a challenging endeavor, even with the integration of advanced CT technology. The inherent limitations in the orientation of imaging planes for needle trajectory, compounded by the confined gantry bore dimensions of CT systems and the absence of real-time imaging capabilities, collectively contribute to the technical intricacies of CT-guided percutaneous lung punctures for small pulmonary nodules. Consequently, the diagnostic precision of CT-guided PCNB for small pulmonary nodules exhibits significant variability, with reported accuracies ranging from 52–90%6,7,8,9.

Electromagnetic guidance system has been developed to provide guidance for percutaneous pulmonary nodule puncture10. By establishing a comprehensive magnetic field around the patient, this system precisely delineates the sensor’s spatial coordinates within it. These coordinates are seamlessly integrated into existing CT images, providing a clear visual correlation between the sensor’s position and the surrounding anatomy. This technology is validated as secure, feasible, and time-efficient. Nonetheless, the current system, optimized for electromagnetic navigational bronchoscopy, presents certain operational complexities. It solely visualizes needle position and orientation in two-dimensional CT slices, lacking three-dimensional anatomical representation11. Consequently, a more streamlined and intuitive guidance approach for transthoracic percutaneous puncture of small pulmonary nodules is urgently needed.

Deep learning algorithms have shown great potential in medical image analysis tasks, including pulmonary nodule detection12, breast cancer diagnosis13, cerebral hemorrhage segmentation14, and medical image reconstruction and registration15,16. Previous studies have also demonstrated the ability of deep learning algorithms to automate the segmentation of anatomical structures and assist in surgical planning17. Combining deep learning algorithms and electromagnetic needle tracking techniques has great potential in providing very accurate needle targeting of small pulmonary nodules. However, the application of deep learning algorithms to facilitate percutaneous lung puncture remains unclear.

In this study, we developed a novel deep-learning based electromagnetic navigation system (DL-EMNS) to assist percutaneous transthoracic puncture for small pulmonary nodules. The technical reliability and effectiveness of the proposed DL-EMNS were evaluated through phantom and animal experiments.

Materials and methods

All current research methods were carried out under guidelines approved by the Ethics Committee of The Second Xiangya Hospital of Central South University. All experimental procedures involving animals were conducted in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the IACUC of The Second Xiangya Hospital of Central South University (No. 2021842).

Development of DL-EMNS

By combining multiple deep learning models with spatial localization technologies, the proposed DL-EMNS provides operators with the detailed information of patient’s three-dimensional (3D) anatomy structures, the real-time position of the needle, and the virtual puncture trajectory.

The DL-EMNS system consists of a control system, a magnetic field generator, three patches, a sensor, and a monitor (Fig. 1). The performance of DL-EMNS mainly contains two steps: pre-operative preparation step and intra-operative navigation step, as shown in Fig. 2A. The first includes segmentation of thoracic organs and puncture trajectory planning based on multiple deep learning models. The second includes localization of positioning patches and real-time tracking based on electromagnetic technology. The whole navigation system is shown in Fig. 2B.

Fig. 1
figure 1

The components of the navigation system include: 3 patches (A) for spatial registration, 1 sensor (B) for tracking the position of the needle within the magnetic field, 1 field generator (C) for creating the magnetic field, and a navigation controller and monitor (D).

Full size image

Preparation step

(A) 3D reconstruction of thoracic structures. Lung nodules and thoracic structures, including lobes, bronchus, vessels, chest wall, and bones, were automatically segmented and reconstructed using deep learning-based networks. The algorithm has been proven effective in assisting lung segmentectomy in a short time frame18. The 3D models can directly display the thoracic structures and the target lesion. Supplemental Materials Methods-1 provides more detailed information.

(B) Puncture trajectory planning. Once the puncture point on the body’s surface is chosen, the system will automatically create a virtual puncture path by connecting the selected puncture point with the nodule. This allows real-time visualization to determine if there are obstructions such as ribs or scapulae along the puncture path, as well as important blood vessels, trachea, etc., thereby avoiding puncture failure or complications. It will also continuously provide real-time distance feedback between the puncture needle and the nodule throughout the entire puncture procedure.

Navigation step

In the prior art, manual marking of internal or external feature points is generally required for registering preoperative images and intraoperative patient coordinates, resulting in inadequate puncture accuracy19. We novelly adopt positioning patches with distinct characteristics to establish the coordinate mapping relationship in real-time. Positioning patches are sensors fixed on the patient’s surface for real-time tracking of the patient’s posture. Three positioning patches with different shapes were involved in the system. An efficient and accurate localization algorithm of positioning patches was developed to rapidly align the image coordinate system with the patient’s physical coordinate system. The localization of positioning patches was predicted from coarse to fine. A ResUnet model was introduced to coarsely segment and detect the patch centers. The coarse center coordinates were then input to a fine segmentation model, returning the final center coordinates and segmentation labels of the positioning patches. The segmentation results were used for location confirmation and graphic visualization, and the fine patch center positions served as the fixed image reference coordinate system (Fig. 2C). More information was provided in Supplemental Materials Methods-2.

Operation process

At the beginning of the operation, the subject is placed on a magnetic field generator, and three patch sensors were fixed on its skin in a triangle shape covering the nodule. The subject is subsequently positioned on a CT scanner (Siemens SOMATOM Perspective), then its CT images were acquired to reconstruct and visualize the nodules, ribs, bronchus, and vessels. The puncture trajectories were automatically planned in the 3D model as descripted before.

During the procedure, the physical space of the subject was mapped to the preoperative image space based on the relationship of the three fixed sensors on both coordinate systems under the positioning of the electromagnetic device. The puncture site was decided with the assistance of the guidance system. When the operator positioned the needle tip at the surface of the phantom or animal’s thorax, the system automatically provided a simulated puncture path, indicating whether there were ribs or major blood vessels along the path and showing the distance from the needle tip to the target. As the operator moved the needle tip across the surface of the phantom or animal’s thorax, the simulated puncture path was updated in real-time. This allowed the operator to choose the puncture site based on factors such as distance to the target and the presence of obstacles.

Operators then conducted the transthoracic percutaneous lung puncture under the guidance of virtual insertion planning. When the needle reaches the nodule on the real-time guided image, a CT scan was then conducted for evaluation. If the needle tip reached the nodule, the operation was completed. If the needle tip did not reach the nodule, another deep-learning based 3D reconstruction, localization of positioning patches and puncture trajectory planning was conducted again according to the latest CT images. The procedure process is shown in Fig. 2D.

Fig. 2
figure 2

Technical flow chart of the DL-EMNS. (A) The technical flow chart of pre-operative preparation step and the intra-operative navigation step. In the first step, after a CT scan, images were segmented and reconstructed by deep learning-based approaches. In the second step, the real coordinate system in the magnetic field is fused with the three-dimensional reconstructed image, enabling the real-time display of the puncture needle, thoracic structures, and target nodules in the virtual image, thereby indicating the position of the puncture needle within the thoracic cavity. (B) Schematic diagram of DL-EMNS. (C) Schematic diagram of magnetic navigation system and coordinate matching. (D) schematic diagram of the operation process.

Full size image

Phantom study

Three customized thoracic phantoms were used to validate the accuracy of the DL-EMNS, as shown in Fig. 3A-C. The “skin”, “ribs”, “vessels”, “bronchus”, and “nodules” of the phantoms were made from different materials, which could be distinguished on CT images (Fig. 3D). Each side of the “chest cavity” of each phantom is equipped with 3 nodules. The nodule sizes of the three phantoms are 6, 8, and 10 millimeters, respectively. Five doctors used DL-EMNS and traditional methods to simulate puncture on the nodules of these three phantoms respectively, with a wash period of 2 weeks (Fig. 3E). After more than five unsuccessful attempts at reaching the target, the operator’s confidence is often lost, and the procedure is then considered a failure. More information was provided in Figure S1.

In vivo animal study

An animal study was conducted to further evaluate the performance of the DL-EMNS in vivo, as shown in Fig. 3F and Figure S2. Nine Bama pigs, including four boars and five sows, with weights ranging from 19.5 kg to 35 kg, were used in the animal experiment. All experimental animals were obtained from qualified suppliers, ensuring compliance with ethical standards and regulatory requirements. The sex of the animals had no effect on the test results.

The animals were randomly assigned to either the DL-EMNS group (n = 6) or the traditional CT-guided group (n = 3). The Bama pigs were intubated after the induction of anesthesia, and maintained with orotracheal intubation during the operation. To mimic a small lung nodule, 0.1 ml of tissue adhesive (B. Braun Surgical SA) mixed with 0.1 ml of iohexol was randomly injected into the pig’s lung parenchyma under CT guidance, as we described before20. Three doctors used traditional methods and DL-EMNS to simulate puncture on the nodules of these nine pigs respectively (Fig. 3G). Six pigs underwent four punctures per side of the thoracic cavity. Two pigs, due to their smaller body weight (and consequently smaller thoracic cavities), received three punctures per side. One pig had four punctures on the left side and five punctures on the right side. After the experiments were completed, the pigs were sacrificed by injecting 2 mmol/kg of potassium chloride to induce asystole.

Fig. 3
figure 3

The phantom study and animal study. (A) The appearance of the customized thoracic phantom. (B) The customized phantom after removing the “skin”. (C) The internal structure and stimulated nodules (blue arrow). (D) The CT cross-sectional view of the simulated nodules (blue arrow) during puncture. (E) Design of the phantom study. Five doctors used traditional CT guidance way and DL-EMNS to simulate puncture on three models, respectively. (F) The process of animal study. After general anesthesia and tracheal intubation, the experimental animals lay on their side on the CT bed with magnetic navigation coil. The doctors performed percutaneous punctures of the nodules under the guidance of DL-EMNS or traditional CT. (G) Shows the schematic diagram of animal experiment. Three doctors performed traditional and DL-EMNS punctures on nine pigs, respectively.

Full size image

Evaluation criteria

Technical success is defined as when the needle tip touched the nodule. On the contrary, if there is a technical malfunction or other reason (such as pneumothorax, hemothorax) during the procedure that prevents the operation from being completed, it is defined as a technical failure. Error is defined as the spatial distance between the center of the simulated nodule and the needle tip measured on the CT scan. It is used to evaluate the accuracy of needle placement. Additionally, the procedure duration (time from the start of the puncture until the nodule was reached or the operation ended) and the total number of CT acquisitions were recorded. For the animal study, complications such as hemothorax and pneumothorax were recorded.

Statistical analysis

All statistical analyses were performed using SPSS software (version 21.0). Continuous variables were compared using the Mann-Whitney U test, and categorical variables were compared using Fisher’s exact test. A two-sided p-value less than 0.05 was considered statistically significant.

Results

Performance of DL-EMNS in the phantom study

In the phantom study, five physicians performed 90 “transthoracic percutaneous pulmonary punctures” on three phantoms, with 45 punctures using traditional CT guidance, and 45 punctures using DL-EMNS.

The DL-EMNS significantly improved the performance of the punctures, with a higher technical success rate compared to the traditional CT-guided group (95.6% vs. 77.8%, p = 0.027), as shown in Fig. 4A. Puncture success rates showed improvement across all nodule sizes. Particularly noteworthy was the marked enhancement in success rates observed specifically among 6 mm nodules (Fig. 4B). Of the 5 participating physicians, 4 demonstrated an improvement in puncture success rates upon integrating DL-EMNS into their procedures, while one physician (Doctor C) experienced a marginal decrease in success rate (Fig. 4C).

Additionally, as shown in Fig. 4D and E, the error was smaller in the DL-EMNS assisted group (2 mm, [0–2]) than in the traditional CT-guided group (4 mm, [2–4]) (p < 0.001). Furthermore, in the puncture of three nodules, the DL-EMNS system significantly shortened the puncture time (291.6 ± 150.3 s vs. 676.4 ± 246.1 s, p < 0.001) and reduced the number of CT acquisitions (1.2 ± 0.66 for the DL-EMNS vs. 2.93 ± 0.98 scans for the traditional CT-guided group, p < 0.001) (Fig. 4F-I).

Fig. 4
figure 4

Results of the phantom study.

Full size image

Performance of DL-EMNS in the animal study

In the animal study, a total of 69 nodules were prepared with an average size of 9.7 ± 2.8 mm. 25 nodules were punctured under traditional CT guidance, while 44 nodules were punctured under DL-EMNS guidance. There was no significant difference in nodule size between the two groups (9.32 ± 2.78 mm vs. 9.39 ± 2.14 mm, p = 0.912, Fig. 5A).

Compared to the traditional CT-guided group, the DL-EMNS assisted punctures showed a higher success rate (100% vs. 84%, p = 0.015) in all three participated operators (Fig. 5B-C).

Furthermore, as shown in Fig. 5D-E, the DL-EMNS system significantly shortened the puncture time (121.40 ± 38.97 s vs. 321.60 ± 129.10 s, p < 0.001) and reduced the number of required CT acquisitions (1 [1, 1] vs. 3 [2, 3], p < 0.001). No pneumothorax or hemothorax were observed in the DL-EMNS assisted group, while in the traditional CT-guided group, the incidence of complications was 20.0% (5/25), with two cases of pneumothorax, two cases of hemothorax, and one case of both pneumothorax and hemothorax (Fig. 5F).

Fig. 5
figure 5

Results of the animal study.

Full size image

Discussion

In this study, we introduced a novel deep learning based electromagnetic navigation system for percutaneous transthoracic puncture of small pulmonary nodules. Implementation of the DL-EMNS significantly improved the accuracy of needle positioning, reduced the CT acquisitions in a series of phantom and animal studies. To our knowledge, this is the first study to use deep learning algorithms to assist percutaneous pulmonary puncture for small pulmonary nodules.

Accurate needle placement is a vital factor in ensuring the success of CT image-guided percutaneous interventions. Incorrect needle positioning can result in wasted time, unnecessary X-ray exposure, adverse events, and various failures, such as obtaining non-diagnostic biopsy samples or incomplete ablation. The outcome of the CT-guided intervention can be greatly influenced by both its complexity and the individual skill of the physician, especially when the lesion is very small or an out-of-plane trajectory is necessary to achieve the anatomically safest route. Studies have shown that the smaller the nodule, the lower the success rate and the higher the complication rate21,22. As shown in our phantom and animal study, the technical success rate in the traditional group for percutaneous puncture of small pulmonary nodules was 77.8% and 84%, respectively, which is similar in previous studies23. This is because traditional CT-guided percutaneous lung puncture is performed free-handed, and the puncture site is determined by a positioning grid, which relies heavily on the operator’s experience22. Continual adjustment of the needle’s direction and depth can prolong the procedure time and increase the risk of complications, while repeated CT scans increase radiation exposure to patients24,25.

Several guidance and navigation systems including electromagnetic, optical, or hybrid tracking of devices during interventions have been introduced for more accurate and reproducible puncture results26,27,28. Electromagnetic navigation systems have demonstrated significant potential in facilitating the manipulation of puncture probes with enhanced flexibility and ease. These systems typically consist of a field generator that generates an electromagnetic field, alongside sensors capable of measuring the magnetic field strength, registering their position and orientation relative to the field generator. The sensors are arranged in a tetrahedral configuration of coils. Their compact size allows for integration into various tools, including needles. With a technical accuracy of 1.5 mm for position and 0.4° for orientation, electromagnetic navigation systems offer precise tracking capabilities. The electromagnetic navigation system has been proven to be effective and safe for CT-guided percutaneous lung biopsy of peripheral lung lesions29. The system allows users with no prior experience of the system to perform faster and more accurate CT-guided biopsies with out-of-plane trajectories30. Virtual electromagnetic tracking demonstrates a remarkable level of accuracy in needle placement, offering the potential to decrease both time and radiation exposure when compared to conventional CT techniques for small lesion biopsies31.

The highlight of our proposed DL-EMNS is the combination of deep learning models with spatial localization technologies. Automatic detection, segmentation, and reconstruction of lesions and anatomical structures are implemented by the involved deep-learning method, which can greatly shorten the operation time. Real-time tracking of the needle during the entire procedure significantly improves accuracy with the guidance of automatic puncture path planning. Additionally, innovative positioning patches work as sensors to track the real-time position of the patient and serve as external feature points for automatic and efficient registration of preoperative image coordinate system and intraoperative patient spatial coordinate system. The combination of deep learning and electromagnetic technology improves puncture accuracy and reduces the number of CT acquisitions compared to traditional CT-guided approach, especially when the nodule is small or an out-of-plane trajectory is necessary to achieve the anatomically safest route.

Several limitations are evident in this study. Firstly, the reliance on animal and phantom experiments falls short of fully replicating human conditions. Although this study employs nodules with consistent shape and density to isolate the effects of size variations, we recognize that real-world nodules in the human lung exhibit diverse morphologic and density characteristics. Future investigations could expand on this work by incorporating nodules of varied shapes and densities, which would provide additional insights into the robustness of the proposed methodology. Secondly, the absence of breathing artifacts in the phantoms used in our study represents a significant limitation. Puncturing subcentimetric nodules in human lungs is more complex than described in our study due to the patient’s conscious state, physical movements, and intricate respiratory patterns, which can interfere with the needle trajectory and make the puncture technically challenging. Additionally, the need for multiple needle replacements posts initial puncture, such as for biopsy or ablation procedures, introduces heightened risks of displacement, bleeding, and pneumothorax. To address these gaps, our next endeavor will encompass a randomized controlled clinical study, aimed at robustly validating the feasibility of DL-EMNS in pulmonary nodule puncture. Concurrently, we are committed to the development of integrated biopsy and ablation needles equipped with magnetic navigation sensors. We are also in the process of designing a puncture robot based on DL-EMNS, aiming to minimize the impact of hand tremors during the puncture procedure. This design aims to make the puncture process simpler and more precise. Through these strategic advancements, we aspire to enhance the precision and safety of our approach, yielding substantial clinical benefits.

In conclusion, our study developed a pulmonary puncture navigation system by applying a deep learning approach. We demonstrated the system’s capability for improving the accuracy of needle positioning, as well as reducing operation time, total number of CT acquisitions and complications. Its effectiveness in clinical practice still requires a large sample randomized controlled trial to verify.