Introduction

Surgical procedures, whether involving delicate microsurgery or advanced robotic systems, have traditionally depended on the surgeon’s personal experience, visual judgment, and manual dexterity1,2,3,4. This reliance leads to variability in precision and repeatability, challenges in standardization, which can affect surgical outcomes and potentially increase complications5,6,7. The integration of artificial intelligence (AI) for digital guidance offers a pathway to enhance both the accuracy and standardization of surgeries, providing critical data for the evolution towards surgical robotization.

Previous research has leveraged surgical video data to train AI models, enabling the identification of tissue structures and surgical steps through machine learning and deep learning algorithms8,9,10. These systems have been used to evaluate surgical skills and support training, but their application in real-time clinical decision-making, path design, and intraoperative guidance has been limited.

Cataract surgery, one of the most prevalent microsurgeries globally11, includes a technically demanding step known as capsulorhexis12. Without digital assistance, achieving the optimal shape, position, and size of the capsulorhexis based solely on visual assessment and experience poses significant challenges for surgeons13. Poor execution of capsulorhexis can lead to increased risks of intraoperative and postoperative complications, adversely impacting patient visual outcomes14,15,16,17. The rise in use of advanced intraocular lenses (IOLs) further necessitates precise capsulorhexis to maximize optical performance.

In this study, we introduce Meta Surgery (MetaS), an AI-driven system designed to analyze and extract the characteristics of ideal capsulorhexis from a vast repository of surgical videos, fit ideal capsulorhexis path, and provide real-time guidance for surgeons during cataract surgery. We retrospectively analyzed 17,538 videos of cataract surgeries performed by experienced surgeons to develop and validate MetaS. This system evaluated multiple features of capsulorhexis, selected ideal examples, and captured their digital attributes, such as size and position. To apply the features of ideal capsulorhexis to guide surgeons in cataract surgery, we modified the commonly used intraocular irrigation needle into a scale-equipped lens caliper (LC). It can directly measure and locate the position of capsulorhexis on the lens surface to assist surgeons to perform capsulotomy, demonstrating significant efficacy improvements. Additionally, the digital features from MetaS were successfully applied to guide a surgical robot in performing capsulorhexis on porcine eyes, showcasing potential for autonomous surgical tasks.

Results

Development of the MetaS intelligent system

The MetaS intelligent surgical system incorporates three functional modules, including intelligent evaluation, feature extraction, and surgical guidance. The concept overview of the MetaS system is illustrated in Fig. 1a and Supplementary Fig. 1. First, MetaS automatically evaluates the capsulorhexis quality and screens out ideal capsulorhexis from a large database of images of cataract surgical videos performed based on the surgeon’s experience. Second, MetaS extracts and digitalizes the path and characteristics of the ideal capsulorhexis. Third, these digital features were applied to assist surgeons via a measurable LC to mark or the real-time guided module of MetaS to project the path of an ideal capsulorhexis on the lens anterior capsule, thus improving the surgical accuracy. Moreover, these digital features were also utilized to guide a surgical robot to perform an accurate capsulorhexis independently on porcine eyes.

Fig. 1: Development of the Meta Surgery (MetaS) system.
figure 1

a Schematic of the concept of the MetaS system. b Flow chart of the development of the MetaS system. A large number of cataract surgery videos and images of IOLs and COs were collected to train MetaS. This system can extract the path and features of ideal capsulorhexis, which are subsequently applied to guide surgeons and autonomous surgical robots to perform capsulorhexis and improve surgical performance.

Full size image

A flow chart presenting additional details on the formation of the MetaS intelligent surgery system is shown in Fig. 1b and Supplementary Fig. 2. A total of 26,466 cataract surgical videos were collected. After excluding patients who did not meet the inclusion criteria (details in Methods), 17,538 surgical videos were included. All surgeries were performed by 7 experienced cataract surgeons who have performed more than 5000 cataract surgeries with 5 or more years of experience. The mean age of the patients was 68.3 ± 11.7 years, and 10,651 participants (58.9%) were female. Images of the IOLs and capsulorhexis were obtained from the videos and were classified into three grades “ideal,” “acceptable,” or “poor” according to the quality of capsulorhexis by two cataract specialists. An ideal capsulorhexis opening (CO) should be slightly less than the diameter of the IOL, round, centered, and continuously covering the edge of the IOL13. However, we found that the capsulorhexis created based on the surgeon’s experience tended to be too large, too small, or off center, with only 2931 (16.7%) cases being ideal, 9681 (55.2%) being acceptable, and 4,926 (28.1%) being poor.

Afterward, images representing different grades of capsulorhexis quality were used to train and validate the intelligent evaluation function of MetaS. A dataset containing 14,400 images (about 80%), including 2405 ideal, 7949 acceptable, and 4046 poor capsulorhexis images, was used to train the evaluation module of MetaS, and another 3138 images (about 20%) were randomly selected for validation. No individuals overlapped between the training and validating sets. After training with a deep convolutional neural network (CNN) architecture, Inception Residual Network (ResNet) V218,19, MetaS could recognize corneal limbus (CL), pupils and CO, and the accuracy achieved 88%, 92%, and 86%, respectively. The evaluation module of MetaS further automatically classified the COs into “ideal,” “acceptable,” and “poor” (Fig. 2a). After internal validation, the evaluation module of MetaS achieved AUCs of 0.963 (95% CI 0.956 to 0.970), 0.917 (95% CI 0.907 to 0.926), and 0.960 (95% CI 0.952 to 0.967) for the ideal, acceptable, and poor test datasets, respectively (Fig. 2b).

Fig. 2: Performances of feature extraction and digitalization modules of MetaS.
figure 2

a The evaluation module of MetaS was trained to classify capsulorhexis into ideal, acceptable, or poor using the InceptionResNetV2 algorithm. b ROC curves for the classification of capsulorhexis with our ensemble method. AUC = area under the receiver operating characteristic curve. ROC = receiver operating characteristic. c The Mask R-CNN was used to establish the extraction module of MetaS. According to the ratio of the area of the CL before and after capsulorhexis SCL1/SCL2, the area of the CO, SCO2, was shifted and scaled into the frame before the capsulorhexis, and its area was measured as SCO1. By calculating the area ratio of SCO1/ SCL1 in all ideal capsulorhexis operations, the radius (RC) of ideal CO could be deduced as the radius of the limbus (RL) × 0.58. d The extraction module of MetaS was used to extract the specific features of ideal capsulorhexis using images, and found that the diameter of ideal CO falls within the range of 5.15–5.39 mm. e Ideal CO should be concentric with the pupil and the CL. f The off-center distance of ideal CO should be less than 0.30 mm.

Full size image

Path and feature extraction for the ideal capsulorhexis by the MetaS system

We improved and optimized the feature extraction algorithm Mask Region-based Convolutional Neural Network (Mask R-CNN)20 to develop the feature extraction module of the MetaS system, which was designed to extract the path and specific characteristics of the ideal capsulorhexis from dynamic videos and images. For path extraction from surgical videos, the videos of ideal capsulorhexis were segmented into individual image frames. By identifying and calculating the proportional relationship between the pixel area of the CO (SCO) and the CL (SCL), the radius of ideal CO (RC) could be deduced as the radius of the CL (RL) × 0.58 (Fig. 2c).

Furthermore, the specific characteristics of ideal capsulorhexis were also extracted using surgical images. The feature extraction module of MetaS was capable of automatically conducting batch measurements of multiple parameters of the capsulorhexis. It was found that the diameter of the ideal CO falls within the range of 5.15–5.39 mm (Fig. 2d). The distance from the edge of the CO to the CL was 2.56 ± 0.24 mm, and the distance from all points was basically equal (Fig. 2e). The CO exhibited a relatively high degree of circularity, with a circularity value of 0.98, and is positioned centrally, and the distance from the center of the CO to the center of the cornea was <0.30 mm (Fig. 2f). More specifically, the CO was concentric with both the pupil and the limbus. Taken together, it can be concluded that the ideal capsulorhexis trajectory can be determined as a circle centered at the pupil center with a radius equal to 0.58×RL.

Dramatic improvement in capsulorhexis accuracy with a lens caliper-assisted capsulorhexis using the features of ideal capsulorhexis from MetaS system

To further apply the features of ideal capsulorhexis to assist surgeons, we designed an LC, which was modified from a common intraocular irrigation needle. Standard scale was engraved on the tip of the intraocular irrigation needle, and the tip was processed to form a raised blunt end, which allowed it could be used to determine the accurate size of capsulorhexis and mark on the lens surface during cataract surgery (Supplementary Fig. 3). With the pupil center as the center of capsulorhexis, the surgeon used the LC to measure and locate the position of the ideal capsulorhexis with 5.3 mm diameter, and then gently created 8 key markers of the path on the lens anterior capsule using the blunt head of the LC. Afterwards, the surgeon carried out capsulotomy according to the marks (Fig. 3a, b).

Fig. 3: Dramatic improvement in capsulorhexis accuracy with a lens caliper-assisted capsulorhexis using the features of ideal capsulorhexis from MetaS.
figure 3

a Schematic of the lens caliper (LC)-assisted capsulorhexis. Before performing capsulotomy, the surgeon uses the LC to measure and locate the position of capsulorhexis with 5.3 mm diameter, create 8 key markers of capsulorhexis path on the lens anterior capsule using the blunt head of the LC, and then carry out capsulotomy according to the marks. b Clinical application of the LC to assist capsulorhexis. c The performance of capsulorhexis assisted by LC was compared with that for experience-based capsulorhexis. LC-assisted capsulorhexis dramatically improved the centralization of CO and the alignment of capsulorhexis and the IOL. d The average horizontal and vertical diameters of the experience-based COs were larger than those of the LC-assisted COs, which were closer to the size of the ideal opening. e LC assistance also effectively reduced the proportion of overlarge COs. f LC assistance significantly increased the ratio of complete capsulorhexis-IOL overlap, and decreased the ratio of partial IOL overlap. g LC-assisted capsulorhexis dramatically enhanced the rate of ideal COs.

Full size image

To evaluate the performance of LC-assisted capsulorhexis, we consecutively recruited 9498 patients, all of whom underwent standard phacoemulsification cataract extraction and in-the-bag implantation of a foldable IOL. The mean age was 72.3 ± 0.48 years, and 5795 patients (61.01%) were female. The quality of their capsulorhexis was then compared with that of traditional experience-based capsulorhexis. As shown in Supplementary Fig. 4 and Supplementary Table 1, experience-based capsulorhexis had a high degree of uncertainty, with various opening sizes and positions. The average vertical diameter of them in the experience-based group was 5.50 ± 0.51 mm, and the horizontal diameter was 5.55 ± 0.47 mm, both of which exceeded the generally recognized ideal capsulorhexis diameter range of 5.0–5.5 mm. Owing to the CO being oversized or off-centred, 69.3% of them failed to completely cover the IOL edge, with 18.7% of them covering less than half of the IOL edge. Only ~30.7% could fully cover the IOL edge. In addition, the proportion of ideal capsulorhexis did not significantly increase with the number of surgeries and cumulative surgical experience. Furthermore, the CO sizes varied significantly among different surgeons and even for individual surgeons (Supplementary Fig. 4e). Therefore, the accuracy of experience-based capsulorhexis is insufficient, and there is substantial room for improvement with the aid of digital guidance technology.

Compared with experience-based capsulorhexis, LC-assisted resulted in a significant improvement in capsulorhexis accuracy. LC-assisted dramatically improved the centration and the matching degree of capsulorhexis and the IOLs (Fig. 3c). In the LC-assisted group, the average horizontal capsulorhexis diameter was 5.32 ± 0.48 mm, and the average vertical diameter was 5.35 ± 0.48 mm, both smaller than those found in experience-based group and closer to the ideal size (Fig. 3d). The LC-assisted reduced the proportion of oversized capsulorhexis from 53.0% to 32.9%, off-center capsulorhexis (the off-center distance ≥0.3 mm) from 12.4% to 2.9%, while the rate of complete IOL overlap increased significantly (Fig. 3e, f). More importantly, the overall rate of ideal capsulorhexis increased from 16.7% to 64.1% (Fig. 3g). These results suggest that the accuracy of capsulorhexis can be dramatically improved with LC assistance according to the features of ideal capsulorhexis extracted from MetaS system.

The guidance module of MetaS also dramatically improves the accuracy of capsulorhexis by guiding the surgeon

We also developed a real-time guidance module of MetaS to guide surgeons to perform capsulorhexis using the digital features of ideal capsulorhexis. As shown in Supplementary Table 2, GhostNet backbone integrated with Feature Pyramid Network (FPN Neck) (Model 2) outperformed the standalone GhostNet backbone (Model 1) in all evaluation metrics. We ultimately selected Model 2 to develop the guidance module of MetaS.

At the beginning of surgery, MetaS automatically recognized CL, iris, pupil, and lens. Subsequently, an ideal capsulorhexis path (8 green key points) was immediately provided on the anterior surface of the lens, allowing the surgeon to perform capsulorhexis following the path (Fig. 4a, b and Supplementary Movie 1).

Fig. 4: The guidance module of MetaS provides a customized path to guide the surgeon in real-time.
figure 4

a Schematic of the real-time guidance module of MetaS. The GhosNet backbone integrated with FPN Neck were used for the development of the model. During cataract surgery, the model can identify the CL, pupil, iris and lens and automatically provide the path for an ideal CO (8 green points) with the radius of CL*0.58 as radius at the 1.5, 3, 4.5, 6, 7.5, 9, 10.5, and 12 o’clock positions on the anterior capsule in real-time to guide the surgeon to perform the operation. b Clinical application of the real-time guidance module for capsulorhexis. The surgeon carried out CCC according to the path provided by MetaS.

Full size image

To evaluate the performance of MetaS-guided capsulorhexis, we first consecutively recruited 53 age-related cataract patients who were scheduled to undergo cataract surgery and received MetaS guidance. Subsequently, another 52 consecutive patients were recruited to undergo conventional capsulorhexis without any assistance. All operations were performed by the same cataract surgeon at the ZOC (Y.L.) following a standardized procedure. Compared with experience-based capsulorhexis, MetaS-guided significantly increased the rate of ideal capsulorhexis from 44.2% to 85.0%. In the experience-based group, although 73.7% of capsulorhexis could completely cover the IOL edge, 34.2% of them were too small (with a diameter less than 5.0 mm). In the MetaS-guided group, however, only 9.4% of them were too small. Moreover, 11.5% in the experience-based group were off-center (the off-center distance ≥ 0.3 mm), while there was no off-center capsulorhexis in the MetaS-guided group. These results indicate that with the help of the MetaS guidance module, the accuracy and centricity of capsulorhexis can also be significantly improved.

An autonomous surgical robot can create a precise capsulorhexis opening using digital features extracted by MetaS

Finally, we applied the extracted digital features of ideal capsulorhexis to guide a robot to perform capsulorhexis autonomously. The robot is a new robotic system developed to perform ophthalmic surgeries using a conventional ophthalmic microscope-camera system that guides a hybrid parallel-serial micromanipulator. Its kinematics were analyzed by Nasseri et al. 21. A previous study successfully applied it to perform autonomous clear corneal incision on ex vivo porcine eyes22. In this study, we replaced the end effector of the robot with a diathermic capsulorhexis tip. As shown in Fig. 5, after making a cornea incision, the robotic arm holding a diathermic capsulorhexis tip moved from the initial position along the perpendicular line of the corneal incision to the corneal incision. The distance between the initial position of the tip and the corneal incision was defined as S (Fig. 5b). After the tip entered the anterior chamber through the main incision, it moved straight to the target position according to the diameter of target CO. Once it reached the appropriate position, the tip was moved downward under impedance control until it touched the anterior capsule. Under the control of a remote center of motion (RCM), the surgical robot controlled the tip to complete the capsulorhexis autonomously according to the ideal capsulorhexis path provided by MetaS (Fig. 5c). Using the digital features of ideal capsulorhexis, we successfully created a precise CO with the desired size centered on the anterior capsule of the lens in the porcine eye (Supplementary Fig. 5).

Fig. 5: Autonomous surgical robot creates a precise capsulorhexis opening using digital features from MetaS.
figure 5

a Schematic diagrams of the autonomous surgical robot built to perform capsulorhexis in an ex vivo porcine eye. b The movement path of the robotic arm holding a diathermic capsulorhexis tip in the porcine eye. The robotic arm moved from the initial position along the perpendicular line of the corneal incision, and the distance between the initial position of the diathermic capsulorhexis tip and the corneal incision was defined as S. R is the corneal radius; r is the target radius of the CO (5.3 mm); and a is the maximum RCM angle. c Performing autonomous diathermic capsulorhexis. After the diathermic capsulorhexis tip reached the target position, it completed a diathermic capsulorhexis autonomously according to the capsulorhexis path provided by the MetaS.

Full size image

Discussion

In this study, we developed and validated MetaS, a novel AI-driven system that integrates capsulorhexis quality assessment, feature extraction, and real-time surgical guidance. We also developed a LC and supporting software to translate the digital features of an ideal capsulorhexis into clinical practice. MetaS offers the potential to standardize capsulorhexis by providing digital assistance, moving beyond reliance on a surgeon’s subjective experience. Moreover, we applied these digital features to guide an autonomous surgical robot in creating precise CO in ex vivo porcine eyes.

Capsulorhexis is one of the most important steps in modern cataract surgery, and its success contributes to the safety of phacoemulsification cataract surgery by helping to maintain the stability of IOL and reduce complications12. Large capsulorhexes failing to completely cover the edge of IOL have been found to be associated with IOL anterior movement and decentration, as well as an increased incidence of posterior capsular opacification (PCO)14,15,16,23,24. Conversely, small capsulorhexes may pose significant challenges during subsequent surgical steps, including nucleus chopping and phacoemulsification. They may also increase the risk of postoperative complications such as anterior capsule opacification, capsule contraction, or IOL tilt and dislocation during follow-up17,25,26,27. In addition, a capsulorhexis decentration exceeding 0.4 mm was associated with a 0.25 diopter change in postoperative spherical equivalent, which highlights its potential impact on refractive outcomes15. Moreover, a well-centered capsulorhexis of optimum size is particularly crucial for patients receiving implantation of multifocal or toric IOLs. These specialized IOLs rely heavily on precise centration to deliver excellent visual outcomes, which underscores the significance of achieving an ideal capsulorhexis during surgery28,29. For many years, however, the capsulorhexis size and location have mainly depended on the surgeon’s subjective experience, resulting in distinct fluctuations in the accuracy of the operation.

Our MetaS system consists of a multi-layer neural network framework. Among the classic CNN models like LeNet, AlexNet, Xception, Inception, and ResNet, we chose InceptionResNetV2 due to its superior performance in diagnosing eye conditions such as cataracts, glaucoma, retinopathy, and age-related macular degeneration18,30,31,32. This model achieved over 90% accuracy in classifying capsulorhexis into ideal, acceptable, or poor categories. We further optimized Mask R-CNN for our feature extraction module, which excels in instance segmentation and object detection, providing detailed morphometric measurements of the capsulorhexis, including diameter, area, position, and shape relative to the IOL’s optical surface (6 mm)33,34. Through this analysis, we defined the ideal capsulorhexis parameters: a diameter of 5.15 to 5.39 mm, centered with the pupil and CL, nearly circular (roundness of 0.98), with an off-center distance less than 0.30 mm.

To validate these findings, we developed a modified LC with standard calibration on the common irrigation needle, allowing surgeons to measure and position the capsulorhexis accurately. Our comparisons showed that LC assistance significantly enhanced the precision of capsulorhexis, correcting issues like oversized or undersized openings. Compare with the current advanced technologies like digital overlay device Callisto (Zeiss, Germany), and various different automated capsulotomy devices such as femtosecond lasers, CPASULaser and precision pulse capsulotomy (Zepto), the LC-assisted capsulorhexis does not require expensive equipment, is simple to operate and easy to master, and thus could be used widely in cataract surgery. It should be noted that since pressing on the lens surface with LC to make markers may exert some pressure on the zonule, thus, this technique is not recommended for application in patients with weakened zonule and lens dislocation. Despite its benefits, the LC’s utility in the confined space of the anterior chamber might introduce minor positional inaccuracies.

To address these limitations, we incorporated a GhostNet Backbone-based guidance module into MetaS, which recognizes ocular structures and provides a dynamic path for ideal capsulorhexis. This real-time guidance markedly improved procedural accuracy, allowing for immediate assessment of the capsulorhexis-IOL overlap, which is crucial for selecting the appropriate IOL. Although utilization of femtosecond laser and CAPSULaser can also achieve reproducible, well-centered, and more precise capsulorhexis, these not only incur much higher surgical cost but also complicate the surgical workflow and prolong the surgical duration35. Additionally, some complications or challenges have been reported while using Zepto nano-pulse capsulotomy, including incomplete capsulotomy, radial tear, sudden complete collapse of the anterior chamber, and the iris tissue was sucked into the suction cup36,37. The real-time guidance by MetaS does not require complex equipment, avoids adding to surgical complexity and risks, and thus may be more clinically scalable.

The exploration of robotic systems in cataract surgery, as shown in previous studies using intraocular robotic interventional surgical system (IRISS) with optical coherence tomography (OCT) or the da Vinci Xi system, has been predominantly human-operated38,39,40,41. In our study, we advanced this concept by guiding a surgical robot using MetaS-extracted digital features, achieving precise capsulorhexis in porcine eyes, which could revolutionize surgical consistency and reduce variability due to individual surgeon skill42. In reference to the level-of-automation (LoA) system established for Robot-Assisted Minimally Invasive Surgery (RAMIS), the autonomous execution of capsulorhexis procedure by our surgical robot is categorized as LoA 2-Task-level Autonomy, which can complete certain tasks in an autonomous manner43. However, due to the thicker lens capsules of porcine eyes than human eyes, issues of incomplete and discontinuous capsulorhexis may occur during capsulorhexis, potentially leading to capsule rupture.

However, our MetaS system still has several limitations that require further study and improvement. First, our system was developed based on routine cataract surgery in adults and those whose pupils can be fully dilated. It is not applicable to patients with complicated cataract or those with poorly dilated pupils. Second, the absence of femtosecond laser capsulotomy videos in our dataset limited our model to extract the features of capsulorhexis performed by femtosecond laser. Whether the ideal capsulorhexis features demonstrated in our study are applicable to femtosecond laser capsulotomy still requires further clinical researches. Third, the real-time guidance module is installed on an external computer, and surgeons need to watch an external screen to perform capsulorhexis, which is relatively inconvenient. If the module could be installed under the microscope and synchronized with the image under the eyepiece in the future, this would greatly improve the convenience of the operation. Forth, since it could not record surgeons’ hand-movement trajectories during capsulorhexis procedure under a surgical microscope, we were unable to train our model for the intelligent evaluation of surgical skills, just like other study on the autonomous assessment of surgical skills44. Finally, further studies are needed to validate the safety and accuracy of autonomous robotic capsulorhexis in vivo in patients.

Looking ahead, we will continue to conduct clinical studies to explore the long-term outcomes of patients who underwent cataract surgery guided by MetaS, especially the impact on the postoperative visual function of patients who had multi-focal IOL implantation. Moreover, we will further enhance MetaS’s capabilities with more diverse datasets, potentially integrating it with three-dimensional ophthalmic systems for real-time, immersive surgical guidance. This not only broadens its clinical application but also marks a significant advancement in using AI to achieve safer, more consistent surgical outcomes, lessening the dependence on individual surgeon experience.

Methods

Cataract surgery video and image collection

A total of 26,466 surgical videos from patients (26,466 eyes) who underwent routine phacoemulsification combined with IOL implantation at the Zhongshan Ophthalmic Center (ZOC, Guangzhou, China) of Sun Yat-Sen University from January 2016 to March 2019 were collected for capsulorhexis evaluation. A senior cataract clinician conducted strict screening of the video quality. Videos with the following conditions were excluded: (1) blurry focus; (2) incomplete exposure of the eyeball within the field of view of the video; (3) complicated cataract surgeries, including congenital cataract, traumatic cataract, concomitant corneal diseases and uveitis; (4) poor pupil mydriasis (<6 mm) affecting capsulorhexis evaluation; and (5) occurrence of surgical complications. Finally, 17,538 videos of cataract surgery met the criteria and were included in the study. The screenshots of the IOL and capsulorhexis were obtained for evaluating the quality of the CO after centering the IOL at the end of the operation.

Quality analysis of traditional experience-based capsulorhexis

All IOL and capsulorhexis images obtained from surgical videos were independently analyzed by two professionally trained graders from the ZOC image reading center (Guangzhou, China) using ImageJ image analysis software. The horizontal diameter, vertical diameter, and area of the CO were measured using the diameter of the IOL optical surface (6 mm) as the reference, and then the average values of the two graders were obtained. In addition, the IOL edge coverage of the CO was assessed as 360°, 270 to 360°, 180 to 270°, 90 to 180°, or <90° of coverage and recorded. A capsulorhexis diameter between 5.0 and 5.5 mm and complete coverage of the edge of the IOL was considered ideal capsulorhexis. A CO diameter larger than 5.5 mm was considered too large, while a diameter <4.5 mm was considered too small. When there was a discrepancy between the two graders, a cataract specialist with over 10 years of experience made the final decision.

Image labeling

All capsulorhexis images were further classified into three grades according to their quality, including the capsulorhexis-IOL overlap, centricity, and circularity, as follows: (1) ideal capsulorhexis: centered and round CO, covering the edge of the IOL optical surface continuously for 360 degrees with a diameter between 5.0 and 5.5 mm; (2) acceptable capsulorhexis: smaller diameter (4.5–5.0 mm) or off-center CO completely covering the edge of the IOL optical surface, or IOL overlap of more than 180° but <360°; and (3) poor capsulorhexis: excessively large CO with IOL overlap of <180° or an insufficient opening size (diameter < 4.5 mm). Two cataract specialists who had more than 5 years of experience were recruited to label the images. For disputed images, arbitration was performed by another senior cataract specialist with over 10 years of experience.

Development of the evaluation module of MetaS

A flow chart showing the development of the evaluation and feature extraction modules of our intelligent MetaS system is shown in Fig. 2. A total of 17,538 screenshots of IOLs and COs from videos of cataract surgery were collected and labeled by cataract specialists as ideal, acceptable, or poor capsulorhexis. Afterwards, they were then applied to train the evaluation module of MetaS. Image standardization was performed prior to deep learning algorithm training. All images were resized to 512 × 512 pixels, and the pixel values were normalized to an interval between 0 and 1. Data augmentation was used to increase the image heterogeneity of the training dataset and thus reduce the chance of overfitting during the deep learning process. The new samples were obtained through simple transformations of the original images and corresponded to “real-world” acquisition conditions. Random horizontal and vertical flipping, random rotations of up to 90° around the center of the image, and random brightness shifts within the range of 0.8 to 1.2 were applied to the images in the training set in real time during training.

The training set contained 14,400 images, and the validation set contained 3138 images. No individual images overlapped among these sets. A state-of-the-art deep CNN architecture, InceptionResNetV2, which mimics the architectural features of two previous state-of-the-art CNNs (Residual Network and Inception Network), was used to train the evaluation model18. Weights pretrained for ImageNet classification were applied to initialize the CNN architectures. The model was trained for up to 500 epochs. In the training process, validating ResNet on loss was assessed using the validation set after each epoch and applied as a reference for model selection. Early stopping was employed, such that when the validation loss did not improve over 120 consecutive epochs, the training process would stop. The model state with the lowest validation loss was saved as the final state of the model.

Development of the feature extraction module of MetaS

Furthermore, we improved and optimized a feature extraction algorithm, Mask R-CNN, to establish an extraction module capable of recognizing and extracting the path and features of ideal capsulorhexis procedure20. As an instance segmentation algorithm model, Mask R-CNN can perform pixel-level object segmentation and target recognition simultaneously.

For the extraction of the capsulorhexis path, 2931 videos of ideal capsulorhexis were decomposed into individual image frames. From these, 1063 pre- and post-capsulorhexis images were used for model training. The CL, pupil margin, and their respective centers were manually annotated in the pre-capsulorhexis images using the annotation tool LabelMe (https://github.com/wkentaro/labelme/). Additionally, the edges of CO and the IOL optical surface were annotated in the post-capsulorhexis images. The annotated dataset was then utilized to train the feature extraction module of MetaS. Given the irregular circular shape of the CO, multiple points along its edge were marked to enhance the model’s accuracy in identifying the opening.

The remaining 1868 pre- and post-capsulorhexis images were used to extract the trajectory of the ideal capsulorhexis. First, the model was applied to identify the CL before and after capsulorhexis, and their pixel areas were measured as SCL1 and SCL2, respectively. The CO was also identified, and its pixel area was measured as SCO2. Then, the translational and scaling relationships before and after capsulorhexis were determined by the ratio of SCL1/SCL2. According to this ratio, SCO2 could be shifted and scaled into the pre-capsulorhexis image, and its pixel area was measured as SCO1. By calculating the area ratio of SCO1/ SCL1 across all images, the radius ratio between the CO and the CL was determined, leading to the conclusion that the radius of the ideal CO (RC) was CL radius (RL) × 0.58. The detailed calculation of RC is presented in Eq. (1):

$${RC}={RL}* \sqrt{\frac{{SCO}1}{{SCL}1}}$$

Therefore, by identifying the CL and pupil margin, the path of ideal capsulorhexis can be determined as a circle centered at the pupil center with a radius equal to 0.58×RL.

Furthermore, the feature extraction module of MetaS was also used to extract the specific characteristics of the ideal capsulorhexis from the images of the IOL and CO. MetaS could automatically measure the CO diameter and area, the linear distance between the capsulorhexis center and the cornea center, and the distances between different points along the capsulorhexis edge and the CL (3, 6, 9, and 12 o’clock positions) in batches using the diameter of the IOL optical surface (6 mm) as the reference.

Lens caliper-assisted capsulorhexis versus conventional experience-based capsulorhexis

A total of 9498 patients (9498 eyes) who planned to undergo phacoemulsification with IOL implantation from April 2019 to October 2021 at the Zhongshan Ophthalmic Center were consecutively enrolled. Ethics committee approval was obtained from the Zhongshan Ophthalmic Center (Guangzhou, China) Institutional Review Board (No.: 2019 KYPJ033), and all study procedures adhered to the principles of the Declaration of Helsinki. Patients provided written informed consent prior to enrollment in the study and were not offered any compensation or incentives. The exclusion criteria were the same as those used in the quality analysis for traditional manual capsulorhexis. All the surgeries were performed by the same seven experienced cataract surgeons in the traditional experience-based group.

Before performing capsulotomy, with the pupil center as the center of capsulorhexis, the LC was used to measure and locate the position of capsulorhexis with 5.3 mm diameter. The surgeon could gently use the blunt head of the LC to further make the key markers for capsulorhexis path on the lens anterior capsule, and then carried out capsulotomy according to the marks. After capsulorhexis, hydro-delination and hydro-dissection, phacoemulsification, irrigation/aspiration of the remaining lens cortex, and IOL implantation were performed as usual. After adjusting the position of the IOL to the center, a screenshot is taken for the evaluation of capsulorhexis quality. We further compared the performance for capsulorhexis assisted by the LC with that for traditional experience-based capsulorhexis.

Development of the MetaS real-time guidance module for cataract surgery

In the guidance module of MetaS, we compared the effectiveness of Model 1 (GhostNet backbone) and Model 2 (GhostNet backbone integrated with FPN Neck). Model 1 employed the GhostNet Backbone. Due to its unique Ghost Module design, it can remarkably reduce the number of parameters and computational load while maintaining high accuracy45,46. This enables the model to run at a faster speed, making it suitable for application scenarios that demand quick responses, such as real-time video analysis or continuous image set processing47. It can also support real-time tracking during surgical procedures.

Model 2 was formed by superimposing the GhostNet Backbone and the FPN Neck. By fusing the Feature Map to a quarter of the original image size, it substantially reduced the computational burden and enhanced the system’s real-time processing capabilities48,49. It was more adept at handling continuous detection tasks in dynamically changing environments, like real-time tracking during surgery. Moreover, it further improved the accuracy.

To evaluate the performance of two models, we adopted several common statistical methods in this paper, including accuracy (Acc), sensitivity (Sen), precision (Pre), Matthew’s correlation coefficient (MCC), F1 score, and the area under ROC curves (AUC). Detailed definitions and calculation methods are referred to in previous studies50.

During cataract surgery, the model can automatically identify the eyeball structure, including the CL, iris, pupil, and lens. Then, the model automatically provides an ideal capsulorhexis path (8 points) with the radius of CL*0.58 as radius at the 1.5, 3, 4.5, 6, 7.5, 9, 10.5, and 12 o’clock positions on the anterior capsule in real-time to guide the surgeon to perform capsulorhexis (Fig. 4a, b). To maintain a stable display and automatically track eyeball movement during the operation, an unsupervised video object segmentation algorithm was also used. Finally, to assess the quality of capsulorhexis at the end of the operation, a deep residual network (ResNet) model was also introduced to increase the depth and width of the convolutional network and improve the accuracy. It can measure the actual diameter of the CO and evaluate the corresponding IOL edge coverage after adjusting the IOL position to the center.

Real-time MetaS-guided capsulorhexis versus conventional experience-based capsulorhexis

To evaluate the performance of MetaS-guided capsulorhexis, a total of 53 consecutive patients (53 eyes) with age-related cataract who planned to undergo cataract surgery were recruited between July 2021 and December 2021 to receive MetaS-guided capsulorhexis. Another 52 consecutive patients (52 eyes) were recruited to undergo conventional capsulorhexis without any assistance. All operations were performed by the top cataract surgeon (Y.L.) at ZOC following a standardized procedure. Our real-time guidance module was installed on an external computer and captured the surgical video through a high-definition multimedia interface (HDMI) port. The surgeon could control the guidance system with the connected footswitch during surgery and perform capsulorhexis under system guidance. After capsulorhexis, the following procedures were performed as usual. After adjusting the position of the IOL, our model automatically measured the capsulorhexis diameter and offered quality evaluation results. We compared the performance for capsulorhexis guided by the digital features of ideal capsulorhexis using the real-time guidance module of MetaS with that for traditional experience-based capsulorhexis.

Capsulorhexis by an autonomous surgical robot using the digital features extracted by MetaS

Finally, the digital features of ideal capsulorhexis were used to guide a robot to perform autonomous capsulorhexis in ex vivo porcine eyes. The detailed procedures are described below:

Step 1: The robot arm holds the diathermic capsulorhexis tip and starts at the initial position.

Step 2: The movement path of diathermic capsulorhexis tip is planned from the initial position to the target position in the porcine eye. As shown in Fig. 4b, the diathermic capsulorhexis tip moves from the initial position to the corneal incision along the vertical line of the corneal incision. The distance between the initial position of the diathermic capsulorhexis tip and the corneal incision is defined as S. The detailed calculation of S is presented in Eq. (1):

$$S=\frac{r}{\sin \left(a\right)}-R$$

R is the corneal radius; r is the target radius of capsulorhexis (5.3 mm); and a is the maximum RCM angle.

After entering the corneal incision, the diathermic capsulorhexis tip moves 1 mm vertically upward and then moves to the target position. The distance that the diathermic capsulorhexis device moves within the anterior chamber is calculated according to the diameters of the cornea and target CO. Force feedback technology is used to ensure that the device does not damage intraocular tissue.

Step 3: Perform diathermic capsulorhexis. After the diathermic capsulorhexis tip reaches the target position, it moves downward under impedance control until it touches the lens anterior capsule with a 20mN micro force. Under the control of the RCM, the surgical robot controls the diathermic capsulorhexis device to draw a circular capsulorhexis trajectory and complete the procedure autonomously.

As illustrated in Fig. 4c, the motion track of the motor was calculated according to the capsulorhexis trajectory, the maximum RCM angle a (±12°), and the radius of the CO (2.65 mm).

Step 4: Upon completion of the electronic capsulorhexis, the robot safely exits according to its original entry path.

Statistical analysis

Statistical analysis was performed using StataSE15 (version 15.0, Stata Corp LP, TX, USA). A P value less than 0.05 was considered statistically significant. All continuous variables are expressed as the mean ± standard deviation, and categorical variables are expressed as counts and percentages. To validate the performance of the evaluation module of MetaS, the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity of the capsulorhexis evaluation were calculated according to the reference standard. The 95% confidence intervals (95% CIs) were estimated for all performance metrics. Independent t tests were performed to assess the differences in the capsulorhexis diameter and area between experience-based or verification-guided capsulorhexis and MetaS-guided capsulorhexis. The Pearson chi-square test was performed to assess differences in categorical variables between the two groups.