Introduction

Arteriovenous (AV) access in the form of an AV fistula or graft is important for long-term hemodialysis in patients with kidney failure requiring kidney replacement therapy1. In the United States, more than 800,000 individuals are on kidney replacement therapy2. Globally, the most common form of kidney replacement therapy is dialysis, and approximately 90% of patients who undergo dialysis are treated with hemodialysis3. Surgical AV access is preferred over central venous catheters (CVC) for long-term hemodialysis as it reduces the risk of infectious, thrombotic, and other complications related to a foreign body exposing the circulatory system to the environment4. However, up to 60% of AV accesses may not be usable for dialysis due to insufficient maturation or other complications5. Furthermore, the 2-year cumulative patency for AV accesses is less than 70%5,6. Maturation and cumulative patency rates are strongly associated with clinical decisions regarding AV access planning, including the choices of inflow artery, outflow vein, and fistula vs. graft, among others5,6. Furthermore, it can take weeks to months to determine whether an AV access has matured sufficiently for hemodialysis7. In the event of an unsuccessful AV access creation procedure, whereby the access cannot be used for hemodialysis, the patient generally requires a temporary CVC, additional access planning, and re-intervention and/or creation of a new AV access. This may lead to significant patient burden, clinical risks, and costs8,9. Therefore, accurate prediction of the long-term outcomes of an AV access may support better access planning, monitoring, and patient counselling to facilitate personalized care and potentially improve access outcomes.

Currently, there are no widely used tools to support the prediction of the successful clinical use of a planned AV access for long-term hemodialysis using pre-operative data. For example, Heindel et al.10 and Robbin et al.11 both used up to 6-week post-operative ultrasound measurements as input features for their models to predict AV fistula maturation and/or success rates10,11. Although there are good prospective data to show that post-operative ultrasound measurements can help predict AV access readiness for use, previous work has not been able to identify strong correlations between pre-operative variables and AV access outcomes, potentially due to smaller sample sizes and restrictive clinical cohorts10,11. These limitations could be addressed with the use of clinical registry data, such as the Vascular Quality Initiative (VQI)12. A pre-operative prediction model may help guide decision-making regarding the index AV access creation procedure (e.g., choices of inflow artery, outflow vein, and/or fistula vs. graft) and complement the use of existing post-operative ultrasound predictions for AV access maturation and readiness for use, supporting improved AV access creation and management. Elsewhere, the Society for Vascular Surgery (SVS) Cardiac Risk Index (CRI) predicts outcomes following aortic, carotid, and peripheral arterial interventions, but not AV access creation procedures13. Other tools such as the National Surgical Quality Improvement Program (NSQIP) online surgical risk calculator use modelling techniques that require manual input of clinical variables, which deters routine use in busy medical settings14. Therefore, there is an important need to develop better and more practical outcome prediction tools for patients being considered for AV access creation.

Machine learning (ML) is a rapidly advancing technology that allows computers to learn from data and make accurate predictions15. Using advanced analytics, ML can model complex relationships between inputs (e.g., patient characteristics) and outputs (e.g., clinical outcomes)15. This field has been driven by the explosion of electronic information combined with increasing computational capabilities15. The advantage of newer ML techniques over traditional statistical methods is that they can better model complex, multicollinear relationships between covariates and outcomes16, which is common in health care data17. Previously, ML was applied to the NSQIP database to develop an algorithm that predicts peri-operative outcomes for over 2,900 distinct procedures18. Given the heterogeneity of this cohort, better predictive performance may be achieved by building ML algorithms specific to patients undergoing AV access creation using the VQI database, a dedicated vascular registry containing procedure-specific variables12. We previously described ML algorithms trained on VQI data for predicting outcomes following aortic, carotid, and peripheral arterial interventions, which achieved better performance compared to traditional statistical techniques such logistic regression and existing tools19,20,21. The development of a ML-based prediction algorithm for AV access outcomes may complement these existing models and expand clinical guidance for the management of patients with end stage kidney disease (ESKD). In this study, we used VQI data to develop ML algorithms that predict 1-year outcomes following AV access creation using pre-operative data. Secondarily, we performed sensitivity analyses by training additional models using intra- and post-operative variables to determine their relative impact on model performance compared to using pre-operative data alone. We hypothesized that our ML models could achieve good predictive performance using only pre-operative data and achieve better performance than traditional statistical techniques such as logistic regression.

Results

Patients, events, and follow-up

From an initial cohort of 60,975 patients who underwent AV access creation for hemodialysis in the VQI database between August 1, 2011 and January 2, 2024, a total of 1,301 patients were excluded for the following reasons: no reported access type of fistula or graft (n = 123), inflow artery (n = 762), or outflow vein (n = 367), or the patient underwent endovascular AV fistula creation (n = 49). The final analysis included 59,674 patients who underwent AV access creation, including 45,571 (76.4%) fistulae and 14,101 (23.6%) grafts. Most AV accesses were created in the upper arm at or above the elbow (n = 29,256, 49.0%), followed by the forearm below the elbow (n = 27,852, 46.7%), lower extremity (n = 1,020, 1.7%), or other anatomic location (n = 1,546, 2.6%). Overall, 28,304 (47.4%) had 1-year successful clinical use of the AV access for hemodialysis. For secondary outcomes, AV access thrombosis occurred in 3,553 (6.0%) patients and AV access re-interventions occurred in 9,684 (16.2%) patients over 1 year of follow-up. For re-interventions, 7429 (12.4%) patients had surgical re-interventions, and 4389 (7.4%) patients had endovascular re-interventions. Mean follow-up was 15.7 (SD 1.9) months.

Pre-operative characteristics

Compared to patients without a primary outcome, those who had 1-year successful clinical use of the AV access for hemodialysis were younger (mean age 61.2 [SD 14.7] vs. 62.6 [SD 14.6] years, p < 0.001), less likely to be female (41.9% vs. 45.4%, p < 0.001), had a lower BMI, and were more likely to be Asian, Black, or Hispanic. They were also more likely to receive Medicaid and have their AV access created in an ambulatory center rather than a hospital and less likely to be transferred from another hospital or from a rehabilitation unit to the center of intervention. Patients who had 1-year successful clinical use of the AV access were less likely to be prior or current smokers or have diabetes, coronary artery disease (CAD), congestive heart failure (CHF), dysrhythmia, chronic obstructive pulmonary disease (COPD), and an American Society of Anesthesiologists (ASA) class ≥ 4. They were also more likely to have chronic kidney disease (CKD) stage 5 and be on hemodialysis rather than at the pre-dialysis stage. Functionally, patients with 1-year successful clinical use of the AV access were more likely to live at home and ambulate independently. For laboratory investigations, they had a higher mean creatinine and lower estimated glomerular filtration rate (eGFR). In terms of medications, they were less likely to be on acetylsalicylic acid, P2Y12 antagonists, and statins, and more likely to be on anticoagulants. For vascular access history, patients with 1-year successful clinical use of the AV access were more likely to have had prior AV fistulae or grafts in any anatomic location. They were also more likely to receive pre-operative imaging, including arterial duplex, arteriogram, ultrasound vein mapping, and venogram for vascular access planning. Patients with 1-year successful clinical use of the AV access had higher mean pre-operative diameters for their target inflow artery and outflow vein based on imaging. They were also more likely to receive an AV fistula rather than a graft and an upper arm or lower extremity access rather than a forearm access (Table 1).

Table 1 Pre-operative demographic and clinical characteristics of patients undergoing arteriovenous access creation with and without 1-year successful clinical use of the index access for hemodialysis

Intra-operative characteristics

Patients with 1-year successful clinical use of the AV access were more likely to receive regional anesthesia and have higher mean inflow artery and outflow vein diameters measured intraoperatively. For concomitant procedures, patients with the primary outcome were more likely to receive a patch angioplasty on the target artery and/or vein. They were also more likely to have a completion doppler or fistulogram study intra-operatively to confirm technical success (not a part of pre-operative imaging) (Table 2).

Table 2 Intra-operative characteristics of patients undergoing arteriovenous access creation with and without 1-year successful clinical use of the index access for hemodialysis

Post-operative characteristics

Immediate post-operative complications not requiring access ligation or abandonment, including ischemic steal, access thrombosis, or other complications requiring re-intervention, were rare (~2%) and occurred less frequently in patients with 1-year successful clinical use of the AV access. Mean hospital length of stay was lower in patients with the primary outcome. These patients were less likely to be discharged on acetylsalicylic acid, P2Y12 antagonists, statins, and anticoagulants. Patients with 1-year successful clinical use of the AV access were less likely to have a non-home discharge (Table 3).

Table 3 Post-operative in-hospital characteristics and complications of patients undergoing arteriovenous access creation with and without 1-year successful clinical use of the index access for hemodialysis

Model performance

Of the 6 ML models evaluated on test set data with only pre-operative variables as input features, Extreme Gradient Boosting (XGBoost) had the best performance in predicting 1-year successful clinical use of the index AV access for hemodialysis based on the primary model evaluation metric of area under the receiver operating characteristic curve (AUROC), achieving an AUROC of 0.90 [95% CI 0.89–0.91]. In comparison, the other models had the following AUROC’s [95% CI’s]: random forest (0.88 [0.87–0.89]), radial basis function support vector machine (RBF SVM) (0.85 [0.84–0.86]), Naïve Bayes (0.84 [0.83–0.85]), multilayer perceptron artificial neural network (MLP ANN) (0.77 [0.76–0.78]), and logistic regression (0.70 [0.69–0.71)]. The secondary performance metrics of XGBoost were the following: accuracy 0.82 (95% CI 0.81–0.83), sensitivity 0.82, specificity 0.81, positive predictive value (PPV) 0.81, and negative predictive value (NPV) 0.82. Model performance results are summarized in Table 4.

Table 4 Model performance on test set data for predicting 1-year successful clinical use of an arteriovenous access for hemodialysis using pre-operative features

For sensitivity analyses, we trained additional XGBoost models using intra- and post-operative data. The addition of intra-operative features to pre-operative features did not change model performance, with the AUROC remaining at 0.90 (95% CI 0.89–0.91), p = 0.90. Adding post-operative features marginally improved model performance without reaching statistical significance (AUROC 0.91, 95% CI 0.90–0.92, p = 0.41). The ROC curves are presented in Fig. 1, demonstrating excellent predictive performance of the XGBoost model for 1-year successful clinical use of an index AV access for hemodialysis at the pre-, intra-, and post-operative stages. For secondary outcomes, the XGBoost model predicted 1-year AV access thrombosis and re-interventions with AUROC’s of 0.86 (95% CI 0.85–0.87) and 0.88 (95% CI 0.87–0.89) at both the pre- and intra-operative stages, respectively (Table 5). The XGBoost model was not trained to predict the secondary outcomes using post-operative variables because some post-operative variables were directly related to thrombosis and/or re-intervention and would therefore artificially elevate model performance. Given that excellent model discrimination can sometimes be accompanied by poor calibration, we also assessed the calibration of our models. There was good agreement between predicted and observed event probabilities as demonstrated by the calibration plots in Fig. 2A–C, with Brier scores of 0.08 (pre-operative), 0.08 (intra-operative), and 0.07 (post-operative). This highlights the excellent calibration of the XGBoost model across the entire probability range for predicting 1-year successful clinical use of an index AV access for hemodialysis at the pre-, intra-, and post-operative stages. The top 10 predictors of 1-year successful clinical use of the index AV access for hemodialysis in the final XGBoost model included 9 pre-operative features (planned inflow artery and outflow vein, planned access type, pre-operative ultrasound vein mapping, age, dysrhythmia, number of prior AV fistulae, COPD, and CHF), and 1 post-operative feature (immediate post-operative complications). Four features predicted AV access success: higher diameter inflow artery, higher diameter inflow vein, access type of fistula rather than graft, and use of pre-operative ultrasound vein mapping. In contrast, six features predicted AV access failure: older age, presence of dysrhythmia, fewer number of prior AV fistulae, presence of immediate post-operative complications, presence of COPD, and presence of CHF (Fig. 3). These results highlight that anatomic features (inflow artery and outflow vein diameters), pre-operative planning (decision on fistula vs. graft and use of pre-operative imaging), and patient history (comorbidities and vascular access history) are critical in determining long-term AV access success.

Fig. 1: Receiver operating characteristic curve for predicting 1-year successful clinical use of an arteriovenous access for hemodialysis using Extreme Gradient Boosting (XGBoost) models at the pre-, intra-, and post-operative stages.
Fig. 1: Receiver operating characteristic curve for predicting 1-year successful clinical use of an arteriovenous access for hemodialysis using Extreme Gradient Boosting (XGBoost) models at the pre-, intra-, and post-operative stages.
Full size image

AUROC (area under the receiver operating characteristic curve), CI (confidence interval).

Fig. 2: Calibration plots with Brier scores for predicting 1-year successful clinical use of an arteriovenous access for hemodialysis using Extreme Gradient Boosting (XGBoost) models at the A) pre-operative, B) intra-operative, and C) post-operative stages.
Fig. 2: Calibration plots with Brier scores for predicting 1-year successful clinical use of an arteriovenous access for hemodialysis using Extreme Gradient Boosting (XGBoost) models at the A) pre-operative, B) intra-operative, and C) post-operative stages.
Full size image

Abbreviation: AV (arteriovenous).

Fig. 3: Variable importance scores (gain) for the top 10 predictors of 1-year successful clinical use of an arteriovenous access for hemodialysis in the Extreme Gradient Boosting (XGBoost) model at the post-operative stage.
Fig. 3: Variable importance scores (gain) for the top 10 predictors of 1-year successful clinical use of an arteriovenous access for hemodialysis in the Extreme Gradient Boosting (XGBoost) model at the post-operative stage.
Full size image

Abbreviations: AV (arteriovenous), COPD (chronic obstructive pulmonary disease), CHF (congestive heart failure).

Table 5 XGBoost performance on test set data for predicting 1-year primary and secondary outcomes following arteriovenous access creation at the pre-, intra-, and post-operative stages

Subgroup analysis

Model performance remained robust on subgroup analyses across demographic/clinical populations based on age, sex, race, ethnicity, rurality, median area deprivation index (ADI) percentile, access type (fistula or graft), and access location (forearm, upper arm, lower extremity, or other), with AUROC’s ranging from 0.89-0.91 and no significant differences between majority and minority groups (Supplementary Figs. 18).

Discussion

We used data from a large clinical registry (VQI) consisting of 59,674 patients who underwent surgical AV fistula or graft creation to develop ML models that can accurately predict 1-year successful clinical use of the index AV access for hemodialysis with AUROC’s ≥0.90 using pre-operative data. There were several key findings. First, patients with successful AV accesses have predictive features at the pre-, intra-, and post-operative stages. ML-based modeling allowed us to determine the relative impact of these factors on AV access success. Second, we evaluated 6 ML models on our dataset and XGBoost achieved the best performance, demonstrating excellent discrimination and calibration. Furthermore, predictive performance remained robust across demographic and clinical subpopulations. Third, while intra- and post-operative factors contributed to long-term risk, 9 of the top 10 predictors for 1-year successful clinical use of the AV access for hemodialysis were pre-operative features. Additionally, the addition of intra-operative features to the pre-operative model did not change predictive performance and the addition of post-operative features marginally improved model performance. This highlights an important opportunity for our risk prediction tool to guide pre-operative AV access planning and procedural selection. Overall, these models have the potential to support clinical decision-making both pre-operatively and throughout a patient’s perioperative course, thereby facilitating individualized risk assessment, counselling, monitoring, and management.

Heindel and colleagues10 used pooled patient-level data from 704 patients undergoing new radiocephalic AV fistula creation from the PATENCY-1 and PATENCY-2 randomized controlled trials to develop ML models that predict 1-year successful unassisted AV fistula use with AUROC’s ranging from 0.78–0.81 and accuracies ranging from 69.1–73.6%10. They demonstrated that the performance of their models was superior to the National Kidney Foundation Kidney Disease Outcomes Quality Initiative (KDOQI) and University of Alabama at Birmingham ultrasound threshold criteria10. There were several key differences between our studies. First, while Heindel and colleagues10 used randomized controlled trial data consisting of 704 patients, we used clinical registry data consisting of 59,674 patients10. While trial data have several advantages including prospective follow-up, adjudicated AV access outcomes, rigorous data collection methods, low missingness, high internal validity, and low misclassification of predictors and outcomes, registry data contains larger sample sizes critical for robust ML model training and evaluation and may better reflect real-world outcomes10. Second, Heindel and colleagues10 developed prediction models for radiocephalic AV fistulae, while we developed prediction models for all types of upper and lower extremity AV fistulae and grafts10. Models built for specific access configurations benefit from more granular data but may be less generalizable10. Although our approach may increase heterogeneity, it increases applicability of our model for AV access planning, as clinicians must consider different AV access approaches to determine the optimal strategy. Importantly, we demonstrated through subgroup analyses that our models performed well when evaluated specifically on AV fistulae vs. grafts and based on anatomic location (forearm vs. upper arm vs. lower extremity vs. other). Third, while we developed intra- and post-operative prediction models, our focus was on the ability for our models to make accurate predictions regarding long-term AV access success using pre-operative variables alone. In contrast, Heindel and colleagues10 used 4–6 week post-operative ultrasound parameters in addition to baseline characteristics as model input features10. While post-operative ultrasound parameters have important associations with long-term AV access success, a pre-operative prediction model may additionally support clinical decision-making and procedural planning for the index access creation intervention. In our study, we demonstrated that the use of pre-operative demographic and clinical variables alone from a large clinical registry allowed us to build highly accurate predictive ML models with an AUROC of 0.90 and accuracy of 82% for 1-year AV access success. Fourth, while Heindel and colleagues10 predicted the primary outcome of 1-year successful unassisted AV fistula use10, our models were trained to predict 1-year successful clinical use of the index AV access for hemodialysis, with or without re-interventions. While unassisted AV fistula use is an important outcome, it is well-established that AV accesses may require relatively low-risk re-interventions to support maturation and/or maintain patency22. Therefore, there is important clinical value of AV accesses that remain functional with re-interventions. Additionally, our models were trained to predict the secondary outcomes of re-intervention and thrombosis, which are of clinical importance to vascular surgeons, nephrologists, and other clinicians. Overall, we build on the work of Heindel and colleagues10 by using registry data with larger sample sizes, allowing us to develop pre-operative prediction models that can be used in conjunction with post-operative ultrasound measurements to optimize AV access success rates10.

Our findings regarding the important predictors of long-term AV access success align with the existing literature1. Fedorova and colleagues23 analyzed VQI data from 2011-2019 and found a significant association between pre-operative vein mapping and the incidence of the loss of secondary patency at 1 year of follow-up23. Similarly, we demonstrated that pre-operative vein mapping was one of the most important input features in our model for predicting 1-year successful clinical use of the AV access for hemodialysis. Like Heindel and colleagues10, who demonstrated that a larger outflow vein diameter, higher flow volume, and absence of >50% luminal stenosis were the most important covariates in their final model, we also showed that pre-operative anatomic characteristics including the inflow artery and outflow vein were the most important predictors in our best-performing XGBoost model10. Specifically, we demonstrated that higher diameters of the inflow artery and outflow vein were associated with 1-year successful clinical use of the AV access for hemodialysis. Furthermore, we showed that AV fistulae were associated with better 1-year outcomes than AV grafts in terms of clinical use for hemodialysis, potentially due to fewer infectious and/or thrombotic complications, which corroborates previous findings24. Overall, the important predictors in our ML model align with clinical intuition and existing literature related to factors associated with successful AV accesses1.

Notably, we showed that less than half of the AV accesses created in our cohort were successfully used for hemodialysis at 1-year of follow-up, which corroborates prior literature6. This highlights the need for better prediction of planned AV access outcomes to guide pre-operative planning and potentially improve success rates. We demonstrated that several factors were associated with long-term AV access success. We showed that anatomic factors including the inflow artery, outflow vein, access type, and pre-operative ultrasound vein mapping were the most important predictive features, which corroborates previous literature1. Our study also demonstrated that younger age and fewer comorbidities, such as dysrhythmias, CHF, and COPD were associated with better AV access outcomes, which aligns with existing knowledge1. Within the overall cohort, prescription of cardiovascular risk reduction medications including acetylsalicylic acid and statins was relatively low. Given the high number of vascular comorbidities in this population, there may be an opportunity to improve cardiovascular risk reduction in patients with ESKD25. Interestingly, we showed that patients who had prior AV access procedures at any anatomic location were more likely to have successful clinical use of the newly created access for hemodialysis at 1-year of follow-up. This may be because AV accesses are generally created distally first (e.g., radiocephalic) to preserve more proximal access options (e.g., brachiocephalic)1. Therefore, when a distal AV access fails and a more proximal AV access is created, the more proximal AV access may have a greater likelihood of success, given the likely larger diameters of the inflow artery/vein and higher flow volumes. This is corroborated by our data and previous work demonstrating that upper arm accesses were more likely to succeed than forearm accesses26. For intra-operative variables, we found that the diameters of the inflow artery and outflow vein measured intra-operatively were significantly associated with AV access outcomes. Although pre-operative imaging modalities are improving, the accuracy of vessel diameter measurements may be influenced by the temperature of the room, patient hydration status, recent dialysis, technician skills, tourniquet use, and regional anesthesia, among other factors27. However, given that pre-operative inflow artery and outflow vein diameters generally correlate well with intra-operative measurements, these additional variables likely did not contribute significantly to the model27. This may explain why intra-operative variables did not improve model performance compared to using only pre-operative variables. For post-operative variables, we found that immediate post-operative complications not requiring access ligation or abandonment, including bleeding, ischemic steal, and thrombosis, were important predictors of long-term AV access success, which corroborates prior literature28. Importantly, we demonstrated that pre-operative features can accurately predict 1-year successful clinical use of the index AV access for hemodialysis, while intra-operative variables did not change model performance and post-operative variables marginally improved model performance. This suggests that the ability to accurately predict long-term outcomes following AV access creation can be established pre-procedurally to support decision-making regarding AV access planning, procedural selection, counselling, peri-procedural management, and follow-up29.

Our ML models performed better than existing tools for several potential reasons. Compared to traditional logistic regression, advanced ML techniques can better model complex, non-linear relationships between inputs and outputs30. This is especially important in health care data, as patient outcomes can be influenced by many demographic, clinical, and system-level factors31. Our top-performing algorithm was XGBoost, which has unique advantages including relatively fewer issues with overfitting and faster computing while maintaining precision32. Furthermore, XGBoost works well with structured data, which may explain its superior performance compared to more complex algorithms such as neural networks on our dataset32.

Importantly, the performance of our models remained robust on subgroup analyses of specific demographic and clinical populations. This is a notable finding given that algorithm bias against underrepresented populations is a frequently encountered issue in ML models33. We were likely able to avoid such biases due to the excellent capture of sociodemographic data by VQI12. Important to this study, we demonstrated that our models performed well on both AV fistulae and grafts, as well as on different AV access locations, including the forearm, upper arm, lower extremity, or other anatomic location. Therefore, our models are versatile and have clinical utility across a wide range of options available for AV access creation, providing clinicians with a useful tool to support their complex decision-making.

Our ML models can be used to guide clinical decision-making in several ways for patients being considered for AV access creation. Pre-operatively, a patient predicted to have a low probability (<50%) of 1-year successful clinical use of the AV access for hemodialysis should be further assessed in terms of anatomic and physiologic risk factors1. Patients with anatomic risk factors may benefit from further imaging (e.g., ultrasound vein mapping, arterial duplex, arteriogram, or venogram) to support access planning (e.g., decisions on the inflow artery, outflow vein, and/or fistula vs. graft)1. Patients with physiologic risk factors, including CHF, COPD, dysrhythmias, or other comorbidities, may benefit from referrals to medical specialists to optimize their cardiovascular risk factors to potentially improve AV access outcomes1. Additionally, these patients should receive adequate counselling so that they understand the potential risk of failure and can make informed decisions regarding their care. Intra-operatively, patients predicted to be at high risk for AV access failure (potentially due to intra-operative findings of inadequate artery or vein diameters from vasospasm or other issues) may benefit from choosing a different inflow artery or outflow vein for access creation1. Additionally, these patients may benefit from a short admission to hospital for monitoring and treatment of any potential early AV access complications34. Post-operatively, patients predicted to have a low probability of long-term AV access success may benefit from close follow-up in clinic with functional imaging34. Additionally, for patients at heightened risk for AV access failure, clinicians may begin early planning for future AV access options with appropriate patient workup and counselling. Figure 4 illustrates a proposed clinical workflow for the use of our ML tool in supporting clinical decision-making at the pre-, intra-, and post-operative stages for patients being considered for an AV access creation procedure. Importantly, this workflow highlights the excellent performance of our ML tool in predicting long-term AV access success throughout a patient’s surgical course, which can guide clinical decision-making at each stage of their treatment. These peri-operative decisions guided by our tool have potential to improve patient-centered care and AV access outcomes by supporting individualized patient treatment, monitoring, and counselling. This is particularly important given that the KDOQI ESKD-Life Plan emphasizes the importance of a personalized approach to dialysis access, focused on the right access for the right patient at the right time with continuous monitoring35.

Fig. 4: Clinical workflow for the use of machine learning algorithms to guide clinical decision-making at the pre-, intra-, and post-operative stages for patients being considered for arteriovenous access creation.
Fig. 4: Clinical workflow for the use of machine learning algorithms to guide clinical decision-making at the pre-, intra-, and post-operative stages for patients being considered for arteriovenous access creation.
Full size image

Abbreviations: AUROC (area under the receiver operating characteristic curve), AV (arteriovenous). High risk defined as a model prediction negative for 1-year successful clinical use of an arteriovenous access for hemodialysis. Low risk defined as a model prediction positive for 1-year successful clinical use of an arteriovenous access for hemodialysis.

The programming code used to develop our ML models is publicly available on GitHub, allowing clinicians involved in the peri-operative management of patients being considered for AV access creation to use our tool. At a system-wide level, our models can be implemented by the over 1000 VQI participating centers12. The VQI database managers at these institutions routinely capture the input features used in our ML algorithms12. The number of VQI centers has grown considerably from 400 in 2019 to over 1000 in 202312. Recently, the VQI recorded over 1 million procedures12. Therefore, our models have broad and growing utility. They also have potential for use beyond VQI sites, as our predictors are commonly captured variables for the routine care of patients being considered for AV access creation36. Given the challenges of deploying prediction models into practice, thoughtful consideration of implementation science principles is critical. A key advantage of our ML models is their ability to provide automated risk predictions, thereby enhancing feasibility in busy clinical settings compared to traditional risk predictors that often require manual input of variables14. Specifically, our ML algorithms can autonomously extract a patient’s VQI data to generate risk predictions. To facilitate successful implementation of our ML tool, we recommend establishing and supporting data analytics teams at the institutional level. Such teams can provide important benefits to patient care, and their expertise can facilitate the deployment of our ML models37.

Our study has several limitations. First, our models were developed with VQI data, a voluntary registry primarily comprising data from North American centers12. Model performance may not translate entirely to other healthcare systems, such as single-payer systems outside North America or regions with differing patient demographics, surgical techniques, or post-operative monitoring standards. Future studies are needed to assess the generalizability of model performance beyond VQI sites. Additionally, the VQI-reported endpoint of successful clinical use of an AV access should be validated in future studies. Furthermore, the capture of high-flow related complications may further enhance the clinical utility of AV access outcome prediction models. Second, although we evaluated 6 different ML models, there are other ML models available. We chose these 6 models because of their established efficacy for predicting postoperative outcomes38. We achieved excellent performance; however ongoing evaluation of novel ML techniques would be prudent. Third, our cohort included various AV access types (fistulae and grafts) and anatomic locations (forearm, upper arm, lower extremity, or other). This may increase heterogeneity during model training. However, we demonstrated that our models performed well when evaluated on different AV access types and anatomic locations through subgroup analyses. This may improve potential clinical utility as clinicians must consider various access types and locations when planning for an optimal AV access1. However, future models trained on a single access type and/or location may provide more granular information. Fourth, most existing models incorporate post-operative ultrasound measurements as input features10,11. This information was not available in the VQI database. Although our models achieved excellent performance with pre-operative variables alone, future algorithms that can incorporate both our pre-operative variables and relevant post-operative ultrasound measurements may further enhance overall model performance to guide post-operative clinical decision-making.

In conclusion, we used a large, vascular-specific clinical registry (VQI) to develop robust ML models that can predict 1-year successful clinical use of an AV access with excellent performance using pre-operative data (AUROC’s ≥0.90). Our models can guide clinical decision-making regarding AV access planning, monitoring, and counselling, among other strategies, to facilitate personalized care and potentially improve the probability of AV access success. Importantly, we demonstrated that using pre-operative variables alone in our models can accurately predict long-term AV access outcomes, while intra- and post-operative variables marginally improved model performance. This demonstrates the potential utility of our models for guiding pre-operative decision-making. Notably, our models remained robust across demographic/clinical subpopulations and outperformed existing prediction tools and logistic regression, and therefore, have potential for important utility in the care of patients with kidney failure requiring hemodialysis. Prospective validation of our ML algorithms is planned to confirm clinical utility and impact.

Methods

Study approval

The SVS Patient Safety Organization (PSO) Research Advisory Council approved this project and provided the anonymized dataset. Patient consent was not required as the data originated from an anonymized registry.

Design

This was an ML-based prognostic study with findings reported based on the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis + Artificial Intelligence (TRIPOD + AI) statement39.

Dataset

The VQI database is a clinical registry maintained by the SVS PSO with the goal of improving vascular care (www.vqi.org)12. Vascular surgeons, interventionalists, and other specialists across more than 1000 academic and community hospitals in the United States, Canada, and Singapore prospectively contribute demographic, clinical, and outcomes data on consecutive eligible vascular patients including information from their index procedure up to approximately 1 year of follow-up12. Routine audits comparing submitted data to hospital claims are performed to ensure data accuracy40.

Patient cohort

All patients who underwent surgical AV fistula or graft creation for hemodialysis access from August 1, 2011 to January 2, 2024 in the VQI database were included. The VQI hemodialysis access registry captures AV fistula and graft creation procedures in the upper/lower extremities and other anatomic locations, and excludes dialysis catheter insertions and revisions of an existing access41. Patients with no reported access type, inflow artery, or outflow vein were excluded. Patients who underwent endovascular AV fistula creation were also excluded as they accounted for <0.1% of procedures in the hemodialysis access registry.

Features

Predictor variables (features) used in the ML models were divided into pre-, intra-, and post-operative stages. Given the advantage of ML in handling many input features, all available VQI variables were used to maximize predictive performance. Models were first trained and evaluated using pre-operative features alone, and secondarily trained using intra- and post-operative variables to determine the impact of these features on model performance in a sensitivity analysis. Pre-operative features (n = 111) included demographics, comorbidities (including CKD stage and dialysis status), functional status, laboratory investigations (including creatinine and eGFR calculated using the CKD – Epidemiology Collaboration [EPI] 2021 Equation42), medications, vascular access history, imaging and anatomic characteristics (including access type [fistula or graft], inflow artery, and outflow vein). To capture the impact of surgeon and center volumes on AV access outcomes, the deidentified physician and center identification numbers were included as pre-operative features.

For sensitivity analysis, intra-operative features (n = 22) included the type of anesthesia (local, regional, and/or general), intra-operative target artery and vein diameters, concomitant procedures, and completion imaging studies. Post-operative features (n = 17) included immediate post-operative complications (e.g., bleeding, ischemic steal, thrombosis, etc.) not requiring access ligation or abandonment and medical/surgical management of these complications, hospital length of stay, discharge medications, and non-home discharge. A complete list of features and their definitions can be found in Supplementary Tables 13.

Outcomes

The primary outcome was 1-year successful clinical use of the index AV access for hemodialysis, defined as an AV access that can provide the prescribed dialysis consistently with 2 needles for more than two-thirds of dialysis sessions within 4 consecutive weeks, with or without re-interventions, determined through direct clinical follow-up35. This outcome is based on follow-up information collected by the participating centers based on dialysis data from recorded treatment sessions and vascular access utilization information from electronic health records12. Although VQI primarily includes peri-operative and short-term follow-up data, 1-year dialysis outcomes data is well-captured within the hemodialysis access registry12. Therefore, linkage with dialysis databases or other longitudinal follow-up mechanisms beyond VQI was not required. These data are captured and verified by trained VQI database managers at each site, with routine audits comparing submitted data to hospital claims to ensure data accuracy40. The primary outcome is equivalent to functional cumulative patency as defined by the National Kidney Foundation KDOQI 2019 Clinical Practice Guidelines for Vascular Access35. This primary outcome was chosen because it is clinically relevant, as it requires the AV access to be both anatomically patent and functionally active35. A primary/secondary patency endpoint was not chosen because an AV access may be patent but not functional35. Similarly, a thrombotic primary endpoint was not chosen as AV accesses are known to suffer from thrombotic or other complications requiring re-interventions but may remain functionally active afterwards35. The analysis includes the full period from access creation to maturation, and therefore, accesses that fail to mature are considered to be unsuccessful, which is clinically relevant as they cannot be used for hemodialysis. Secondary outcomes were 1-year AV access thrombosis (defined as complete occlusion of the index AV access by a thrombus) and re-intervention (defined as endovascular or surgical re-intervention on the index AV access to promote maturation and/or to treat a complication)35. These diagnostic criteria are based on VQI registry data12. Patients who underwent multiple re-interventions were only counted once in this dataset, as re-intervention was considered a binary outcome in our study to simplify predictive modelling. The primary outcome was chosen because accurate prediction of 1-year successful clinical use of an AV access for hemodialysis can provide clinicians with a better understanding of patients who may or may not benefit from a specific AV access creation procedure. This information may guide clinical decision-making regarding the consideration of other AV access approaches, appropriate patient counselling, close post-operative monitoring, and/or other strategies to support AV access maturation and maintenance43.

Model development

We trained 6 different ML models to predict 1-year successful clinical use of an AV access for hemodialysis: XGBoost, random forest, Naïve Bayes classifier, RBF SVM, MLP ANN with a single hidden layer, sigmoid activation function, and cross-entropy loss function, and logistic regression. These models were chosen based on their established efficacy in predicting postoperative outcomes using structured data38,44,45. Specifically, these 6 ML models are widely used for surgical outcomes prediction, demonstrating excellent performance on a variety of datasets, procedures, and patient populations38,44,45. Logistic regression was selected as the baseline comparator to assess relative model performance because it is the most commonly applied statistical technique in traditional risk prediction tools46.

The data was randomly divided into training (70%) and testing (30%) sets. Testing data was reserved for model evaluation and not used for training to ensure fair model assessment. To determine the optimal model hyperparameters, 10-fold cross-validation and grid search were applied to training data47,48. Initial analysis demonstrated that the primary outcome occurred in 28,304/59,674 (47.4%) patients in our cohort. To achieve class balance, Random Over-Sample Examples (ROSE) was applied to training data49. ROSE uses a smoothed bootstrapping approach to generate new samples from the feature space surrounding the minority class, a commonly used method to support predictive modelling49. The models were then evaluated on test set data and ranked based on the primary discriminatory metric of AUROC. The models were compared at the pre-operative stage because predictions at this timepoint offer the most potential to guide clinical decision-making, such as whether to proceed with a specific AV access creation procedure50. Our best performing model was XGBoost, which had the following optimized hyperparameters: number of rounds = 200, maximum tree depth = 4, learning rate = 0.05, gamma = 0, column sample by tree = 0.8, minimum child weight = 1, and subsample = 1. Supplementary Table 4 outlines the process for selecting these hyperparameters, as well as the model architecture, feature engineering, and optimized hyperparameters for the 6 ML models.

After identifying the best performing ML model at the pre-operative stage, we performed sensitivity analyses by training additional algorithms using intra- and post-operative data. This approach involved considering different features at each phase of the perioperative course. At the pre-operative stage, only pre-operative characteristics were considered. At the intra-operative stage, both pre- and intra-operative features were used. At the post-operative stage, all pre-, intra-, and post-operative features were inputted into the model. Post-operative variables were included because the immediate post-operative in-hospital course may have an important impact on the long-term success of an AV access51. To ensure that post-operative variables did not artificially elevate model performance for predicting 1-year successful clinical use of the AV access for hemodialysis, the only immediate post-operative complications used as input features were those that did not lead to access ligation or abandonment. This multi-phase approach allows clinicians to gain insights into the probability of long-term AV access success/failure at different stages of a patient’s perioperative course, thereby guiding decision-making before, during, and after intervention. This model training method has been previously described, particularly for the development of prediction tools for long-term outcomes52,53. Importantly, the focus of this study was on the pre-operative model, and the creation of the additional intra- and post-operative algorithms was primarily to assess the relative impact of intra- and post-operative variables on model performance compared to pre-operative variables alone.

Statistical analysis

Pre-, intra-, and post-operative features were summarized as means (standard deviation) or medians (interquartile range) for continuous variables and numbers (%) for categorical variables. Differences between patients with and without 1-year successful clinical use of the AV access for hemodialysis were assessed using independent t-tests (continuous variables) or chi-square tests (categorical variables). To account for multiple comparisons, Bonferroni correction was used to set statistical significance. The primary model evaluation metric was AUROC (95% CI), a validated measurement of discriminatory ability that considers both sensitivity and specificity54. Differences in AUROC’s between the pre-operative model and the intra- and post-operative models were evaluated using the DeLong test55. Secondary performance metrics were accuracy, sensitivity, specificity, PPV, and NPV. To assess model robustness, we plotted calibration curves and calculated Brier scores, a measurement of the agreement between predicted and observed event probabilities56. In the final model, feature importance was determined by ranking the top 10 predictors based on variable importance scores (gain), a measurement of the relative importance of individual covariates in contributing to an overall prediction57. To assess model bias, we evaluated predictive performance across demographic/clinical subgroups based on age, sex, race, ethnicity, rurality, median ADI percentile, access type (fistula or graft), and access location (forearm, upper arm, lower extremity, or other).

Based on a validated sample size calculator for clinical prediction models, to achieve a minimum AUROC of 0.7 with an outcome rate of approximately 47% and 111 pre-operative features, a minimum sample size of 7730 patients with 3634 events is required58. Our cohort of 59,674 patients with 28,304 primary events satisfied this sample size requirement. For variables of interest, missing data was less than 5%; hence, we adopted a complete-case analysis approach, considering only non-missing covariates for each patient. This is a valid analytical method for datasets with minimal missing data (<5%) and reflects predictive modelling of real-world data, which inherently includes missing information59,60. Furthermore, data was missing completely at random, with no association between center volume and missing data. Patients who underwent planned two-stage procedures were included once within the cohort, with data from both the primary and secondary procedures accounted for within a single entry. Patients who died, received a kidney transplant, changed dialysis modalities, were pre-dialysis and did not progress to ESKD (defined as CKD stage 5), or were lost to follow-up were censored. In total, there were 1964/59,674 (3.3%) censored cases. Given the relatively low proportion of censored cases, they were considered non-events, with a sensitivity analysis excluding censored observations demonstrating no significant impact on model training or performance. This approach is supported by previous work61,62,63. All analyses were conducted using R version 4.3.164.