Introduction

Osteoporosis is a major concern for orthopaedic clinicians, primarily affecting aged people worldwide1. People over the age of 50 experience a major bone mass reduction due to osteoporosis. Osteoporosis or bone loss often occurs due to low bone mineral density and structural depreciation of the bone2. Nowadays it has become an extreme concern within the realm of clinical orthopaedics as it alters the bone mineral density (BMD) and bone mineral apposition rate (BMAR). There are various biomechanical and pharmaceutical interventions available in the literature that can be useful in treating such diseases. For example, dietary and lifestyle changes or medication such as Alendronate, Ibandronate, Risendronate and Bisphosphonates are useful in inhibiting bone loss3,4. However, these drugs negatively influence the bone remodelling process in long-term use, and they also can cause gastrointestinal discomfort such as heartburn and nausea5. However, physiological loading activities such as aerobic movements, athletic events, and polyphyletic exercise have been found useful in the treatment of osteoporosis as they prevent and potentially reverse the osteoporotic bone loss6. In vivo studies have shown that the osteogenesis (i.e., bone adaptation) rate depends upon the physiological loading and their parameters such as loading cycles7, frequency8, and rate of strain/stress9.

Animal loading experiments such as three-point, four-point and cantilever bending, axial loading and knee loading model have demonstrate that the mechanical loading and their associated parameters affect the bone mineral apposition rate (BMAR)10. For example, Recently Kumar and Pathak11 established bone mineral apposition rate as a function of mechanical loading parameters using the adaptive neuro-fuzzy inference system (ANFIS). They reported that the ANFIS model shows more promising results than other artificial neural network (ANN) models. Further, Dattatrey et al.,12 noticed that the strain magnitude, loading cycles and frequency play an important (a vital) role in predicting the optimal conditions of physiological loading which would promote bone mineral apposition rate at the cortical boundaries. In addition, Tiwari and Prasad13 used ANN-based models to predict bone modelling parameters as a function of mechanical loading cycle, frequency and strain rate. However, the dataset used by the previous study was very much limited due to unavailability of the data. Therefore, it was essential to enhance the dataset using the data augmentation by adding gaussian noise (noise level of 10%). Further, we have employed ML regressors as they predict the optimum output values within a reasonable range based on the input feed data. The present study has adopted four ML regressors namely Random Forest, Support Vector Machine, K-Nearest Neighbours, and XGBoost. These algorithms are implemented due to their ability to quickly learn from the input data and optimize their operations to enhance performance and gradually develop intelligence. For example, Sohail et al.,14 adopted a machine learning regressor to predict how the trabecular bone gradually diminishes into a porous and brittle structure during osteoporosis. Wu et al.,14 employed machine learning-based modelling approach to stimulate osteogenesis in bone scaffolds. They noticed that the machine learning-based modelling shows adequate accuracy and efficiency over the finite element model. Tanphiriyakun et al.,15 employed several machine learning algorithms to analyses the available osteoporotic treatments such as medication, demographic data, diagnoses to improve bone mineral density. Bui et al.,16 attempted to develop an ML model for the individual assessment risk of osteoporosis in the case of Vietnamese women where height, weight, and age were the most significant factors. Tu et al.,17 applied several ML-based predictive models to assess the individuals at substantial risk of osteoporosis based on the data namely gender, age, disorders, cancer, hypertension, diabetes, and stroke. They reported that age and gender in a person emerged as the most significant factors for the evaluation of osteoporosis in high-risk individuals. For example, post-menopausal women and senescent people experience a drastic decrease in bone mineral density which exacerbates the risk of osteoporosis. Recently, Mouloodi et al.,18 have substantiated that several ambiguities produced during the use of traditional computational techniques in bone mechanics research can be overcome by utilising machine learning models such as ANN, ML and Artificial intelligence (AI). Furthermore, it is suggested that the machine learning algorithm can be employed along with the constitutive model in computation and numerical-based simulation package (FEA and CFD) to significantly reduce the required computational effort.

Although the use of machine learning algorithms in the modelling of osteoporosis-related conditions has been well reported, however, the existing studies have primarily focused on Bone Mineral Density (BMD). BMD refers the amount of mineral available during a given time-period that represents the static strength of the bone19. Whereas the bone mineral apposition rate (BMAR) signifies the dynamic rate at which the new bone is depositing on the existing bone surfaces20,21. Therefore, the present study considers the Bone Mineral Apposition Rate (BMAR) as the targeted output, which serves as a more direct and dynamic indicator of bone formation and adaptation at the cortical surfaces, namely the periosteal and endosteal regions. Moreover, the differential analysis of BMAR at these two surfaces using machine learning regressors remains largely unexplored in the existing literature. The present study incorporates the Gaussian noise-based data augmentation to simulate real-world variability and improve model robustness of the existing soft computing model. Furthermore, the adoption of XGBoost a state-of-the-art ensemble learning algorithm known for its superior performance on structured data added a novel dimension to the modelling framework proposed.

The facts mentioned above from the literature encouraged us to develop a Machine learning regressor model to establish an optimal corelation between the loading parameters (such as strain, frequency and loading cycles) and bone mineral apposition rate which will boost osteogenesis in the bone and prove to be a potential solution to osteoporosis prevention. This study has twofold significance in filling gaps in existing knowledge of bone loss as well as informing the development of personalized orthopaedic treatments by harnessing the power of machine learning.

Machine learning Implementation/Modelling

This section addresses the machine learning regressor used to investigate the significance of various parameters like strain magnitude, number of cycles and frequency on bone mineral apposition rate (BMAR) at cortical boundaries. In the present study, we have recruit supervised machine learning regressors, in which a machine learning model is trained to predict bone mineral apposition rate based on the input parameters such as frequency, number of cycles and strain. The machine learning modelling is divided into the following subsections:

Machine learning algorithms used in the present study

K-nearest neighbour (KNN) algorithm

The K-Nearest Neighbour (KNN) algorithm is a simple, supervised machine learning algorithm that solve both classification and regression problems. Usually, KNN classifies new instances efficiently and accurately into the appropriate category as it maintains the existing data and utilises it to assign a new data point for classification based on similarity. Due to this specific property, it is also referred to as “lazy learning algorithm”22. In the present study, the number of neighbours (k) was varied as3,5,7 using GridSearchCV to identify the optimum value that minimizes prediction error. In the present case, KNN exhibits lower predictive accuracy, particularly on the endosteal surface, indicating limited suitability for datasets with complex non-linear relationships such as those present in biological systems.

Random forest (RF) algorithm

The Random Forest algorithm is an amalgamation of various classifiers that employs ensemble learning to manage complicated problems and substantially enhances the model’s performance and accuracy. It is a well-known machine learning approach that employs numerous decision trees on various subsets of the supplied data and averages the results to improve the forecasting accuracy in case of postmenopausal osteoporosis22. The number of estimators and maximum depth are varied as [100, 200] and3,5, respectively, to assess their effect on model performance. Random Forest can capture non-linear interactions between input features and output responses, making it suitable for regression problems involving physiological parameters. Another advantage of the model is its ability to rank the relative importance of input features, which is beneficial in interpreting the effect of strain, frequency, and number of cycles on BMAR.

XGBoost machine learning algorithm

Extreme Gradient Booster is a machine learning algorithm which uses the decision tree technique to train the ML algorithm and anticipate the residuals or errors of earlier models before combining these forecasts to produce the final prediction. The method is called “gradient boosting” because it uses a gradient descent methodology to minimise losses while introducing new models23. The number of estimators, learning rate, and maximum depth were selected as hyperparameters and varied as [100, 200], [0.01, 0.03], and3,5, respectively. XGBoost is particularly effective for structured datasets with non-linear relationships and has been widely used in biomedical applications.

Support vector machine (SVM) algorithm

The Support Vector Machine (SVM) Algorithm is a highly effective supervised machine learning approach. This algorithm can be used for different applications such as regression, linear classification, and output prediction. SVM is found beneficial especially where the number of dimensions exceeds the number of samples24, which is often the case in biomedical datasets. In the present study, the C and ε values were varied as1,10,47 and [0.01, 0.1], respectively, to determine the best trade-off between model complexity and error tolerance.

Data acquisitions and augmentation

Data acquirement is a vital step in any machine learning algorithm, as the data quality will be crucial towards the performance of the machine learning models. This study is designed and trained to take the input from various in vivo animal loading experiments25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54 available in the literature to predict their significance in bone mineral apposition rate at cortical boundaries. Most of these experiments were performed on mice/rats with physiological loading was applied on the tibial and femur bone. Based on the different types of loading such as cantilever bending32, axial/ulnar loading55, Knee loading56 and three57 or four points of bending28 and their parameters namely strain, frequency, and number of cycles were selected in various combinations to elicit different responses on bone adaptation. We have presented the animal loading experiment data using univariate and bivariate distributions as illustrated in Fig. 1a, b to exhibit relationship and coherence among the input and output variables. The diagonal histograms illustrate the different ranges and distributions of each input and output parameter. Whereas the off-diagonal scatter plots represent the pairwise distributions of parameters against the other. We observe from the Fig. 1a, b that the BMAR increases with the strain and frequency in certain ranges, that aligns with previous findings58,59.

Further, to enhance the datasets, data augmentation is applied by adding gaussian noise (noise level of 10%) to loading parameters such as strain and the number of cycles as gaussian noise stimulates the real-world measurement variability. This ensures a better training to the unseen data. It is applied specifically to strain and number of cycles to mimic the natural fluctuations in these variables. List of input and output parameters used in the present study are mentioned in Table 1.

Table 1 List of input parameters used in machine learning Regressor.
Full size table
Fig. 1
Fig. 1
Full size image

Multiple bivariate pair plot between the input and output variables at (a) Periosteal and (b) Endosteal surface.

The Fig. 2 represents the plot between the original data and the augmented data observed at periosteal and endosteal surface. The augmented data is labelled in a separate column to differentiate from the original data as shown in Fig. 3. Moreover, we have also visualized the strength of the relationship between the input and output parameters using a correlation heatmap as shown in Fig. 3a, b. A positive and negative correlation value signifies a linear and non-linear correlation among the input variables and output response, respectively. Nevertheless, a correlation value of zero indicates no correlation between the variables. It is also evident from Fig. 3a, b, that MAR shows highest positive correlation value with respect to strain and frequency as compared to number of cycles at periosteal and endosteal surface, respectively. The figure also suggested that how the independently mechanical loading parameters affects the mineral apposition rate at cortical boundaries. Several studies like Manske et al.,59 and Lamothe et al.,48 have substantiated that bone adaptation is usually influenced by greater strain magnitude and loading frequencies. Along the same line Chen et al.,58 reported that the BMAR increases with an increase in strain magnitude and frequency. The result of the present paper is aligned with the previous findings.

Fig. 2
Fig. 2
Full size image

Plot between augmented data versus actual data at (a) Periosteal surface, (b) Endosteal surface.

Fig. 3
Fig. 3
Full size image

Correlation Heatmap between the input variables and output response at (a) Periosteal and (b) Endosteal surface.

The dataset originally consists of 50 endosteal surface data points and 38 periosteal surface data points. Combing the original and augmented data (through Gaussian noise addition at 10%) the total dataset comprised of 300 data points for the endosteal surface and 228 for the periosteal surface. The upper and lower bounds for the endosteal and periosteal surfaces are mentioned in the Table 2.

Table 2 Range of input parameters considered in the machine learning Regressor.
Full size table

Additionally, the dataset was split into 80:20 ration for training and testing, respectively.

Hyperparameter tuning

To find the optimal machine learning parameters hyperparameter tuning plays a crucial step as these parameters directly affects the training of dataset. Hyperparameter tuning can be done in various ways such as random search and Bayesian optimization. GridSearchCV technique has some advantages and drawbacks over the other techniques such as efficiency and ability to reproduce result as and when required. Present study employs a 5-fold cross-validation as it ensures computational efficiency and reliability of performance due to the moderate size of our datasets (300 and 228 data points respectively). Whereas 10-fold cross-validation would result in smaller training splits and does not assure a proportionate gain in reliability. These ranges of hyperparameters used in the present study have been chosen based on the several biomedical machines learning available in the literature15,22,23. These studies combined with the empirical tuning strategies prioritize robustness of the model and its generalizability. The details of the ML regressors model along with their hyper-parameters used in the present study are listed below (Table 3).

Table 3 Details of Hyper-parameters used in the present study.
Full size table

Results and discussion

Evaluation matrix (predicting accuracy of machine learning regressor)

The performance of a machine learning regressor is a measure of how well the model can precisely predict the output response. Fundamentally, it tunes the hyperparameter in such a way that the supervision of input and output variables increases over time, sequentially boosting the overall performance of the ML regressor. The accuracy of the ML regressor is expressed in percentage and it can be computed as

$$\:Accuracy\:=\:\frac{True\:Positive\:+\:True\:Negative}{True\:Positive\:+\:False\:Positive\:+\:True\:Negative\:+False\:Negative}$$
Table 4 Accuracy (R-square) of K-Nearest neighbour (KNN), random Forest, support vector machine (SVM) and XGBoost classification model.
Full size table

It is evident from Table 4 that the XGBoost model outperformed at both the surfaces i.e., periosteal and endosteal as compared to other ML regressors. One of the possible reasons for this could be the ability to capture the nonlinearity and complex interactions between the datasets. This helps in generalization of the dataset which prevents the overfitting in case of XGBoost regressor. The XGBoost model achieved an accuracy of 98% and 94% at endosteal and periosteal surfaces, respectively (Fig. 4). XGBoost model exhibited a 33% improvement compared to Support Vector Machine regressor at endosteal surface (Fig. 4). In addition, the Support vector Regressor (SVM) demonstrated the worst performance for endosteal surface demonstrates the sensitivity towards the parameter tunning and limitation in scalability. Whereas KNN exhibits worst performance at periosteal surface due to its dependence on the local patterns and sensitivity towards noise in the dataset. This exhibits that the endosteal surface was more mechanoresponsive to bone mineral apposition rate as compared to the periosteal surface. Our findings align with Birkhold et al.,60 where they have observed that the endosteal surface was more responsive to bone formation and resorption as compared to the periosteal surface. The prediction of MAR at the periosteal and endosteal surfaces using machine learning is important for understanding site-specific bone remodelling. These predictions help capture complex relationships influencing bone formation, enabling better assessment of bone health and aiding in the development of targeted treatments and future research on bone-related diseases.

Fig. 4
Fig. 4
Full size image

Comparison of R2 for ML regressors after testing at cortical surfaces.

Fig. 5
Fig. 5
Full size image

Comparison of MSE for ML regressors after testing at cortical surfaces.

Figure 5 demonstrates the comparison of mean squared error (MSE) computed for ML regressors at cortical boundaries after the testing phase. From the Fig. 5 it can also be observed that the XGB ML regressor exhibit minimum MSE error of 0.004 and 0.007 at endosteal and periosteal surface, respectively. This shows that the XGB outperformed as compared to other ML regressor in predicting the MSE. Consequently, the percentage deviation for MSE is computed using the formula61

$$\:Percentage\:deviation\:=\:\frac{|Predicted\:value\:-\:Actual\:value|}{\left|Actual\:value\right|}\:x\:100$$

The percentage deviation is used to compare the percentage errors of various ML models by providing scale-independent metrices. The percentage deviation plot for MSE is depicted in Fig. 6 which aligns with the findings of this paper.

Fig. 6
Fig. 6
Full size image

Percentage deviation of machine learning regressor models.

Our results in Figs. 4 and 5 support that XGBoost regressor has a superior accuracy compared to the other ML regressors. This can be verified by Fig. 5 since MSE exhibits a minimum value for XGB regressor meanwhile the R2 is highest for the same which indicates its precise results. Nevertheless, the proposed ML regressors in this study and the ANN model proposed by Dattatrey et al.,12 are found close during comparison. In addition, a study performed by Tanphiriyakun et al.,15 shows that random Forest outperformed other ML models with an accuracy of 0.69, closely followed by XGBoost while prediction of most suitable osteoporotic treatments available to improve the bone mineral density.

Feature importance graph

The feature importance graph represents the relative influence of each input parameter namely magnitude of strain, frequency, and number of cycles on the model’s output prediction i.e., BMAR. Since Random Forest and XGBoost regressors are tree-based learning models that split the data based on the feature values of the input variables. However, KNN regressor is an instance-based model, therefore there are no feature coefficients present which means we cannot generate feature importance graph. Moreover, the Support Vector regressor creates a hyperplane, that separates the feature space and therefore does not allow the computation of feature importance scores62. Therefore, the present study computes the feature importance parameters only for XGBoost and Random Forest classifier as shown in Figs. 7 and 8. Figures 7 and 8 show distinct variations in the feature importance of variables presented by both models. The most significant predictors for XGBoost algorithm are the frequency at periosteal and endosteal surface followed by strain and then number of cycles. A similar pattern is also observed for the endosteal surface using Random Forest (Figs. 7 and 8). Whereas, for the periosteal surface the pattern shifted to the strain being the most significant factor followed by frequency and then the number of cycles in the case of Random Forest classifier. These graphs are plotted on a scale of 0 to 1, where 0 indicates the feature with no importance and as the value of features increases the importance of the parameters also increases significantly.

Fig. 7
Fig. 7
Full size image

Visualization plot for feature Importance at endosteal surface.

At the endosteal surface, XGBoost and Random Forest regressor showed that the number of cycles does not significantly predict bone mineral apposition rate (Fig. 7). Nevertheless, at cortical boundaries frequency plays an important role in predicting BMAR as illustrated in Figs. 7 and 8 which also aligns with the results of preceding experimental studies. For example, Rubin et al.,63 have shown that high-frequency and low-magnitude mechanical loading is most effective in case of bone formation. Therefore, the optimal loading frequency and regime must be considered during the design of loading protocols for bone formation.

Fig. 8
Fig. 8
Full size image

Visualization plot for feature Importance using at periosteal surface.

Q-Q plot (quantile-quantile plot)

The Q-Q plot as depicted in Fig. 9 is a graphical tool used in statistics to compare the distribution of predicted BMAR values with the actual distribution of BMAR value. This also allows us to analyse the data as the plots can indicate underfitting, overfitting or non-Gaussian error trends which are common in small datasets64,65. The Q-Q plot of the Periosteal and Endosteal surfaces are illustrated as shown in Fig. 9.

Fig. 9
Fig. 9
Full size image

Q-Q plot for (a) Periosteal and (b) Endosteal surface.

In Fig. 9, it is observed that the points for the endosteal surface create approximately a line parallel to the diagonal. However, at periosteal surface points deviate significantly from the diagonal, indicating that the data may not follow the theoretical distribution. This signifies that the prediction of bone mineral apposition rate at the endosteal surface based on the input data is more accurate than that of the periosteal surface.

Limitation of the present study

The present study was designed and developed based on the data collected from several experimental study on mice, However, physiological differences between mice and humans, especially in bone remodelling dynamics and responses to mechanical loading, might limit the direct translation of findings to clinical trials. In addition, the Machine learning regressor model has not been validated over extended periods of disuse or varied recovery phases due to the limitation of data availability, which are critical for understanding the long-term effects of interventions on bone health. The model appeared to focus primarily on mechanical loading and their parameters and may not account for systemic biological factors such as hormonal changes, nutritional status, or inflammatory responses that influence bone remodelling. Therefore, the present study may be extended to capture the exact parameters responsible for bone formation. Along the same line the Machine learning regressor model was used to obtain the Feature importance scores of only tree models, whereas methods to obtain feature impact for non-tree-based models like Support Vector Machines (SVM) and k-Nearest Neighbor (kNN) could also be explored in follow-up work.

Conclusion

This study determined a robust correlation between the loading parameters such as strain, frequency and number of cycles, and mineral apposition rate at cortical surfaces namely periosteal and endosteal surface.

Based on the results, the following conclusions are drawn which are as follows:

  • The correlation observed at the endosteal surface was much stronger than at the periosteal surface as observed in the Q-Q plot.

  • Out of four ML regressors used in the present study, XGBoost regressor outperformed with an accuracy of 98% and 94% at the endosteal and periosteal surface, respectively.

  • The accuracy of the machine learning regressor prediction is remarkable compared to other artificial neural network models.

The findings of the present study are beneficial to the health-care community assisting them in designing optimal loading protocols for the improvement of BMAR to reduce the risk of osteoporosis at cortical boundaries. This also governs a new way of improving therapeutic effectiveness in osteoporosis.