Table 5 XGBoost model hyperparameters and performance before and after feature selection.
Category | Parameter/Metric | Before feature selection | After feature selection |
|---|---|---|---|
Hyperparameter tuning | Optimal CV score | ||
(neg_MAE) | -627.79 | -555.41 | |
Optimal hyperparameters | Subsample | 1 | 0.9 |
reg_lambda | 0.5 | 0.5 | |
reg_alpha | 0.5 | 1 | |
n_estimators | 200 | 500 | |
max_depth | 7 | 5 | |
Learning_rate | 0.1 | 0.05 | |
Gamma | 0.1 | 0.2 | |
colsample_bytree | 0.8 | 0.9 | |
Average train performance | MAE | 104.22 | 68.66 |
RMSE | 122.10 | 90.90 | |
R2 score | 0.96 | 0.98 | |
Average test performance | MAE | 643.80 | 587.65 |
RMSE | 810.90 | 742.98 | |
R2 score | 0.32 | 0.42 |
