Table 1 Hyperparameter search ranges and optimized values for each machine learning model.
Model name | Hyperparameter search range | Best hyperparameters |
|---|---|---|
AdaBoost-DT | estimator__max_depth: [3, 5, 7] n_estimators: [50, 100, 200] learning_rate: [0.01, 0.1, 1] | estimator__max_depth: 7 n_estimators: 200 learning_rate: 1 |
Decision Tree (DT) | max_depth: [3, 5, 7, 10] min_samples_split: [2, 5, 10] min_samples_leaf: [1, 2, 4] max_features: [None, ‘sqrt’, ‘log2’] | max_depth: 10 min_samples_split: 2 min_samples_leaf: 1 max_features: None |
Gradient Boosting (GBoost) | n_estimators: [50, 100, 200, 300, 400] max_depth: [2, 4, 8, 10, 12, 16] subsample: [0.25, 0.5, 0.75, 1] learning_rate: [0.01, 0.03, 0.05, 0.07, 0.1] | n_estimators: 300 max_depth: 8 subsample: 0.25 learning_rate: 0.03 |
LightGBM | n_estimators: [300, 500, 800] max_depth: [3, 5, 7] subsample: [0.7, 0.75] learning_rate: [0.05, 0.01] colsample_bytree: [0.3, 0.4] subsample_freq: [1, 2] num_leaves: [5, 8, 10] | n_estimators: 800 max_depth: 7 subsample: 0.75 learning_rate: 0.05 colsample_bytree: 0.4 subsample_freq: 2 num_leaves: 10 |
Deep Neural Network (DNN) | module__hidden_layers: [1–3] module__neurons: [16, 32, 64] module__activation: [‘relu’, ‘tanh’] optimizer__lr: [0.001, 0.01, 0.1] batch_size: [10, 20, 40] max_epochs: [50, 100] | module__hidden_layers: 1 optimizer__lr: 0.001 module__activation: relu module__neurons: 64 batch_size: 10 max_epochs: 100 |
TabNet | n_d: [8, 16] n_a: [8, 16] n_steps: [3, 5] gamma: [1.0, 1.3, 1.5, 2.0] lambda_sparse: [1e-3, 1e-2] optimizer_params: [‘lr’: 1e-4, ‘lr’: 1e-3, ‘lr’: 1e-2, ‘lr’: 2e-2] | n_d: 16 n_a: 16 n_steps: 3 gamma: 1.0 lambda_sparse: 0.001 optimizer_params: ‘lr’: 0.02 |