Table 1 Search scope and description for hyperparameter optimization.
Model | Hyperparameter | Description | Range |
|---|---|---|---|
Lasso | ‘alpha’ | Regularization strength: controls model complexity and prevents overfitting. | 0.001–10 (loguniform) |
RF | ‘n_estimators’ | Number of trees in the forest; more trees can improve performance but increase computation. | 50–300 (quniform, step 10) |
‘max_depth’ | Maximum depth of each tree; controls tree size and helps to avoid overfitting. | 5–20 (quniform, step 1) | |
‘min_samples_split’ | Minimum number of samples required to split an internal node. | 0.1–1.0 (uniform) | |
XGBoost | ‘n_estimators’ | Number of boosting rounds; influences model accuracy and computation time. | 50–300 (quniform, step 10) |
max_depth | Maximum depth of a tree; deeper trees capture more complexity but risk overfitting. | 3–10 (quniform, step 1) | |
‘learning_rate’ | Step size shrinkage used to prevent overfitting; smaller values provide better generalization. | 0.01–0.3 (uniform) | |
NN | ‘hidden_layer_sizes’ | Number of neurons in each hidden layer; affects model capacity and complexity. | 50 or 100 (choice) |
‘activation’ | Activation function applied to neurons; affects non-linearity in model learning. | logistic, tanh, relu (choice) | |
‘alpha’ | L2 regularization term; prevents overfitting by penalizing large weights. | 0.0001–0.1 (loguniform) | |
‘max_iter’ | Maximum number of iterations; controls model convergence. | 300–1000 (quniform, step 100) | |
SVM | ‘C’ | Regularization parameter; controls the trade-off between smooth decision boundaries and correctly classifying training points. | 0.001–100 (loguniform) |
‘epsilon’ | Epsilon-insensitive loss function; specifies the margin within which no penalty is given. | 0–1 |
