Table 1 Hyperparameter search ranges and optimized values for each machine learning model.

From: Machine and deep learning models for predicting high pressure density of heterocyclic thiophenic compounds based on critical properties

Model name

Hyperparameter search range

Best hyperparameters

AdaBoost-DT

estimator__max_depth: [3, 5, 7]

n_estimators: [50, 100, 200]

learning_rate: [0.01, 0.1, 1]

estimator__max_depth: 7

n_estimators: 200

learning_rate: 1

Decision Tree (DT)

max_depth: [3, 5, 7, 10]

min_samples_split: [2, 5, 10]

min_samples_leaf: [1, 2, 4]

max_features: [None, ‘sqrt’, ‘log2’]

max_depth: 10

min_samples_split: 2

min_samples_leaf: 1

max_features: None

Gradient Boosting (GBoost)

n_estimators: [50, 100, 200, 300, 400]

max_depth: [2, 4, 8, 10, 12, 16]

subsample: [0.25, 0.5, 0.75, 1]

learning_rate: [0.01, 0.03, 0.05, 0.07, 0.1]

n_estimators: 300

max_depth: 8

subsample: 0.25

learning_rate: 0.03

LightGBM

n_estimators: [300, 500, 800]

max_depth: [3, 5, 7]

subsample: [0.7, 0.75]

learning_rate: [0.05, 0.01]

colsample_bytree: [0.3, 0.4]

subsample_freq: [1, 2]

num_leaves: [5, 8, 10]

n_estimators: 800

max_depth: 7

subsample: 0.75

learning_rate: 0.05

colsample_bytree: 0.4

subsample_freq: 2

num_leaves: 10

Deep Neural Network (DNN)

module__hidden_layers: [1–3]

module__neurons: [16, 32, 64]

module__activation: [‘relu’, ‘tanh’]

optimizer__lr: [0.001, 0.01, 0.1]

batch_size: [10, 20, 40]

max_epochs: [50, 100]

module__hidden_layers: 1

optimizer__lr: 0.001

module__activation: relu

module__neurons: 64

batch_size: 10

max_epochs: 100

TabNet

n_d: [8, 16]

n_a: [8, 16]

n_steps: [3, 5]

gamma: [1.0, 1.3, 1.5, 2.0]

lambda_sparse: [1e-3, 1e-2]

optimizer_params: [‘lr’: 1e-4, ‘lr’: 1e-3, ‘lr’: 1e-2, ‘lr’: 2e-2]

n_d: 16

n_a: 16

n_steps: 3

gamma: 1.0

lambda_sparse: 0.001

optimizer_params: ‘lr’: 0.02