Table 3 Comparison of the Linear Regression (LR) using Standard Least Squares method, Generalized Regression (GR) model using a logarithmic transformation and the double-lasso method, and the main ablation experiments of the Artificial Neural Network (ANN) methods tried.

# Layers	# Hidden nodes	Activation functions			R²			Root Mean Square Error (RMSE)			Mean absolute deviance			Average processing time (s)
# Layers	# Hidden nodes	Sigmoidal	Gaussian	Linear	Train	Test	Valid	Train	Test	Valid	Train	Test	Valid
1 (LR)	0				0.731	0.739	0.770	3.107	3.062	2.928	0.040	− 0.243	− 0.157	1
1 (GR)	0				0.757	0.770	0.783	3.261	3.174	3.155	− 0.229	0.035	− 0.993	1
2**	0	2	2	2	0.794	0.780	0.822	3.174	3.221	3.088	2.195	2.308	2.123	100
		2*	2*	2*	0.809	0.787	0.831	3.117	3.192	3.038	2.117	2.273	2.070	240
		4	4	4	0.814	0.784	0.824	3.102	3.269	3.092	2.088	2.289	2.114	120
		4*	4*	4*	0.813	0.789	0.832	3.097	3.196	3.031	2.092	3.196	3.031	660
		6	6	6	0.818	0.789	0.824	3.065	3.178	3.083	2.066	2.262	2.117	150
		6*	6*	6*	0.818	0.797	0.830	3.066	3.144	3.022	2.066	2.222	2.075	1800
3**	6	2	2	2	0.798	0.785	0.822	3.147	3.214	3.091	2.177	2.285	2.126	75
	12	4	4	4	0.810	0.790	0.832	3.088	3.193	3.030	2.110	2.256	2.065	90
	18	6	6	6	0.828	0.797	0.824	3.010	3.165	3.115	2.005	2.217	2.217	180

The ablation study identified the 2 hidden-layer ANN design (i.e., each layer with three sigmodal, three linear and three Gaussian activation functions) as the best compromise between improved performance and processing speed (bold characters).
*Using an additive sequence of 100 models based on a learning rate of 0.1.
**Using for optimization a robust fit with a squared penalty method and transformed covariates.

Quick links

Search