Table 3 Performance analysis of EGMF-GR across datasets. Avg denotes the arithmetic mean over horizons 96, 192, 336, and 720 under the matched protocol. A dagger indicates statistical significance after Benjamini Hochberg correction based on paired Wilcoxon signed rank tests on rolling origin errors, comparing EGMF-GR with the corresponding backbone baseline under the same setting. For compactness, significance marks are reported for Avg MSE.
From: Hybrid evolutionary-gradient training improves long-term time series forecasting
Models | Our iTransformer | iTransformer | Our crossformer | Crossformer | Our TimesNet | TimesNet | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Metric | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |
ETTm1 | 96 | 0.328 | 0.367 | 0.334 | 0.368 | 0.347 | 0.401 | 0.404 | 0.426 | 0.332 | 0.369 | 0.338 | 0.375 |
192 | 0.374 | 0.388 | 0.377 | 0.391 | 0.398 | 0.428 | 0.450 | 0.451 | 0.377 | 0.392 | 0.374 | 0.387 | |
336 | 0.408 | 0.411 | 0.426 | 0.420 | 0.551 | 0.557 | 0.532 | 0.515 | 0.430 | 0.426 | 0.410 | 0.411 | |
720 | 0.474 | 0.447 | 0.491 | 0.459 | 0.765 | 0.674 | 0.666 | 0.589 | 0.516 | 0.467 | 0.478 | 0.450 | |
Avg | 0.396 \(^{\dagger }\) | 0.403 | 0.407 | 0.410 | 0.515 | 0.515 | 0.513 | 0.495 | 0.414 \(^{\dagger }\) | 0.414 | 0.415 | 0.406 | |
ETTm2 | 96 | 0.177 | 0.262 | 0.180 | 0.264 | 0.237 | 0.329 | 0.287 | 0.366 | 0.179 | 0.260 | 0.187 | 0.267 |
192 | 0.241 | 0.301 | 0.250 | 0.309 | 0.388 | 0.443 | 0.414 | 0.492 | 0.252 | 0.305 | 0.249 | 0.309 | |
336 | 0.304 | 0.342 | 0.311 | 0.348 | 0.979 | 0.722 | 0.597 | 0.542 | 0.311 | 0.342 | 0.321 | 0.351 | |
720 | 0.407 | 0.399 | 0.412 | 0.407 | 3.305 | 1.212 | 1.730 | 1.042 | 0.417 | 0.403 | 0.408 | 0.403 | |
Avg | 0.282 \(^{\dagger }\) | 0.326 | 0.288 | 0.332 | 1.227 | 0.677 | 0.757 | 0.611 | 0.290 \(^{\dagger }\) | 0.328 | 0.291 | 0.332 | |
ETTh1 | 96 | 0.386 | 0.400 | 0.386 | 0.405 | 0.388 | 0.409 | 0.423 | 0.448 | 0.414 | 0.428 | 0.384 | 0.402 |
192 | 0.440 | 0.431 | 0.441 | 0.436 | 0.485 | 0.486 | 0.471 | 0.474 | 0.461 | 0.454 | 0.436 | 0.429 | |
336 | 0.483 | 0.454 | 0.487 | 0.458 | 0.606 | 0.575 | 0.570 | 0.546 | 0.488 | 0.472 | 0.491 | 0.469 | |
720 | 0.497 | 0.483 | 0.503 | 0.491 | 0.832 | 0.711 | 0.653 | 0.621 | 0.525 | 0.506 | 0.521 | 0.500 | |
Avg | 0.452 \(^{\dagger }\) | 0.442 | 0.454 | 0.447 | 0.578 | 0.545 | 0.529 | 0.522 | 0.472 \(^{\dagger }\) | 0.465 | 0.473 | 0.450 | |
ETTh2 | 96 | 0.291 | 0.341 | 0.297 | 0.349 | 0.633 | 0.584 | 0.745 | 0.584 | 0.319 | 0.361 | 0.340 | 0.374 |
192 | 0.382 | 0.397 | 0.380 | 0.400 | 0.740 | 0.606 | 0.877 | 0.656 | 0.404 | 0.409 | 0.402 | 0.414 | |
336 | 0.425 | 0.433 | 0.428 | 0.432 | 1.814 | 1.062 | 1.043 | 0.731 | 0.423 | 0.432 | 0.452 | 0.452 | |
720 | 0.430 | 0.446 | 0.427 | 0.445 | 3.751 | 1.636 | 1.104 | 0.763 | 0.455 | 0.467 | 0.462 | 0.468 | |
Avg | 0.382 \(^{\dagger }\) | 0.404 | 0.383 | 0.407 | 1.735 | 0.972 | 0.942 | 0.684 | 0.400 \(^{\dagger }\) | 0.417 | 0.414 | 0.427 | |
Electricity | 96 | 0.150 | 0.241 | 0.148 | 0.240 | 0.142 | 0.241 | 0.219 | 0.314 | 0.165 | 0.267 | 0.168 | 0.272 |
192 | 0.163 | 0.253 | 0.162 | 0.253 | 0.157 | 0.257 | 0.231 | 0.322 | 0.175 | 0.275 | 0.184 | 0.289 | |
336 | 0.178 | 0.270 | 0.178 | 0.269 | 0.183 | 0.285 | 0.246 | 0.337 | 0.191 | 0.293 | 0.198 | 0.300 | |
720 | 0.220 | 0.305 | 0.225 | 0.317 | 0.275 | 0.305 | 0.280 | 0.363 | 0.221 | 0.315 | 0.220 | 0.320 | |
Avg | 0.178 | 0.267 | 0.178 | 0.270 | 0.189 | 0.272 | 0.244 | 0.334 | 0.188 \(^{\dagger }\) | 0.288 | 0.193 | 0.295 | |
Exchange | 96 | 0.095 | 0.216 | 0.086 | 0.206 | 0.267 | 0.383 | 0.256 | 0.367 | 0.097 | 0.223 | 0.107 | 0.234 |
192 | 0.194 | 0.314 | 0.177 | 0.299 | 0.499 | 0.532 | 0.470 | 0.509 | 0.223 | 0.342 | 0.226 | 0.344 | |
336 | 0.346 | 0.429 | 0.331 | 0.417 | 0.853 | 0.705 | 1.268 | 0.883 | 0.406 | 0.470 | 0.367 | 0.448 | |
720 | 0.876 | 0.707 | 0.847 | 0.691 | 1.446 | 0.977 | 1.767 | 1.068 | 0.964 | 0.746 | 0.964 | 0.746 | |
Avg | 0.378 | 0.417 | 0.360 | 0.403 | 0.791 | 0.649 | 0.940 | 0.707 | 0.423 | 0.445 | 0.416 | 0.443 | |
Traffic | 96 | 0.400 | 0.272 | 0.395 | 0.268 | 0.513 | 0.256 | 0.522 | 0.290 | 0.575 | 0.310 | 0.593 | 0.321 |
192 | 0.419 | 0.278 | 0.417 | 0.276 | 0.521 | 0.268 | 0.530 | 0.293 | 0.605 | 0.315 | 0.617 | 0.336 | |
336 | 0.431 | 0.287 | 0.433 | 0.283 | 0.520 | 0.302 | 0.558 | 0.305 | 0.608 | 0.317 | 0.629 | 0.336 | |
720 | 0.460 | 0.302 | 0.467 | 0.302 | 0.535 | 0.315 | 0.589 | 0.328 | 0.615 | 0.321 | 0.640 | 0.350 | |
Avg | 0.428 | 0.285 | 0.428 | 0.282 | 0.522 | 0.285 | 0.550 | 0.304 | 0.601 \(^{\dagger }\) | 0.316 | 0.620 | 0.336 | |
Weather | 96 | 0.180 | 0.221 | 0.174 | 0.214 | 0.167 | 0.236 | 0.158 | 0.230 | 0.167 | 0.216 | 0.172 | 0.220 |
192 | 0.227 | 0.260 | 0.221 | 0.254 | 0.208 | 0.279 | 0.206 | 0.277 | 0.216 | 0.259 | 0.219 | 0.261 | |
336 | 0.282 | 0.300 | 0.278 | 0.296 | 0.268 | 0.326 | 0.272 | 0.335 | 0.279 | 0.301 | 0.280 | 0.306 | |
720 | 0.357 | 0.348 | 0.358 | 0.347 | 0.360 | 0.399 | 0.398 | 0.418 | 0.364 | 0.357 | 0.365 | 0.359 | |
Avg | 0.262 | 0.282 | 0.258 | 0.278 | 0.251 | 0.310 | 0.259 | 0.315 | 0.257 \(^{\dagger }\) | 0.283 | 0.259 | 0.286 | |