Table 9 Comparison of outliers counts and percentages outside 2 and 3—sigma limits in original and hybrid models before and after addressing heterogeneity.
Machine learning models | Robust methods | Highest important variables | Before heterogeneity | After heterogeneity | ||
|---|---|---|---|---|---|---|
\(\mu \pm 2\sigma \left( \% \right)\) | \(\mu \pm 3\sigma \left( \% \right)\) | \(\mu \pm 2\sigma \left( \% \right)\) | \(\mu \pm 3\sigma \left( \% \right)\) | |||
Elastic Net | Original | 25 | 142(7.42) | 6(0.31) | 161(8.41) | 12(0.63) |
35 | 125(6.53) | 10(0.52) | 167(8.73) | 12(0.63) | ||
45 | 120(6.27) | 6(0.31) | 163(8.52) | 12(0.63) | ||
55 | 117(6.11) | 9(0.47) | 158(8.25) | 11(0.57) | ||
100 | 113(5.90) | 4(0.21) | 163(8.52) | 8(0.42) | ||
M Bi-Square | 25 | 44(2.30) | 11(0.57) | 57(2.97) | 17(0.89) | |
35 | 51(2.66) | 12(0.63) | 49(2.56) | 18(0.94) | ||
45 | 44(2.30) | 14(0.73) | 52(2.72) | 13(0.68) | ||
55 | 26(1.36) | 11(0.57) | 58(3.03) | 17(0.89) | ||
100 | 26(1.36) | 3(0.16) | 49(2.56) | 15(0.78) | ||
M Hampel | 25 | 44(2.30) | 14(0.73) | 60(3.13) | 17(0.89) | |
35 | 58(3.03) | 10(0.52) | 53(2.77) | 16(0.84) | ||
45 | 54(2.82) | 17(0.89) | 50(2.61) | 14(0.73) | ||
55 | 34(1.78) | 12(0.63) | 58(3.03) | 16(0.84) | ||
100 | 33(1.72) | 8(0.42) | 49(2.56) | 15(0.78) | ||
M Huber | 25 | 46(2.40) | 13(0.68) | 57(2.98) | 17(0.89) | |
35 | 57(2.98) | 10(0.52) | 48(2.51) | 16(0.84) | ||
45 | 55(2.87) | 16(0.84) | 51(2.66) | 16(0.84) | ||
55 | 43(2.25) | 11(0.57) | 62(3.24) | 15(0.78) | ||
100 | 44(2.30) | 10(0.52) | 45(2.35) | 15(0.78) | ||
MM | 25 | 42(2.19) | 9(0.47) | 60(3.13) | 17(0.89) | |
35 | 33(1.72) | 8(0.42) | 47(2.46) | 16(0.84) | ||
45 | 46(2.40) | 14(0.73) | 56(2.93) | 14(0.73) | ||
55 | 41(2.14) | 8(0.42) | 23(1.20) | 14(0.73) | ||
100 | 20(1.04) | 10(0.52) | 39(2.04) | 15(0.78) | ||
S | 25 | 27(1.41) | 16(0.84) | 40(2.09) | 8(0.42) | |
35 | 45(2.35) | 11(0.57) | 38(1.99) | 7(0.37) | ||
45 | 37(1.93) | 15(0.78) | 36(1.88) | 0(0) | ||
55 | 27(1.41) | 24(1.25) | 23(1.20) | 0(0) | ||
100 | 19(0.99) | 6(0.31) | 50(2.61) | 13(0.68) | ||
Ridge | Original | 25 | 144(7.52) | 9(0.47) | 148(7.73) | 5(0.26) |
35 | 133(6.95) | 7(0.37) | 152(7.94) | 8(0.42) | ||
45 | 135(7.05) | 5(0.26) | 162(8.46) | 10(0.52) | ||
55 | 135(7.05) | 6(0.31) | 164(8.57) | 12(0.63) | ||
100 | 121(6.32) | 5(0.26) | 168(8.78) | 12(0.63) | ||
M Bi-Square | 25 | 14(0.73) | 1(0.05) | 40(2.09) | 7(0.37) | |
35 | 23(1.20) | 0(0) | 53(2.77) | 12(0.63) | ||
45 | 24(1.25) | 13(0.68) | 51(2.66) | 12(0.63) | ||
55 | 24(1.25) | 12(0.63) | 46(2.40) | 14(0.73) | ||
100 | 25(1.31) | 9(0.47) | 34(1.78) | 14(0.73) | ||
M Hampel | 25 | 22(1.15) | 2(0.10) | 37(1.93) | 7(0.37) | |
35 | 42(2.19) | 7(0.37) | 52(2.72) | 15(0.78) | ||
45 | 20(1.04) | 12(0.63) | 52(2.72) | 12(0.63) | ||
55 | 34(1.78) | 10(0.52) | 48(2.51) | 14(0.73) | ||
100 | 26(1.36) | 11(0.57) | 46(2.40) | 17(0.89) | ||
M Huber | 25 | 16(0.84) | 1(0.05) | 39(2.04) | 8(0.42) | |
35 | 37(1.93) | 3(0.16) | 48(2.51) | 13(0.68) | ||
45 | 28(1.46) | 10(0.52) | 50(2.61) | 12(0.63) | ||
55 | 43(2.25) | 12(0.63) | 45(2.35) | 14(0.73) | ||
100 | 39(2.04) | 11(0.57) | 40(2.09) | 15(0.78) | ||
MM | 25 | 45(2.35) | 10(0.52) | 42(2.19) | 8(0.42) | |
35 | 21(1.10) | 1(0.05) | 53(2.77) | 12(0.63) | ||
45 | 15(0.78) | 1(0.05) | 33(1.72) | 12(0.63) | ||
55 | 28(1.46) | 23(1.20) | 42(2.18) | 12(0.63) | ||
100 | 19(0.99) | 6(0.31) | 45(2.35) | 14(0.73) | ||
S | 25 | 42(2.19) | 11(0.57) | 34(1.78) | 6(0.31) | |
35 | 38(1.99) | 26(1.36) | 28(1.46) | 2(0.10) | ||
45 | 18(0.94) | 1(0.05) | 17(0.89) | 5(0.26) | ||
55 | 28(1.46) | 17(0.89) | 21(1.10) | 1(0.05) | ||
100 | 20(1.04) | 8(0.42) | 33(1.72) | 13(0.68) | ||
LASSO | Original | 25 | 150(7.84) | 7(0.37) | 167(8.73) | 9(0.47) |
35 | 143(7.47) | 5(0.26) | 162(8.46) | 8(0.42) | ||
45 | 136(7.11) | 7(0.37) | 163(8.52) | 14(0.73) | ||
55 | 131(6.84) | 6(0.31) | 158(8.25) | 9(0.47) | ||
100/89 for after | 115(6.01) | 6(0.31) | 161(8.41) | 8(0.42) | ||
M Bi-Square | 25 | 35(1.83) | 4(0.21) | 44(2.30) | 10(0.52) | |
35 | 56(2.93) | 17(0.89) | 54(2.82) | 16(0.84) | ||
45 | 57(2.98) | 20(1.04) | 53(2.77) | 14(0.73) | ||
55 | 59(3.08) | 14(0.73) | 56(2.93) | 19(0.99) | ||
100/89 for after | 24(1.25) | 9(0.47) | 52(2.72) | 18(0.94) | ||
M Hampel | 25 | 41(2.14) | 7(0.37) | 42(2.19) | 9(0.47) | |
35 | 54(2.82) | 10(0.52) | 57(2.98) | 20(1.04) | ||
45 | 55(2.87) | 16(0.84) | 53(2.77) | 17(0.89) | ||
55 | 56(2.93) | 13(0.68) | 52(2.72) | 19(0.99) | ||
100/89 for after | 30(1.57) | 7(0.37) | 53(2.77) | 15(0.78) | ||
M Huber | 25 | 39(2.04) | 6(0.31) | 46(2.40) | 10(0.52) | |
35 | 55(2.87) | 10(0.52) | 55(2.87) | 17(0.89) | ||
45 | 54(2.82) | 16(0.84) | 52(2.72) | 18(0.94) | ||
55 | 56(2.93) | 12(0.63) | 55(2.87) | 16(0.84) | ||
100/89 for after | 34(1.78) | 8(0.42) | 54(2.82) | 14(0.73) | ||
MM | 25 | 31(1.62) | 8(0.42) | 43(2.25) | 10(0.52) | |
35 | 45(2.35) | 10(0.52) | 56(2.93) | 14(0.73) | ||
45 | 32(1.67) | 15(0.78) | 54(2.82) | 15(0.78) | ||
55 | 28(1.46) | 23(1.20) | 28(1.46) | 13(0.68) | ||
100/89 for after | 30(1.57) | 23(1.20) | 52(2.72) | 18(0.94) | ||
S | 25 | 23(1.20) | 12(0.63) | 29(1.52) | 9(0.47) | |
35 | 28(1.46) | 14(0.73) | 38(1.99) | 2(0.10) | ||
45 | 29(1.52) | 23(1.20) | 16(0.84) | 1(0.05) | ||
55 | 29(1.52) | 24(1.25) | 20(1.04) | 9(0.47) | ||
