Table 4 5-fold cross-validated paired t-tests comparing macro-average precision, recall, F1 scores, and accuracy of FinBERT-based models against the xFiTRNN model.
Embedding | Model | t-value (A) | p-value (A) | t-value (P) | p-value (P) | t-value (R) | p-value (R) | t-value (F1) | p-value (F1) |
|---|---|---|---|---|---|---|---|---|---|
FinBERT | 1 Hidden Layer NN | 42.57 | 3.24e–11 | 30.14 | 5.53e–10 | 38.79 | 2.13e–11 | 35.23 | 9.83e–11 |
2 Hidden Layers NN | 47.83 | 1.52e–12 | 34.92 | 2.34e–11 | 45.31 | 3.41e–12 | 40.15 | 7.63e–12 | |
3 Hidden Layers NN | 54.27 | 4.83e–14 | 40.56 | 6.27e–13 | 60.81 | 1.24e–14 | 50.34 | 3.14e–13 | |
BiGRU + 3 Hidden Layers NN | 58.34 | 2.73e–15 | 45.63 | 1.42e–14 | 65.44 | 8.93e–16 | 55.85 | 4.53e–15 | |
BiGRU + CNN | 62.13 | 9.13e–16 | 50.28 | 3.24e–15 | 70.39 | 2.41e–17 | 60.57 | 1.82e–16 |