Table 11 Cross-validation results for classifiers based on the ANN model and the BERT model.

From: Analysis of the retraining strategies for multi-label text message classification in call/contact center systems

Model name

Reference model

STRATEGY 1

STRATEGY 2

Accuracy

SD

Emotica

SD

Accuracy

SD

Emotica

SD

Accuracy

SD

Emotica

SD

[%]

[%]

[%]

[%]

[%]

[%]

[%]

[%]

[%]

[%]

[%]

[%]

Training data

 ANN-PCA

93.62

0.18

77.68

0.49

97.47

0.34

89.47

1.30

98.88

0.09

94.92

0.33

 ANN-LSA

95.35

0.19

82.99

0.87

97.82

0.03

90.82

0.40

99.45

0.02

97.08

0.28

 ANN-ICA

95.50

0.10

83.44

0.31

96.01

0.03

84.65

0.22

97.72

0.14

90.46

0.71

 BERT

96.27

0.10

86.32

0.59

99.90

0.07

99.45

0.35

99.77

0.04

98.70

0.28

Testing data

 ANN-PCA

93.62

0.37

77.68

0.99

93.97

0.52

78.58

1.74

93.97

0.32

79.03

0.70

 ANN-LSA

95.35

0.39

82.98

1.74

95.38

0.48

82.72

1.34

95.38

0.54

83.79

1.77

 ANN-ICA

95.50

0.20

83.44

0.62

95.68

0.28

83.71

0.89

95.52

0.21

83.44

1.10

 BERT

96.54

0.16

87.41

0.24

96.56

0.35

87.72

0.41

96.49

0.44

86.92

2.06

  1. Significant values are in bold.
  2. SD Standard deviation.