Table 3 Evaluation (Precision, Recall, and F-1 score) of five popular libraries and a few heuristics based on human perspective. The best and second-best results are highlighted in bold and italics.
Library/Heuristic | Precision | Recall | F-1 Score | Accuracy (%) | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
Negative | Neutral | Positive | Negative | Neutral | Positive | Negative | Neutral | Positive | ||
L-1: VADER | 0.73 | 0.57 | 0.73 | 0.76 | 0.79 | 0.26 | 0.74 | 0.66 | 0.39 | 66 |
L-2: Afinn | 0.71 | 0.64 | 0.50 | 0.79 | 0.54 | 0.48 | 0.75 | 0.59 | 0.49 | 64 |
L-3: TwitterSentiment | 0.55 | 0.50 | 0.25 | 0.76 | 0.01 | 0.41 | 0.63 | 0.01 | 0.31 | 44 |
L-4: Transformer | 0.58 | 0.83 | 0.52 | 0.96 | 0.03 | 0.54 | 0.72 | 0.06 | 0.53 | 57 |
L-5: TextBlob | 0.90 | 0.40 | 0.44 | 0.43 | 0.59 | 0.59 | 0.58 | 0.48 | 0.5 | 52 |
H-1: Majority voting on all five (L-1 to L-5) | 0.70 | 0.60 | 0.56 | 0.81 | 0.52 | 0.51 | 0.75 | 0.56 | 0.53 | 65 |
H-2: Majority voting on L-1, L-2, and L-4. In case of a tie, decide neutral based on only L-4; otherwise, decide based on only L-1 | 0.69 | 0.68 | 0.68 | 0.64 | 0.59 | 0.64 | 0.76 | 0.56 | 0.54 | 64 |
H-3: Majority voting on L-1, L-2, and L-4. In case of a tie, decide neutral based on only L-4; otherwise, decide based on only L-2 | 0.70 | 0.63 | 0.67 | 0.83 | 0.61 | 0.46 | 0.76 | 0.62 | 0.54 | 68 |
