Table 3 MTLPT compared with baseline methods on real-world vulnerability data prediction performance.

From: A lightweight transformer based multi task learning model with dynamic weight allocation for improved vulnerability prediction

Vulnerability

Method

LSTM

RNN

CNN

RF

AST + ML16

Code 2vec44

Code 2vec + MLP16

MTL PT (without \(L_t\))

MTL PT

CWE-119

Recall

–

–

36.59%

0.73%

51.50%

4.00%

87.30%

80.03%

79.23%

F1-Score

–

–

43.41%

1.44%

50.90%

4.60%

12.00%

57.55%

59.61%

AUC

50.08%

50.25%

86.17%

50.35%

–

–

–

90.62%

90.12%

MCC

–

–

37.94%

6.79%

–

–

–

52.62%

54.69%

CWE-120

Recall

–

0.04%

53.63%

2.97%

44.00%

6.40%

79.00%

62.11%

74.10%

F1-Score

–

0.08%

59.53%

5.66%

42.70%

8.10%

22.50%

68.08%

72.35%

AUC

47.27%

50.58%

85.90%

51.18%

–

–

–

90.26%

90.21%

MCC

–

0.39%

49.17%

9.41%

–

–

–

59.80%

63.50%

CWE-469

Recall

–

–

–

1.63%

18.70%

0.00%

18.80%

13.01%

57.72%

F1-Score

–

–

–

3.20%

9.00%

0.00%

1.70%

18.66%

26.39%

AUC

50.45%

47.60%

83.15%

50.81%

–

–

–

91.81%

89.99%

MCC

–

–

–

12.67%

–

–

–

20.12%

29.99%

CWE-476

Recall

0.082%

–

18.75%

1.81%

52.10%

3.70%

100.00%

57.15%

55.10%

F1-Score

0.16%

–

29.98%

3.55%

59.80%

4.00%

4.70%

44.42%

48.01%

AUC

48.61%

50.15%

77.34%

50.90%

–

–

–

85.47%

85.20%

MCC

2.78%

–

35.78%

12.42%

–

–

–

41.21%

44.64%

CWE-other

Recall

–

0.03%

29.61%

1.35%

35.30%

7.70%

54.70%

60.74%

61.67%

F1-Score

–

0.06%

39.82%

2.66%

27.00%

9.00%

14.80%

58.99%

58.50%

AUC

49.11%

49.68%

78.00%

50.65%

–

–

–

85.03%

84.87%

MCC

–

0.098%

34.71%

9.31%

–

–

–

49.94%

49.16%

Back to article page