Table 9 Comparison of different tools in smart bugs wild dataset.

From: An elegant intellectual engine towards automation of blockchain smart contract vulnerability detection

Vulnerabilities	Tools	Precision	Recall	F1-score	FPR
Time manipulation	Mythril	0.94	0.79	0.90	0.61
	Slither	0.86	0.91	0.78	0.86
	Oyente	0.93	0.92	0.92	0.54
	Osiris	0.86	0.76	0.97	0.77
	Smartcheck	0.72	0.86	0.87	0.65
	Manticore	0.85	0.91	0.79	0.76
	Maian	0.92	0.89	0.91	0.24
	Securify	0.78	0.80	0.89	0.67
	Honeybadger	0.81	0.92	0.78	0.45
Front running	Mythril	0.79	0.93	0.95	0.64
	Slither	0.94	0.89	0.78	0.82
	Oyente	0.90	0.91	0.89	0.56
	Osiris	0.72	0.89	0.75	0.62
	Smartcheck	0.89	0.75	0.90	0.54
	Manticore	0.83	0.79	0.73	0.76
	Maian	0.76	0.82	0.79	0.65
	Securify	0.79	0.90	0.93	0.84
	Honeybadger	0.91	0.82	0.91	0.76
DoS	Mythril	0.90	0.89	0.74	0.64
	Slither	0.78	0.75	0.91	0.65
	Oyente	0.79	0.90	0.88	0.87
	Osiris	0.90	0.82	0.80	0.68
	Smartcheck	0.82	0.79	0.75	0.76
	Manticore	0.90	0.72	0.79	0.87
	Maian	0.92	0.94	0.83	0.43
	Securify	0.89	0.92	0.92	0.73
	Honeybadger	0.93	0.83	0.88	0.84
Reentrancy	Mythril	0.73	0.76	0.87	0.56
	Slither	0.89	0.83	0.94	0.69
	Oyente	0.81	0.92	0.70	0.76
	Osiris	0.87	0.90	0.85	0.54
	Smartcheck	0.92	0.76	0.89	0.65
	Manticore	0.86	0.89	0.90	0.84
	Maian	0.84	0.92	0.92	0.54
	Securify	0.93	0.80	0.76	0.74
	Honeybadger	0.80	0.82	0.72	0.85
Access control	Mythril	0.89	0.76	0.78	0.56
	Slither	0.90	0.82	0.90	0.74
	Oyente	0.74	0.88	0.92	0.81
	Osiris	0.79	0.76	0.76	0.55
	Smartcheck	0.83	0.90	0.87	0.82
	Manticore	0.90	0.89	0.73	0.88
	Maian	0.79	0.72	0.79	0.75
	Securify	0.87	0.80	0.80	0.77
	Honeybadger	0.88	0.76	0.88	0.84
Arithmetic	Mythril	0.78	0.91	0.87	0.80
	Slither	0.90	0.78	0.88	0.63
	Oyente	0.86	0.85	0.90	0.87
	Osiris	0.82	0.91	0.78	0.76
	Smartcheck	0.92	0.70	0.94	0.65
	Manticore	0.79	0.89	0.76	0.80
	Maian	0.91	0.72	0.92	0.56
	Securify	0.92	0.80	0.80	0.52
	Honeybadger	0.76	0.91	0.82	0.76
Unchecked low calls	Mythril	0.87	0.92	0.77	0.89
	Slither	0.92	0.79	0.80	0.80
	Oyente	0.76	0.89	0.93	0.67
	Osiris	0.79	0.76	0.74	0.46
	Smartcheck	0.90	0.70	0.90	0.88
	Manticore	0.82	0.88	0.87	0.67
	Maian	0.89	0.86	0.73	0.49
	Securify	0.78	0.92	0.92	0.81
	Honeybadger	0.91	0.81	0.80	0.69
Others	Mythril	0.91	0.80	0.89	0.84
	Slither	0.78	0.85	0.70	0.87
	Oyente	0.72	0.92	0.76	0.49
	Osiris	0.79	0.87	0.88	0.77
	Smartcheck	0.93	0.75	0.92	0.67
	Manticore	0.87	0.80	0.74	0.80
	Maian	0.82	0.73	0.83	0.66
	Securify	0.80	0.77	0.89	0.67
	Honeybadger	0.73	0.89	0.84	0.59

Back to article page

Table 9 Comparison of different tools in smart bugs wild dataset.

Search

Quick links