Measured Antivirus Effectiveness

I wanted to share with you some insight from the data that originated from our customers’ networks last week. This time, we wanted to provide some information on how different antivirus vendors perform on the .exe, .dll, .pdf, and .zip files seen around the world.
This table shows the relative hit ratio of all the antivirus vendors hosted by Virus Total on 697 confirmed bad files. You will notice that 43% of the time none of the antivirus products detected anything. The top performer is McAfee-GW-Edition with a 37% detection rate.

Looking at the types of samples detected, one can also consider which Antivirus Vendors were able catch the worst malicious code. We assigned an Average Priority of 1 to spyware or unwanted software and an Average Priority of 100 to known Trojans or unclassified malware. Then, we multiplyed the Average Priority by the Detection Rate, giving rise to the Severity column. This column shows which Antivirus Vendors found the most dangerous code. This week Arcabit wins with a Detection Rate of 29%, an Average Priority of 30.17, and a Severity of 8.96.

Antivirus Vendor True Positives Average Priority Detection Rate Severity
None 300 – 0.430416 (mss) –
Arcabit 207 30.17 0.296987 8.96
F-Secure 192 28.84 0.275466 7.95
ESET-NOD32 205 24.18 0.294118 7.11
AVG 129 37.07 0.185079 6.86
Avast 200 23.77 0.286944 6.82
Qihoo-360 207 22.52 0.296987 6.69
GData 223 20.09 0.319943 6.43
McAfee-GW-Edition 264 16.75 0.378766 6.34
CAT-QuickHeal 162 27.28 0.232425 6.34
VIPRE 172 23.45 0.246772 5.79
Cyren 201 19.72 0.288379 5.69
Panda 85 46.42 0.121951 5.66
F-Prot 160 24.51 0.229555 5.63
ClamAV 62 63.27 0.088953 5.63
Fortinet 105 29.29 0.150646 4.41
McAfee 117 25.54 0.167862 4.29
Avira 210 12.79 0.301291 3.85
Bkav 83 30.82 0.119082 3.67
MicroWorld-eScan 162 15.06 0.232425 3.50
BitDefender 161 15.14 0.230990 3.50
Emsisoft 160 15.23 0.229555 3.50
CMC 24 100.00 0.034433 3.44
Kaspersky 86 27.48 0.123386 3.39
TrendMicro 63 37.14 0.090387 3.36
Ad-Aware 140 16.56 0.200861 3.33
Ikarus 209 10.95 0.299857 3.28
AVware 95 23.93 0.136298 3.26
Comodo 69 26.83 0.098996 2.66
Sophos 77 20.29 0.110473 2.24
Rising 195 7.09 0.279770 1.98
Tencent 50 24.76 0.071736 1.78
ALYac 108 9.25 0.154950 1.43
Microsoft 25 36.64 0.035868 1.31
K7AntiVirus 109 5.54 0.156385 0.87
DrWeb 134 3.96 0.192253 0.76
Malwarebytes 222 1.89 0.318508 0.60
K7GW 120 3.48 0.172166 0.60
Antiy-AVL 74 5.01 0.106169 0.53
Symantec 161 1.61 0.230990 0.37
VBA32 53 4.74 0.076040 0.36
nProtect 16 13.38 0.022956 0.31
NANO-Antivirus 76 2.30 0.109039 0.25
SUPERAntiSpyware 38 3.61 0.054519 0.20
Jiangmin 38 3.61 0.054519 0.20
Zillya 131 1.00 0.187948 0.19
ByteHero 4 25.75 0.005739 0.15
Baidu-International 83 1.00 0.119082 0.12
AhnLab-V3 80 1.00 0.114778 0.11
Agnitum 57 1.00 0.081779 0.08
ViRobot 12 1.00 0.017217 0.02
AegisLab 9 1.00 0.012912 0.01
TotalDefense 2 1.00 0.002869 0.00
Zoner 1 1.00 0.001435 0.00
Alibaba 1 1.00 0.001435 0.00

Our sandbox was able to detect the remaining samples (the missing 43%).

antivirus

The bubble graph above illusrates the Severity (Detection Rate * Average Priority) verses the Prevalence (Detection Rate * Total Priority). The detection rate is encoded in color and the size of the bubble is proportional to how many customers saw the malware.
If you are curious about more statistics like this, you can visit https://www.metaflows.com/stats/ (best viewed on a desktop) for a ton of additional information. If you want a quick fix, watch some of our videos at https://www.metaflows.com/saas/.