1. EICAR 2009, 12 May 2009 Checkvir Realtime Anti-Malware Testing and Certification Dr. Ferenc Leitold, Veszprog Ltd. fleitold@veszprog.hu www.checkvir.com

2. EICAR 2009, 12 May 2009 Contents Purpose of Checkvir testing Testing methodology –Technical background –Testing procedures Current state Difficulties Questions

3. EICAR 2009, 12 May 2009 Purpose of Checkvir testing AVG1,7 ESET2,6 F-PROT1,2 F-Secure5 Kaspersky23,2 McAfee35,4 Panda44,7 Sophos5,4 Sunbelt0,6 Symantec233,4 VirusBuster1 source: AV-Test.org Number of updates / day Problems: Big number of updates Cloud technology Solutions are continually changing Testing all versions are impossible

4. EICAR 2009, 12 May 2009 Purpose of Checkvir testing Testing all versions are impossible Executes tests as frequently as possible Automatic methods have to be developed Big number of computers have to be used

5. EICAR 2009, 12 May 2009 Purpose of Checkvir testing The main purposes: Provide reliable, correct and exact information mainly about: - effectiveness - performance in a balanced way (AMTSO’s principle) Provide naming cross-reference information performanceeffectiveness

6. EICAR 2009, 12 May 2009 Pack and save the new image AV update Unpack previous image Publish results Unpack last image New version? Initialize testing Execute test(s) Save results and reports Analyze results yes nono Testing methodology updatetest

7. EICAR 2009, 12 May 2009 Testing methodology Technical background clients “malware proxy” server webserver controller archiver firewall firewall & router

8. EICAR 2009, 12 May 2009 Testing methodology Testing procedures Malware knowledge (detection, disinfection) –against known, unknown malware and clean files –on-demand, on-access and proactive executions “Container” checking capabilities –archives, email clients’ data files, … Speed –on-demand, on-access –boot time Functionality Stability … speedknowledge

9. EICAR 2009, 12 May 2009 Testing methodology Testing procedures Why the speed is so important?

10. EICAR 2009, 12 May 2009 Testing methodology Testing procedures

11. EICAR 2009, 12 May 2009 Testing methodology Testing procedures Testing bootup time What is more important? BOOTUP TIME or SECURE BOOTING DEMO

12. EICAR 2009, 12 May 2009 Testing methodology Testing procedures

13. EICAR 2009, 12 May 2009 Testing methodology Testing procedures Avast AVG Avira Bitdefender Eset e-Trust F-Prot F-Secure Fortinet Ikarus Kaspersky Microsoft Rising Sophos Symantec Trend Micro VirusBuster Bootup protection test

14. EICAR 2009, 12 May 2009 Testing methodology Testing procedures Avast AVG Avira Bitdefender Eset e-Trust F-Prot F-Secure Fortinet Ikarus Kaspersky Microsoft Rising Sophos Symantec Trend Micro VirusBuster Bootup protection test

15. EICAR 2009, 12 May 2009 Testing methodology Testing procedures Avast AVG Avira Bitdefender Eset e-Trust F-Prot F-Secure Fortinet Ikarus Kaspersky Microsoft Rising Sophos Symantec Trend Micro VirusBuster Bootup protection test

16. EICAR 2009, 12 May 2009 Testing methodology Testing procedures Avast AVG Avira Bitdefender Eset e-Trust F-Prot F-Secure Fortinet Ikarus Kaspersky Microsoft Rising Sophos Symantec Trend Micro VirusBuster Bootup protection test

17. EICAR 2009, 12 May 2009 Testing methodology Proactive tests vs. AM cloud technology Problems: AM products use cloud technology –> traffic should be allowed Malware use cloud technology –> traffic should be allowed –> How can we protect the world? –> How can we provide exactly the same environment for solutions?

18. EICAR 2009, 12 May 2009 Testing methodology Proactive tests vs. AM cloud technology clients “malware proxy” server webserver controller archiver firewall firewall & router

19. EICAR 2009, 12 May 2009 Testing methodology Settings By default, DEFAULT settings are used Minimal functionality is required: –Execute tests without user interaction –Automatically clean the infected file (if not possible -> delete) –Report file generation

20. EICAR 2009, 12 May 2009 Current state What is working now? The frame system The website Automatic procedures of some products Preliminary selection and validation of the samples

21. EICAR 2009, 12 May 2009 Current state

22. EICAR 2009, 12 May 2009 Current state

23. EICAR 2009, 12 May 2009 Current state

24. EICAR 2009, 12 May 2009 Difficulties Viewpoint of the average user Automatic methods Testing environment Funcionality problems –Truncate report file Stability problems

25. EICAR 2009, 12 May 2009 Questions