November 19, 2008 CSC 682 Use of Virtualization to Thwart Malware Written by: Ryan Lehan Presented by: Ryan Lehan Directed By: Ryan Lehan Produced By: Ryan Lehan Starring: Ryan Lehan

Introduction Malware Malware McAfee Avert Labs McAfee Avert Labs Prediction of nearly 800,00 security threats for the year 2008 Prediction of nearly 800,00 security threats for the year % growth rate over % growth rate over % fall in 3 categories 99% fall in 3 categories Identity Theft Identity Theft Data Theft Data Theft System Compromise System Compromise

Attack Technique Attack OS directly Attack OS directly Direct access to OS via OS Application Programming Interface (API) Direct access to OS via OS Application Programming Interface (API) Attach other applications Attach other applications Exploiting vulnerability points Exploiting vulnerability points Directly via the application’s API Directly via the application’s API Indirectly via specifically formed files Indirectly via specifically formed files MS 2008 Security Intelligence Report MS 2008 Security Intelligence Report 3 rd party applications are killing their security 3 rd party applications are killing their security Due to the openness of the OS Due to the openness of the OS

Modern Operating System

Problems Thwarting Malware Expensive Expensive Time Time Money Money Resources Resources Thousands of applications each with the possibility of tens or even hundreds of vulnerability points Thousands of applications each with the possibility of tens or even hundreds of vulnerability points Not a single defense Not a single defense Ant-Virus, Spyware, Adware, Firewall Ant-Virus, Spyware, Adware, Firewall Good tactics require above average computer skills Good tactics require above average computer skills Non-Intuitive Non-Intuitive

Current Malware Defensive Techniques Signature Base Signature Base Malicious code recognition by patterns in code (signatures) Malicious code recognition by patterns in code (signatures) Signatures created by security vendor and then downloaded to computer user Signatures created by security vendor and then downloaded to computer user Problem area Problem area Obfuscation – cryptographic technique to masquerade a random code signature Obfuscation – cryptographic technique to masquerade a random code signature Zero Day Attack – window of time from when a vulnerability exists to the time when the security vendors release a patch. Zero Day Attack – window of time from when a vulnerability exists to the time when the security vendors release a patch.

Current Malware Defensive Techniques (cont.) Behavior Blocking Behavior Blocking Malicious code recognition based upon user configurable policies Malicious code recognition based upon user configurable policies Monitors the code as it runs in real time Monitors the code as it runs in real time Code attempts a function that violates a predefined policy then action is taken Code attempts a function that violates a predefined policy then action is taken Can thwart zero day attack Can thwart zero day attack Problem area Problem area Policies too tight can cause high false positives Policies too tight can cause high false positives Policies too loose can allow malicious code to run Policies too loose can allow malicious code to run

Current Malware Defensive Techniques (cont.) Virtual Machines Virtual Machines Isolates guests operating systems from host operating system Isolates guests operating systems from host operating system Allows user to run within a clean environment Allows user to run within a clean environment Contains malicious code to guest environment only Contains malicious code to guest environment only Problem area Problem area Requires above average computer skills Requires above average computer skills Does not recognize malicious code Does not recognize malicious code Malicious code can still run within the guest environment Malicious code can still run within the guest environment

Virtualization Definition: technique of isolation systems, applications, or end users from the physical characteristics of computer resources Definition: technique of isolation systems, applications, or end users from the physical characteristics of computer resources Isolation Isolation Fundamental concept Fundamental concept Process Isolation Process Isolation Data Isolation Data Isolation Virtualized environment should guarantee that any action performed inside the virtual environment cannot interfere outside that environment Virtualized environment should guarantee that any action performed inside the virtual environment cannot interfere outside that environment Break-In: situation when an external process enters into the same environment as another process Break-In: situation when an external process enters into the same environment as another process Break-Out: situation when an internal process escapes from its confined environment Break-Out: situation when an internal process escapes from its confined environment

Virtualization (cont.) Shared Resources Shared Resources Just like operating systems Just like operating systems Each isolated environment views the shared resource as an object for its sole use Each isolated environment views the shared resource as an object for its sole use Data storage example Data storage example Single physical resource appear as multiple logical resources Single physical resource appear as multiple logical resources Multiple physical resources appear as a single logical resource Multiple physical resources appear as a single logical resource

Current Virtualization Techniques Virtual Machines and Emulators Virtual Machines and Emulators Software that emulates a physical computer Software that emulates a physical computer CPU, Hard Disk, Video, Network card, Memory CPU, Hard Disk, Video, Network card, Memory Run modified and unmodified guest operating systems Run modified and unmodified guest operating systems Guest OS does not know that it is running within a host OS Guest OS does not know that it is running within a host OS Good for isolating host OS Good for isolating host OS Requires above average computer skills Requires above average computer skills

Virtual Machine and Emulator

Current Virtualization Techniques Language Dependent Virtual Environments Language Dependent Virtual Environments Some computer languages are designed to run only within a virtual environment (sandbox) Some computer languages are designed to run only within a virtual environment (sandbox) Java Java Does not emulate hardware but creates a set of APIs from which the application interfaces with Does not emulate hardware but creates a set of APIs from which the application interfaces with Security advantages over complete virtual machines, but only for that specific computer language Security advantages over complete virtual machines, but only for that specific computer language If a vulnerability exists, patch the environment not the applications If a vulnerability exists, patch the environment not the applications One area not thousands One area not thousands Only works with specific computer languages Only works with specific computer languages

Current Virtualization Techniques Application Packaging Application Packaging Builds upon the use of Virtual Machines Builds upon the use of Virtual Machines Applications are pre-built into a ready made virtual environment Applications are pre-built into a ready made virtual environment If package becomes infected, just re-download it If package becomes infected, just re-download it Does not prevent the user from installing other software Does not prevent the user from installing other software

Current Virtualization Techniques Virtual Memory Virtual Memory Used by modern operating systems Used by modern operating systems Gives an application the impression that it contiguous working memory all to itself Gives an application the impression that it contiguous working memory all to itself Not designed to thwart malicious code Not designed to thwart malicious code

What We Need Strength and security of isolation Strength and security of isolation Seamless operation for all levels of computer skills Seamless operation for all levels of computer skills Intuitive Intuitive Anti-Virus vs Spyware vs Adware Anti-Virus vs Spyware vs Adware Single area for defense Single area for defense Computer user Computer user Vendor maintenance Vendor maintenance

Virtualization Technique to Thwart Malware Light-weight Virtual Environment Light-weight Virtual Environment Process and Data isolation happens at the application level Process and Data isolation happens at the application level No guest OS is needed No guest OS is needed Malicious code runs isolated from other applications and OS Malicious code runs isolated from other applications and OS Seamless operation for the user Seamless operation for the user Pure isolation can be counter productive Pure isolation can be counter productive Provides an API or a secure communication channel to the OS or other applications Provides an API or a secure communication channel to the OS or other applications

Light-weight Virtual Environment

Virtualization Technique to Thwart Malware (cont.) Layered Security Layered Security Policy based, similar to Behavior Blocking. Policy based, similar to Behavior Blocking. Allows for vendor and user configurations Allows for vendor and user configurations Layered Layered To combat the attack, not just recognize To combat the attack, not just recognize To reduce code complexity To reduce code complexity Separation of duty Separation of duty 3 Layers 3 Layers Process Level Security Policies Process Level Security Policies Dictate level of isolation including Trusted and Stateless Dictate level of isolation including Trusted and Stateless Inter Process Communication Security Policies Inter Process Communication Security Policies Dictate if and how applications communicate with each other Dictate if and how applications communicate with each other Auto Configurable Auto Configurable OS API Security Policies OS API Security Policies Dictate if and how application communicate with the OS Dictate if and how application communicate with the OS Auto Configurable Auto Configurable

Layered Security

Working in Tandem Identity Theft Identity Theft To thwart phishing attacks, many techniques rely on a trusted 3 rd party To thwart phishing attacks, many techniques rely on a trusted 3 rd party 3 rd Party applications will be isolated and can be marked as Trusted 3 rd Party applications will be isolated and can be marked as Trusted Insures the safety of the trusted application as well as enhance the security of applications that use it Insures the safety of the trusted application as well as enhance the security of applications that use it

Working in Tandem Data Theft Data Theft Data is isolated Data is isolated Malicious code will not have access to other applications’ data Malicious code will not have access to other applications’ data Access to other data areas will need to pass through the security policies Access to other data areas will need to pass through the security policies

Working in Tandem System Compromise System Compromise Process is isolated Process is isolated Malicious code will have a difficult time infecting other applications Malicious code will have a difficult time infecting other applications Removal of direct communication between processes and OS Removal of direct communication between processes and OS If an application is exploited, that application itself is contain within an isolated environment If an application is exploited, that application itself is contain within an isolated environment

Dealing with Vulnerabilities Fix the environment Fix the environment No need to fix thousands of applications, just the environment (sandbox) No need to fix thousands of applications, just the environment (sandbox) Language Dependent Virtual Environment (Java) Language Dependent Virtual Environment (Java) Focused Attention Focused Attention Only 4 areas that need to be looked at Only 4 areas that need to be looked at Security policies Security policies Configurable Configurable Virtualization Layer Virtualization Layer OS API OS API OS itself OS itself

Conclusion Currently, many tools and techniques for combating malware but they are lacking in one form or another Currently, many tools and techniques for combating malware but they are lacking in one form or another Virtualization is a proven method for strong process and data isolation Virtualization is a proven method for strong process and data isolation Combined with layered security can defeat many forms of malware Combined with layered security can defeat many forms of malware Many benefits for both users and vendors alike Many benefits for both users and vendors alike