Scalability, Fidelity, and Containment in the Potemkin Virtual Honeyfarm Michael Vrable, Justin Ma, Jay chen, David Moore, Erik Vandekieft, Alex C. Snoeren, Geoffrey M. Voelker, Stefan Savage Collaborative Center for Internet Epidemiology and Defenses Department of Computer Science and Engineering Univsersty of California, San Diego

Background Info Network Telescope Theory HoneyPots – A system of Intrusion/Threat Detection where the value lies in that all traffic in system is not legitimate High Interaction or Low Interaction? Benefit of Low Interaction is large number of IPs can be covered Benefit of Low Interaction is large number of IPs can be covered Benefit of High Interaction is you can gain better insight into the methods used and possible outcomes of attacks Benefit of High Interaction is you can gain better insight into the methods used and possible outcomes of attacks

Bottom Line You can have one a system that represents a larger net so you have better odds of finding something malicious Or, you can have a system that monitors a smaller set of IPs because there is more overhead in providing kernel and system access to the potential threat, and not just mimicking network presence.

Bottom Line ? So why cant you have your cake and eat it too? Is it possible to provide a system that will allow you to combine the best of both worlds. Can you provide a Honeyfarm solution that allows you monitor a large IP set, and provide a valid system for each threat to incubate so analysis can be in-depth? Can you do it with out throwing large amounts of money at it?

Basis of Paper This is the aim of this paper. Utilize VM technology and custom software design to create a system which has high fidelity, and can scale well to monitor a large environment if the need arises. Don’t break the bank doing it either!

Problems Resources Memory Memory CPU CPU HD Space HD SpaceRouting How do we route the packets so Honeyfarm is invisible? How do we route the packets so Honeyfarm is invisible? How do we route packets so as not to cause an outbound attack? How do we route packets so as not to cause an outbound attack?Latency How do we provide interaction so that the attacker does not know he is in a virtual environment? How do we provide interaction so that the attacker does not know he is in a virtual environment?

Solutions! Flash Cloning Allow Farm to scale as need arises Allow Farm to scale as need arises Delta Virtualization (Copy-On-Write) Addresses timing and resource use of each clone Addresses timing and resource use of each clone Creative Routing Limits farm to only dealing with IPs that solicit communication. Limits farm to only dealing with IPs that solicit communication.

Flash Cloning VM Machine instantiation can have high overhead and latency, especially when VM needs to boot and load devices. To work around this, provide a “Reference Image”. An Image of an already loaded O/S is kept frozen and unchanged. When need arises for a new VM, clone this one. It is already to run, just change IPs.

Flash Cloning Benefits Quicker Load time Quicker Load time New VMs can react to each new outside probe/threat New VMs can react to each new outside probe/threat Allows a pristine VM to be examined after compromise. You have a baseline to compare a compromised VM to. Allows a pristine VM to be examined after compromise. You have a baseline to compare a compromised VM to. Clone can be created and threat will only receive initial delay between first packet and response. Clone can be created and threat will only receive initial delay between first packet and response.

Flash Cloning Courtesy of the paper and its authors

Delta Virtualization Essentially an optimized Copy-on-Write technique. For each VM Cloned, the entire image need not be copied. There will always be static parts of the OS memory that does not change. If need for that specific VM to alter memory tables arise, then copy memory for that location and change memory table for VM to point to new location

Delta Virtualization Courtesy of the paper and its authors

Creative Routing Each Incoming Packet is Mirrored at Edge Router to HoneyFarm The farm has it’s own machine dedicated to routing packets. For each packet destined for an IP known to be unused, the gateway notifies Cloning Manager on least busy machine to allocate new clone with specific IP.

Creative Routing After initial lag from cloning, clone is ready and notifies Clone Manager. Clone Manager tells gateway which then flushes buffer of packets waiting for clone and adds routing rule to push all future communication for that IP address to that clone. To prevent horizontal port scans from overwhelming farm, all future unused attempts from that IP are ignored to keep clone numbers in check.

Here is where the creativity comes in What about threats that spread like worms? Viruses that call home? Rootkits that update themselves? Each communication between an outside IP and an Internal IP is considered a Universe and the route reflects it. If compromised clone attempts outside communication, the communication is reflected back toward another clone inside the farm.

Here is where the creativity comes in Thus, the farm can also serve as a ‘incubator’, providing a microcosm for the threat to grow. Also allows for the possibility of cross contamination. You could setup rules to allow to uniquely infected clones to communicate with each other and create hybrid compromises. Another unseen benefit is you can provide a concrete spread rate of a new threat. Thus, providing some reliable scale to rate new threats on.

The numbers don’t lie The largest HoneyFarm known to the authors was Symantec’s DeepSight using 40 servers with VMware to mimic 2000 IP addresses. During Potemkin’s ‘Live Deployment’, the max they were able to simulate was 2100 VMs using one gateway and 9 servers. All using 2.8 GHZ Xeons’s with 2GB of memory and a gigabit NIC. Roughly $10,000 total by current market value.

Performance Numbers Right hand side represents possible future enhancements by recycling data structures and tables of VMs that were tore down. Tables Courtesy of the Paper and it’s Authors.

Strengths Provides some real good ideas to maximize performance with limited hardware. Incubator idea is real interesting. Infection rate idea is real interesting. Considered legalities of HoneyFarm infecting external IPs and also considered Hybrid Infections.

Weaknesses Live testing did not last longer then 10 minutes. A lot of bugs still left to work out before the solution could be considered stable enough for long term deployment. System can be exploited by attacker to exhuast amount of resources in system. Time characteristics can be used against HoneyFarm to signal virtual environment.

Weaknesses Threat could be able to look at limited devices available and conclude in virtual environment. Threat could also reference outside IP to determine if in virtual environment. Could only be useful in examining malicious programs that are not designed to look for virtual environments, as an actual attacker worth their salt could determine it is virtual environment.

Extensions Elaborate on the idea of incubation more. Improve multiple OS support. Enable packet analysis at gateway to determine which OS to clone to provide ‘best fit’ for attack. Stabilize system and introduce VM HD support so each clone can get access to swap space.