Process Coloring: an Information Flow-Preserving Approach to Malware Investigation Eugene Spafford, Dongyan Xu Department of Computer Science and Center for Education and Research in Information Assurance and Security (CERIAS) Purdue University Xuxian Jiang (Presenter) Department of Information and Software Engineering George Mason University NICIAR Site Visit, West Lafayette, IN, July 19, 2007

About myself  Ph.D. Student at Purdue 08/2001–08/2006  Ph.D. Advisor: Prof. Dongyan Xu  Thesis topic:  Virtualization-based malware investigation and defense  Assistant Professor at GMU 08/2006 – present  Research Focus  Stealthy malware detection and defense, especially rootkits and botnets

Outline  Process Coloring project at GMU  GMU Subcontract  Evaluation Facility  Progress Status  Other related projects  Transparent and Reliable VM Monitoring (OBSERV)

Process Coloring project  Task I: Color Diffusion Model (Month 1 ~ 6)  Task II: Process Coloring Prototype (Month 2 ~ 18)  Task II.1: Xen-based log coloring and collection  Task II.2: Coloring-based tools for server-side malware investigation  Task II.3: Coloring-based tools for client-side malware investigation  Task III: Color Mixing Handling (Month 7 ~ 18)  Task III.1: Legitimate color diffusion identification  Task III.2: Information flow insulation  Task III.3: Information flow border control 10% 25% 20% 15% 25% GMU

Process Coloring project -- Evaluation Facility Malware Trap Malware Playground vGround Playground Collapsar Honeyfarm InvestigationCapture Front-end Back-end  An Integrated Malware Research Framework Collapsar: Security’04, NDSS’06, JPDC’06 vGround: RAID’05, RAID’07

Existing Approach: Honeypot Domain B Domain A Domain C Internet  Two Weaknesses  Manageability vs. Detection Coverage  Security Risks  On-Site Attack Occurrences

Collapsar Honeyfarm Domain B Domain A Domain C Front-End VM-based Honeypots Management Station Collapsar Center Correlation Engine Redirector Collapsar Honeyfarm Redirector Benefit 1: Centralized management of honeypots w/ distributed (virtual) presence Benefit 1: Centralized management of honeypots w/ distributed (virtual) presence Benefit 2: Off-site attack occurrences Benefit 2: Off-site attack occurrences Benefit 3: New possibilities for real-time attack correlation and log mining Benefit 3: New possibilities for real-time attack correlation and log mining

VM-based Honeypots Domain B Domain A Domain C Front-End Collapsar Center Redirector Collapsar as a Server-side Honeyfarm  Passive honeypots w/ vulnerable server-side software  Web servers (e.g., Apache, IIS, …)  Database servers (e.g., Oracle, MySQL, …) Blaster (2003)Sasser (2004)Zotob (2005)

Malicious Web Server VM-based Honeypots Domain B Domain A Domain C Front-End Collapsar Center Redirector Collapsar as a Client-side Honeyfarm  Active honeypots w/ vulnerable client-side software  Web browsers (e.g., IE, Firefox, …)  clients (e.g., Outlook, …) [ HoneyMonkey, NDSS’06] PlanetLab (310 sites) 752 malicious URLs/ 288 malicious sites/2 zero-day exploits

10 URL-level Topology Graph for WinXP SP1 Un-patched: 688 URLs from 270 sites Topology Graph of Malicious URLs Site nodes URLs Content Provider Exploit Provider Redirecting URL Exploiting URL

Process Coloring project -- Evaluation Facility Malware Trap Malware Playground vGround Playground Collapsar Honeyfarm InvestigationCapture Front-end Back-end  An Integrated Malware Research Framework Collapsar: Security’04, NDSS’06, JPDC’06 vGround: RAID’05, RAID’07

vGround: A Virtualization-Based Malware Playground lafayette.ise.gmu.edu  High Fidelity  VM: Full-System Virtualization  Strict Confinement  VN: Layer-2 Network Virtualization  Easy Deployment  Locally deployable  Efficient Experiments  Images generation time: 60 seconds  Boot-strap time: 90 seconds  Tear-down time: 10 seconds Virtualization In “Fighting Computer Virus Attacks”, Peter Szor, USENIX Security Symp., 2004

Recent Progress We are here Identifying color diffusion operations in Linux OS Starting to implement log coloring and collection on Xen VMM Setting up the GMU subcontract

Outline  Process Coloring project at GMU  GMU Subcontract  Evaluation Facility  Progress Status  Other related projects  Transparent and Reliable VM Monitoring (OBSERV)

Why OBSERV?  Virtualization introduces strong mutual-isolation between processes “in the box” and “out of the box”  OBSERV: Out-of-Box with SEmantically Reconstructed View  Functions as a One-Way Mirror

OBSERV Application I: Reliable VM Monitoring “In the box” View OBSERV View

OBSERV Application II: Cross-View Malware Detection YYC Backdoor Hack Defender YYC Backdoor “In the box” View OBSERV View

OBSERV Application III: Detection & Prevention of Kernel Rootkits Adore Rootkit Adore_ng Rootkit Suckit Rootkit OBSERV View“In the box” View

Summary Domain B Domain A Domain C Front-End Redirector vGround II vGround I Collapsar Process Coloring Collapsar + vGround Unique virtualization-based malware research platform Collapsar + vGround Unique virtualization-based malware research platform Process Coloring Unique approach for malware investigation and defense Process Coloring Unique approach for malware investigation and defense

Thank you! For more information about the Process Coloring project: