Boxuan Gu, Xiaole Bai, Zhimin Yang,Xiaole BaiZhimin Yang Adam C. ChampionAdam C. Champion, Dong XuanDong Xuan Dept. of Computer Science and Engineering The Ohio State University Columbus, OH, USA IEEE Infocom Boxuan Gu, Xiaole Bai, Zhimin Yang,Xiaole BaiZhimin Yang Adam C. ChampionAdam C. Champion, Dong XuanDong Xuan Dept. of Computer Science and Engineering The Ohio State University Columbus, OH, USA IEEE Infocom A Presentation at Advanced Defense Lab

Outline Introduction Related Work System Design Implementation Experiments Conclusion Advanced Defense Lab2

Introduction - Motivation According to the US-CERT database, buffer overflows are one of the most critical and common software vulnerabilities.US-CERT Difference between DDC-Based and DBC-Based. In this paper, we remove the limitation of DBC-Based approaches. Advanced Defense Lab3

Introduction - Contributions Take snapshots of the target process’s virtual memory immediately before input data are consumed. Use these snapshots to instantiate a runtime environment that emulates the target process’s input data consumption. Advanced Defense Lab4

Introduction - Contributions Implement a prototype system in Debian Linux with Kernel version Conduct extensive experiments based on real traces and thousands of malicious shellcode samples. Advanced Defense Lab5

Outline Introduction Related Work System Design Implementation Experiments Conclusion Advanced Defense Lab6

DBC-Based – Static Analysis Advanced Defense Lab7

DBC-Based – Static Analysis T. Toth and C. Kruegel, “Accurate Buffer Overflow Detection via Abstract Payload Execution” (NOP Sled) Focusing on code-level patterns has limitations regarding accuracy and completeness. U. Bayer, A. Moser, C. Kruegel, and E. Kirda, “Dynamic Analysis of Malicious Code” Advanced Defense Lab8

DBC-Based – Dynamic Analysis M. Polychronakis, K. G. Anagnostakis, and E. P. Markatos, “Network-level Polymorphic Shellcode Detection Using Emulation” “Emulation-Based Detection of Non-Self-Contained Polymorphic Shellcode” They’re still prone to evasion due to insufficient context information about the target process. Advanced Defense Lab9

Generated Block N … … Generated Block 2 Generated Block 1 DBC-Based – Dynamic Analysis Advanced Defense Lab10 Decryptor for block 1 Decryptor for block 2 … … Decryptor for block N Bootstrap Code espeip Launch

Examples to by pass detection Advanced Defense Lab11

DDC-Based Detection is conducted while processes consume input data. Address Space Randomization (ASR) Data Space Randomization (DSR) Troubleshooting is inefficient. Advanced Defense Lab12

Outline Introduction Related Work System Design Implementation Experiments Conclusion Advanced Defense Lab13

System Design - Rationale Advanced Defense Lab14

System Design - Rationale Instantiating Virtual Execution Environment Snapshots are used to initialize this environment and provide two benefits. For observing the input data’s real behaviors. mimic the process’s consumption of input data. For constructing a lightweight virtual environment. Advanced Defense Lab15

System Design - Rationale Facilitating System Call-based Detection No matter how well malicious shellcode disguises itself, it will eventually use system calls to launch attacks. Existing DBC-Based approaches, including both static and dynamic analysis, cannot use system call invocations as detection criteria as they lack such necessary register information. Advanced Defense Lab16

System Design - Architecture Advanced Defense Lab17

System Design - Architecture Advanced Defense Lab18

System Design - Workflow Advanced Defense Lab19

System Design - Workflow Advanced Defense Lab20

Generated Blocks Generated Blocks Decryptors By passed... lidt eax jz +3 Cmp ebx,0x252d252d Mov ebx,0xa0ef(ebp) Erase eax Pop esp System Design - Detection Advanced Defense Lab21 Pop eax Calculate… Push eax Pop eax Calculate… Push eax Push esp lidt eax jz +3 Cmp ebx,0x252d252d Mov ebx,0xa0ef(ebp) Erase eax Pop esp Pop eax Calculate… Push eax Pop eax Calculate… Push eax Push esp With snapshotsWithout snapshots ebx=0x252d252d Context Information Provieded ebx=0x252d252d Context Information Provieded ebx=??? Context Information Unavailable ebx=??? Context Information Unavailable GG…

Outline Introduction Related Work System Design Implementation Experiments Conclusion Advanced Defense Lab22

Implementation Debian Linux with kernel version Emulator can interpret all IA-32 instructions General-purpose 、 FPU instructions System instructions System call instructions (int 0x80 、 sysenter) When an unimplemented instruction is encountered in emulation Skip if it’s not a privileged instruction Stop otherwise Advanced Defense Lab23

Implementation Suitable Threshold Lower Bound : Set exe_depth to 14 Upper Bound : Set exe_depth to 7000 Use an open-source x86 disassembler library to construct the decoder ( ) Integrated into glibc, override the read 、 recv functions… Advanced Defense Lab24

Outline Introduction Related Work System Design Implementation Experiments Conclusion Advanced Defense Lab25

Experiments - Effectiveness Collect 51 unencrypted malicious shellcodes from the Internet that target Linux Systems. Using encryption tools to generate 5000 encrypted malicious shellcodes. ADMMutate: Polymorphic Shellcode Engine Metasploit: TAPiON: Advanced Defense Lab26

Experiments - Effectiveness Enlist 4 volunteers who collect HTTP messages for 6 weeks by using Fiddler ( Advanced Defense Lab27

Experiments - False Pos & Neg Advanced Defense Lab28 False Negative is ZERO.

Experiments - Overhead Dell Dimension 5150 with an Intel Pentium GHz and 1 GB RAM. Advanced Defense Lab29

Experiments - Overhead Hardware Server: Dell Dimension 5150 with an Intel Pentium GHz and 1 GB RAM Client: IBM ThinkPad T60 and 1 GB RAM. Connection: 100 Mbps Ethernet switch Software Server: thttpd (ACME Laboratories)ACME Laboratories Client: Jef Poskanzer’s HTTP Load ProgramJef Poskanzer’s HTTP Load Program Advanced Defense Lab30

Experiments - Overhead Advanced Defense Lab31

Outline Introduction Related Work System Design Implementation Experiments Conclusion Advanced Defense Lab32

Conclusion Drawback Return-into-libc without function calls may still evade this system. Feature work Designing a faster instruction decoder. Using static analysis tech. to analyze input data before or during emulation. The virtual memory snapshot is very useful for analysis. Advanced Defense Lab33