Detection and Prevention of Buffer Overflow Exploit Cai Jun Anti-Virus Section Manager R&D Department Beijing Rising Tech. Corp. LTD.

Review of Buffer Overflow Exploit Time Virus Name Financial Loss 1989 Morris Worm $96,000, CodeRed (I/II) $2,600,000, SQL Slammer $1,200,000, Worm.Blaster$1,200,000, Worm.Sasser$500,000,000 ………

What is Buffer Overflow Exploit Definition of a Buffer Definition of a Buffer How Buffers Are Exploited How Buffers Are Exploited How to Exceed Program Space How to Exceed Program Space Overflow the Stack Overflow the Stack What Follows a Buffer Overflow What Follows a Buffer Overflow

An Example of Buffer Overflow

How to Detect and Prevent Buffer Overflow Exploit Static Detection Static Detection Compile Time Detection Compile Time Detection Network-based Detection Network-based Detection Host-based Detection Host-based Detection

Static Code Analysis (Part I) How it works? How it works? Source code level analysis

Static Code Analysis (Part II) Advantages Advantages Help to improve an application Disadvantages Disadvantages –Program analysis is inadequate –Modification and recompiling of source code are needed

Compile Time Detection (Part I) How it works? How it works? Stack-smashing protection

Compile Time Detection (Part II) Advantages Advantages Nearly 100% protection of “ simple function calls ” Disadvantages Disadvantages –Recompiling is needed –No sane way to protect “ complex function calls ”

Network based Detection (Part I) How it works? How it works? Analyze network data for attack code

Network-based Detection (Part II) Advantages Advantages Detect exploit code by rule Disadvantages Disadvantages Either high number of false positive alert or low number of true positive alert

Host-based Detection (Part I) How it works? How it works? Executable space protection –Hardware solution (CPU) –Software solution

NX Technology What is NX? What is NX? NX stands for ‘ No Execute ’ CPUs which support NX CPUs which support NX Sun's Sparc, Transmeta's Efficeon, newer 64-bit x86 processors: AMD64, IA-64, etc. OSs implement NX OSs implement NX Windows XP SP2, Windows Longhorn Linux with NX patch

Software Solution From Rising Tech. (Part I) Solution 1: TDI driver (only for Windows) How it works? How it works? use TDI driver to detect known buffer overflow exploit

Software Solution From Rising Tech. (Part II) Solution 1:TDI driver Advantages Advantages Detect viruses which exploit known vulnerabilities Disadvantages Disadvantages Fail to protect unknown vulnerabilities

Software Solution From Rising Tech. (Part III) Solution 2: StackChecker (Only for Windows) How it works? How it works? Install kernel driver to inspect system calls and detect invalid user calls from stack or heap

Software Solution From Rising Tech. (Part IV)

Solution 2: StackChecker Advantages Advantages Detect viruses which exploit buffer overflow Disadvantages Disadvantages Victim program will eventually crash despite of the warning

Summary (Part I) If you are a programmer Check your source code manually Check your source code manually Use aid tools to find hidden bugs Use aid tools to find hidden bugs Compile with StackGuard or other tools to avoid buffer overflow Compile with StackGuard or other tools to avoid buffer overflow

Summary (Part II) If you are a network administrator Apply NIDS product Apply NIDS product Update it promptly Update it promptly If you are a user Apply latest updates of your operate system Apply latest updates of your operate system Try StackChecker to detect real-time buffer overflow exploit Try StackChecker to detect real-time buffer overflow exploit

The End