SCP: A System Call Protector against Buffer Overflow Attacks 先進防禦實驗室 國立中央大學 資訊工程系

2 Outline  Introduction  Attacking Method  Related Work  SCP System Design  Experimental Result  Conclusion  Future Work

3 Introduction  Buffer Overflow Attack Easily launched Huge amount of targets Strongly damage One of the most dangerous threats in the Internet  Developing an efficient and effective approach becomes a critical and emergent issue.

4 Difficulty in Protection  Many countermeasures were proposed, but were defeated by new mutants.

5 Function and Stack Frame b return address add_g address of G’s frame point C[0] H’s stack frame G(int a) { H(3); add_g: } H( int b) { char c[100]; int i; while((c[i++]=getch())!=EOF) { } } C[99] Input String: xyz ZYXZYX G’s stack frame 0xabc 0xaba 0xabb

6 Stack Smashing Attack – Principle  Stack Smashing Attack Overflow a return address to transfer program execution flow into injected code (shell code)

7 Stack Smashing Attack - Example b return address add_g address of G’s frame point C[0] H’s stack frame addrress oxabc G(int a) { H(3); add_g: } H( int b) { char c[100]; int i; while((c[i++]=getch())!=EOF) { } } C[99] Injected Code 0xabc Attack String: xxInjected Codexy0xabc Length=108 bytes 0xaba 0xabb xxxx x y

8 Return-into-libc Attack  Overwrite the return address to execute a library function, e.g. system (), inside the attacked process. buffer EBP EIP (RA) Fake RA Pointer to system()’s arg system() ESP String format: AAA…address to system() system()’s return address pointer to system()’s arg “/bin/sh” High Address Low Address

9 Heap Overflow Attack  Similar to stack overflow, but on the heap area

10 Other Mutants  function pointer attack  Jump table attack  setjmp/longjmp attack

11 Related Work  Some Countermeasures StackGuard / StackShield Address Obfuscation (ASLR/PaX) Exec Shield Binary Obfuscation PointGuard ™ Instruction Set Randomization RAD

12 Related Work - StackGuard  StackGuard -- add canary word before return address  Bypassing StackGuard return address canary word Buffer High Address Low Address … return address Buffer … Saved return address … High Address Low Address

13 Related Work – ASLR/PaX  Address Obfuscation PaX/ASLR project - Randomize the base address of memory regions -- Randomize the base address of stack/heap -- Randomize the starting address of dynamic- linked libraries -- Randomize the locations of routines and static data Internal fragmentation problem Crash problem Derandomization Attack Local attack Non-relocatable code

14 Related Work – Bound Check  Bound Check Bound Check for C Program Require source code / recompile Runtime overhead are huge - 4x / 5x when best case - 10x general case - 100x worst case

15 Related Work – Exec Shield  Exec Shield Data/Stack section non-executable Code section non-writable Compatibility problem - ELF file format -- Add PT_GNU_STACK and PT_GNU_HEAP - Nested function - Recompile / porting - sigreturn() system call Return into libc attack can be launched

16 Related Work - PointGuard  PointGuard ™ Encrypt pointer, decrypt when reference object

17 Related Work - ISR  Instruction Set Randomization Hardware solution, encrypt / decrypt CPU instructions  porting binaries

18 Related Work - RAD  RAD Return Address Defender Compiler solution Push return address to RAR when prologue Pop return address from RAR when eprologue  Need recompile

19 SCP System Principle and Goal  Principle Prevent attackers from executing int 80 offered by them Prevent attackers from executing int 80 existed in the attacked system  Goal Low overhead Efficient to protect Do not require source code Compatibility Use less system resource (memory) Easy to maintain

20 Assumption  Assumption Malicious code have to use system calls to damage an attacked system Vulnerable programs use dynamic linked libraries

21 System Call Invocation Convention movl system_call_number, %eax movl first_argument, %ebx … int $0x80

22 System Call Path with/without SCP

23 Detailed System Call Path without SCP 6. return 5. return User Space Kernel Space ( user program ) open(); ( libc wrapper routine ) __libc_open() ( kernel ) sys_open() High Address Low Address 1. go to PLT 4. trap into kernel ( PLT ) jmp *GOT[__libc_open] ( GOT entry ) Address of __libc_open 2. lookup GOT 3. call wrapper routine

24 Secure Enter Kernel  Pseudo Code (b) Trap Code(a) Secure Enter Kernel save_all_registers; page = 0; size = 0; if ( page == 0 ) { page = mmap2(); size = copy_trap_code(page); notify_kernel(page+6); } restore_all_registers; call page; int $0x80; (sysenter) return_to_libc; machine code: \x8B\x44\x24\x04 \xCD\x80 \x83\xC4\x08 \xC3

25 Replacement Use secure enter kernel to replace ALL sysenter (int $0x80)

26 Detailed System Call Path with SCP  New system call path (with SCP system) 8. return 7. return User Space Kernel Space ( user program ) open(); ( libc wrapper routine ) __libc_open() ( kernel ) sys_open() 1. go to PLT 5. trap into kernel ( PLT ) jmp *GOT[__libc_open] ( GOT entry ) Address of __libc_open 2. lookup GOT 3. call wrapper routine ( trap page ) int $0x80 4. call trap page 6. return High Address Low Address read(); jmp *GOT[__libc_read] Address of __libc_read __libc_read() sys_read()

27 Lazy Binding under SCP loader (ld-linux.so.2) kernel program glibc: printf() here 2. Loading program 1. Notify kernel the RA Inject code 3. Load libc.so.6 6. Normal system call 4. Call printf() 5. Notify kernel the RA

28 Lazy Binding under SCP loader (ld-linux.so.2) kernel program glibc: printf() here 2. Loading program 1. Notify kernel the RA Inject code 3. Load libc.so.6 6. Normal system call 4. Call printf() 5. Notify kernel the RA

29 SCP Protection Mechanism – Limit the number of SCP Protection Mechanism – Limit the number of notify_kernel()  Avoid attacker ’ using to change the address of the only legal int 80.  Limit the number of notify_kernel() to two: loader (ld-linux.so.2) and glib (libc.so.6) have their own global variable segment; hence, there are two global variable pages in a process.  Each page variable will cause the system to create a new trap page.

30 SCP Protection Mechanism - Fake Trap Page  Insert fake trap pages around to real trap page to avoid attackers ’ detection of the real int 80 instruction.

31 Attack Analysis Attacker can scan the attacked process ’ s address space to find the real int $0x80 Possible solution: - Non-readable but executable code segment - … future work (user program) system_call (PLT)(GOT) (libc) wrapper routine (heap) int $0x80

32 Efficiency Test – Injected Code Buffer overflow attack with injected code

33 Efficiency Test – Change Legal int 80 Address Calling notify_kernel test

34 Performance Test – Micro Benchmark 10,000,000 times per system call System Call Original libc & Kernel ( μ sec) Secure libc & Secure Kernel ( μ sec) Increment mmap % open % read % write %

35 Number of Page Faults under Original libc and Kernel avg timediff = sec = usec Command exited with non-zero status 81 Command being timed: "./micro-test.open" User time (seconds): 0.69 System time (seconds): Percent of CPU this job got: 100% Elapsed (wall clock) time (h:mm:ss or m:ss): 2:03.00 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 0 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 82 Minor (reclaiming a frame) page faults: 7 Voluntary context switches: 0 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 81

36 Number of Page Faults under SCP libc and Kernel avg timediff = sec = usec Command exited with non-zero status 82 Command being timed: "./micro-test.open" User time (seconds): 0.68 System time (seconds): Percent of CPU this job got: 99% Elapsed (wall clock) time (h:mm:ss or m:ss): 2:53.94 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 0 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 104 Minor (reclaiming a frame) page faults: 13 Voluntary context switches: 0 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 82

37 Performance Test – Micro Benchmark 100,000 times per command Command Original libc & Kernel ( μ sec) Secure libc & Secure Kernel ( μ sec) Increment ls % make % sysctl % tar % gcc %

38 Conclusion  We propose a new method to protect system calls by registering valid int 80 on premise Low performance overhead Protect programs against all known code injection style BOAs No modification to source code or executable files is required Compatible with existing systems and applications Not increase the workload of program maintenance Terminate attacked processes normally Effective in defending local BO attacks

39 Future Work  More secure Implement “ executable but non-readable ” region in segment section on i386 The NX Bit chip AMD 64 CPU

40  Thanks  Q & A