SCP: A System Call Protector against Buffer Overflow Attacks dove 邱秉誠

Outline Introduction Attacking Method Related Work SCP System Design Experimental Result Conclusion Future Work

Introduction (1) Buffer Overflow Attack Easily launched Huge amount of targets Strongly damage The most dangerous threat in the internet

Introduction (2) Many countermeasures were published But also were defeated Developing an efficient and effective approach becomes a critical and emergent issue

Attacking Method (1) Stack Overflow Attack Overflow the return address to launch injected code (shell code)

Attacking Method (2) Return-into-libc Attack Overwrite the return address to execute system() function call High Address Pointer to system()’s arg “/bin/sh” Fake RA ESP system() EIP (RA) EBP buffer Low Address AAA… address to system() system()’s return address pointer to system()’s arg String format:

Attacking Method (3) Heap Overflow Attack Similar to stack overflow, but on the heap area

Attacking Method (4) Scanning Code Attack After overflow, scan the process image to find patterns and then jump into it Usually used to bypass some protections

Related Work (1) Some Countermeasures StackGuard / StackShield Address Obfuscation Exec Shield Binary Obfuscation PointGuard™ Instruction Set Randomization RAD

Related Work (2) StackGuard -- add canary word before return address StackShield --copy return address to confirm when return Bypassing StackGuard and StackShield return address canary word Buffer High Address Low Address … Saved return address Low Address

Related Work (3) Address Obfuscation PaX ASLR project ASLP - Randomize the base address of memory regions -- Randomize the base address of stack/heap -- Randomize the starting address of dynamic-linked libraries -- Randomize the locations of routines and static data ASLP Internal fragmentation problem Derandomization Attack

Related Work (4) Bound Checking Bound Checking for C Require source code / recompile Runtime overhead are huge - 4x / 5x when best case - 10x general case - 100x worst case

Related Work (5) - ELF file format Exec Shield Data/Stack section non-executable Code section non-writable Compatibility problem - ELF file format -- Add PT_GNU_STACK and PT_GNU_HEAP - Nested function - Recompile / porting - sigreturn() system call Return into libc attack can be launched

Related Work (6) Binary Obfuscation Change ELF layout (add new section) Add new system call to notify the return address Insert junk code in binary object Attacker may use the new system call Return into libc attack / scanning code attack can be launched

Related Work (7) PointGuard™ Encrypt pointer, decrypt when reference object

Related Work (8) Instruction Set Randomization Hardware solution, encrypt / decrypt CPU instructions porting binaries

Related Work (9) RAD Return Address Defender Compiler solution Push return address to RAR when prologue Pop return address from RAR when eprologue Need recompile

SCP System Design (1) Principles Goal Prevent attacker from executing int 80 offered by attacker Prevent attacker from executing int 80 existed in system Goal Low overhead Efficient to protect Do not require source code Compatibility Use less system resource Easy to maintain

SCP System Design (2) Assumptions Malicious code have to use system call to damage system vulnerable program is dynamic linked

SCP System Design (3) System overview protect int 0x80’s return address Use secure enter kernel instead of all sysenter (int $0x80) movl sys#, %eax movl arg1, %ebx … int $0x80

SCP System Design (4) The system call path

( libc wrapper routine ) SCP System Design (5) The system call path High Address ( kernel ) sys_open() Kernel Space 5. return 4. trap into kernel ( libc wrapper routine ) __libc_open() User Space 3. call wrapper routine ( GOT entry ) Address of __libc_open 2. lookup GOT 6. return ( PLT ) jmp *GOT[__libc_open] 1. go to PLT ( user program ) open(); Low Address

(a) Secure Enter Kernel SCP System Design (6) Pseudo Code (a) Secure Enter Kernel (b) Trap Code save_all_registers; page = 0; size = 0; if ( page == 0 ) { page = mmap2(); size = copy_trap_code(page); notify_kernel(size+6); } restore_all_registers; call page; restore_sys#_in_eax; int $0x80; (sysenter) return_to_libc; machine code: \x8B\x44\x24\x04 \xCD\x80 \x83\xC4\x08 \xC3

( libc wrapper routine ) SCP System Design (7) New system call path (with SCP system) High Address ( kernel ) sys_open() 6. return sys_read() Kernel Space 5. trap into kernel ( trap page ) int $0x80 4. call trap page ( libc wrapper routine ) __libc_open() User Space __libc_read() 7. return 3. call wrapper routine ( GOT entry ) Address of __libc_open Address of __libc_read 2. lookup GOT 8. return ( PLT ) jmp *GOT[__libc_open] jmp *GOT[__libc_read] 1. go to PLT ( user program ) open(); read(); Low Address

SCP System Design (8) Example with SCP system (lazy binding) Glibc: 4. Call printf() Glibc: printf() here 6. Normal system call 5. Notify kernel the RA 3. Load libc.so.6 kernel Loader (ld-linux.so.2) 1. Notify kernel the RA Inject code 2. Loading program program

SCP System Design (9) Example with SCP system (lazy binding) Glibc: 4. Call printf() Glibc: printf() here 6. Normal system call 5. Notify kernel the RA 3. Load libc.so.6 kernel Loader (ld-linux.so.2) 1. Notify kernel the RA Inject code 2. Loading program program

SCP System Design (10) Modify kernel Add new system call notify_kernel() - It can only be called 2 times per process Add new structures in task_struct to restore addresses - loader_return_address - syscall_return_address do_fork() - Addresses copied from parent process sys_execve() - loader_return_address = 0 - syscall_return_address = 0

SCP System Design (11) Introduce fake pages

SCP System Design (12) Attack analysis Attacker can launch scanning code attack to trace real int $0x80 Possible solution: - Non-readable but executable code segment - … future work (user program) system_call (PLT) (GOT) (libc) wrapper routine (heap) int $0x80

SCP System Design (13) SCP system analysis Allow executable stack - General debugger support - Nested function - Do not require ELF modified - Do not require to recompile Allow execute programs without ASLR - No internal fragment problem - Process crashes are decided by process owners - Easy to maintain

Experimental Result (1) Efficiency test Buffer overflow attack with injected code

Experimental Result (2) Efficiency test Calling notify_kernel test

Experimental Result (3) Micro test 10,000,000 times per system call System Call Original libc & Kernel (usec) Secure libc & Secure Kernel Increment mmap 4.83598861 5.04285570 4.28 % open 5.70100183 12.31045995 115.93 % read 4.44757121 4.65731530 4.72 % write 28.61905470 28.86815789 0.87 %

Experimental Result (4) Original libc & kernel execve("./micro-test.open", ["./micro-test.open"], [/* 29 vars */]) = 0 % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 49.40 828.885786 41 20000000 gettimeofday 25.81 432.1044308 43 10000003 1 open 24.59 411.1630811 41 10000002 close 0.04 0.700248 700248 1 brk 0.04 0.700211 700211 1 mprotect 0.04 0.700202 700202 1 read 0.02 0.402909 201455 2 write 0.02 0.400418 200209 2 munmap 0.01 0.201221 33537 6 old_mmap 0.01 0.100622 33541 3 fstat64 100.00 1677.766736 40000021 1 total

avg timediff = 0.0000073216265842318534575253186069687672 sec = 7.32162658 usec Command exited with non-zero status 81 Command being timed: "./micro-test.open" User time (seconds): 0.69 System time (seconds): 122.31 Percent of CPU this job got: 100% Elapsed (wall clock) time (h:mm:ss or m:ss): 2:03.00 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 0 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 82 Minor (reclaiming a frame) page faults: 7 Voluntary context switches: 0 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 81

Experimental Result (5) Modified libc & kernel execve("./micro-test.open", ["./micro-test.open"], [/* 29 vars */]) = 0 % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 47.44 821.1515639 41 20000000 gettimeofday 28.54 494.834607 49 10000003 1 open 23.82 412.936479 41 10000002 close 0.04 0.700382 700382 1 read 0.04 0.700380 700380 1 brk 0.02 0.403299 201650 2 write 0.02 0.400813 200407 2 mmap2 0.02 0.400782 200391 2 munmap 0.02 0.400761 200381 2 utimes 0.01 0.202308 33718 6 old_mmap 0.01 0.101257 33752 3 mprotect 0.01 0.101186 33729 3 fstat64 100.00 1733.697893 40000027 1 total

avg timediff = 0.0000123534624044317750067014868853298992 sec = 12.35346240 usec Command exited with non-zero status 82 Command being timed: "./micro-test.open" User time (seconds): 0.68 System time (seconds): 173.26 Percent of CPU this job got: 99% Elapsed (wall clock) time (h:mm:ss or m:ss): 2:53.94 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 0 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 104 Minor (reclaiming a frame) page faults: 13 Voluntary context switches: 0 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 82

Experimental Result (5) Macro test 100,000 times per command Command Original libc & Kernel (sec) Secure libc & Secure Kernel Increment ls 0.00781023 0.00922095 18.06 % make 0.01481522 0.01697969 14.61 % sysctl 0.02905236 0.03447007 18.65 % tar 0.00804451 0.00940219 16.88 % gcc 0.98855523 1.00293709 01.45 %

Conclusion We propose a new method to protect system calls by registering valid int 80 on premise Without recompile Allow executable stack Use ASLR optionally Compatible with other protections

Future Work More secure Implement “executable but non-readable” region in segment section on i386 The NX Bit chip AMD 64 CPU

Thanks Q & A