1. SCP: A System Call Protector against Buffer Overflow Attacks
dove
邱秉誠

2. Outline Introduction Attacking Method Related Work SCP System Design
Experimental Result
Conclusion
Future Work

3. Introduction (1) Buffer Overflow Attack Easily launched
Huge amount of targets
Strongly damage
The most dangerous threat in the internet

4. Introduction (2)
Many countermeasures were published But also were defeated
Developing an efficient and effective approach becomes a critical and emergent issue

5. Attacking Method (1) Stack Overflow Attack
Overflow the return address to launch injected code (shell code)

6. Attacking Method (2) Return-into-libc Attack
Overwrite the return address to execute system() function call
High Address
Pointer to system()’s arg
“/bin/sh”
Fake RA
ESP
system()
EIP (RA)
EBP
buffer
Low Address
AAA…
address to system()
system()’s return address
pointer to system()’s arg
String format:

7. Attacking Method (3) Heap Overflow Attack
Similar to stack overflow, but on the heap area

8. Attacking Method (4) Scanning Code Attack
After overflow, scan the process image to find
patterns and then jump into it
Usually used to bypass some protections

9. Related Work (1) Some Countermeasures StackGuard / StackShield
Address Obfuscation
Exec Shield
Binary Obfuscation
PointGuard™
Instruction Set Randomization
RAD

10. Related Work (2) StackGuard -- add canary word before return address
StackShield --copy return address to confirm when return
Bypassing StackGuard and StackShield
return address
canary word
Buffer
High Address
Low Address …
Saved return address
Low Address

11. Related Work (3) Address Obfuscation PaX ASLR project ASLP
- Randomize the base address of memory regions
-- Randomize the base address of stack/heap
-- Randomize the starting address of dynamic-linked
libraries
-- Randomize the locations of routines and static data
ASLP
Internal fragmentation problem
Derandomization Attack

12. Related Work (4) Bound Checking Bound Checking for C
Require source code / recompile
Runtime overhead are huge
- 4x / 5x when best case
- 10x general case
- 100x worst case

13. Related Work (5) - ELF file format Exec Shield
Data/Stack section non-executable
Code section non-writable
Compatibility problem
- ELF file format
-- Add PT_GNU_STACK and PT_GNU_HEAP
- Nested function
- Recompile / porting
- sigreturn() system call
Return into libc attack can be launched

14. Related Work (6) Binary Obfuscation
Change ELF layout (add new section)
Add new system call to notify the return address
Insert junk code in binary object
Attacker may use the new system call
Return into libc attack / scanning code attack can be launched

15. Related Work (7) PointGuard™
Encrypt pointer, decrypt when reference object

16. Related Work (8) Instruction Set Randomization
Hardware solution, encrypt / decrypt CPU instructions
porting binaries

17. Related Work (9) RAD Return Address Defender Compiler solution
Push return address to RAR when prologue
Pop return address from RAR when eprologue
Need recompile

18. SCP System Design (1) Principles Goal
Prevent attacker from executing int 80 offered by attacker
Prevent attacker from executing int 80 existed in system
Goal
Low overhead
Efficient to protect
Do not require source code
Compatibility
Use less system resource
Easy to maintain

19. SCP System Design (2) Assumptions
Malicious code have to use system call to damage system
vulnerable program is dynamic linked

20. SCP System Design (3) System overview
protect int 0x80’s return address
Use secure enter kernel instead of all sysenter (int $0x80)
movl sys#, %eax
movl arg1, %ebx …
int $0x80

21. SCP System Design (4)
The system call path

22. ( libc wrapper routine )
SCP System Design (5)
The system call path
High Address
( kernel )
sys_open()
Kernel Space
5. return
4. trap into kernel
( libc wrapper routine )
__libc_open()
User Space
3. call wrapper routine
( GOT entry )
Address of __libc_open
2. lookup GOT
6. return
( PLT )
jmp *GOT[__libc_open]
1. go to PLT
( user program )
open();
Low Address

23. (a) Secure Enter Kernel
SCP System Design (6)
Pseudo Code
(a) Secure Enter Kernel
(b) Trap Code
save_all_registers;
page = 0; size = 0;
if ( page == 0 ) {
page = mmap2();
size = copy_trap_code(page);
notify_kernel(size+6); }
restore_all_registers;
call page;
restore_sys#_in_eax;
int $0x80; (sysenter)
return_to_libc;
machine code:
\x8B\x44\x24\x04
\xCD\x80
\x83\xC4\x08
\xC3

24. ( libc wrapper routine )
SCP System Design (7)
New system call path (with SCP system)
High Address
( kernel )
sys_open()
6. return
sys_read()
Kernel Space
5. trap into kernel
( trap page )
int $0x80
4. call trap page
( libc wrapper routine )
__libc_open()
User Space
__libc_read()
7. return
3. call wrapper routine
( GOT entry )
Address of __libc_open
Address of __libc_read
2. lookup GOT
8. return
( PLT )
jmp *GOT[__libc_open]
jmp *GOT[__libc_read]
1. go to PLT
( user program )
open();
read();
Low Address

25. SCP System Design (8) Example with SCP system (lazy binding) Glibc:
4. Call printf()
Glibc:
printf() here
6. Normal system call
5. Notify kernel the RA
3. Load libc.so.6
kernel
Loader (ld-linux.so.2)
1. Notify kernel the RA
Inject code
2. Loading program
program

26. SCP System Design (9) Example with SCP system (lazy binding) Glibc:
4. Call printf()
Glibc:
printf() here
6. Normal system call
5. Notify kernel the RA
3. Load libc.so.6
kernel
Loader (ld-linux.so.2)
1. Notify kernel the RA
Inject code
2. Loading program
program

27. SCP System Design (10) Modify kernel
Add new system call notify_kernel()
- It can only be called 2 times per process
Add new structures in task_struct to restore addresses
- loader_return_address
- syscall_return_address
do_fork()
- Addresses copied from parent process
sys_execve()
- loader_return_address = 0
- syscall_return_address = 0

28. SCP System Design (11)
Introduce fake pages

29. SCP System Design (12) Attack analysis
Attacker can launch scanning code attack to trace real int $0x80
Possible solution:
- Non-readable but executable code segment
- … future work
(user program)
system_call
(PLT)
(GOT)
(libc)
wrapper routine
(heap)
int $0x80

30. SCP System Design (13) SCP system analysis
Allow executable stack - General debugger support
- Nested function
- Do not require ELF modified
- Do not require to recompile
Allow execute programs without ASLR
- No internal fragment problem
- Process crashes are decided by process owners
- Easy to maintain

31. Experimental Result (1)
Efficiency test
Buffer overflow attack with injected code

32. Experimental Result (2)
Efficiency test
Calling notify_kernel test

33. Experimental Result (3)
Micro test
10,000,000 times per system call
System Call
Original libc & Kernel
(usec)
Secure libc & Secure Kernel
Increment
mmap
4.28 %
open %
read
4.72 %
write
0.87 %

34. Experimental Result (4)
Original libc & kernel
execve("./micro-test.open", ["./micro-test.open"], [/* 29 vars */]) = 0
% time seconds usecs/call calls errors syscall
gettimeofday
open
close
brk
mprotect
read
write
munmap
old_mmap
fstat64
total

35. avg timediff = 0.0000073216265842318534575253186069687672 sec = 7.32162658 usec
Command exited with non-zero status 81
Command being timed: "./micro-test.open"
User time (seconds): 0.69
System time (seconds):
Percent of CPU this job got: 100%
Elapsed (wall clock) time (h:mm:ss or m:ss): 2:03.00
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 82
Minor (reclaiming a frame) page faults: 7
Voluntary context switches: 0
Involuntary context switches: 0
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 81

36. Experimental Result (5)
Modified libc & kernel
execve("./micro-test.open", ["./micro-test.open"], [/* 29 vars */]) = 0
% time seconds usecs/call calls errors syscall
gettimeofday
open
close
read
brk
write
mmap2
munmap
utimes
old_mmap
mprotect
fstat64
total

37. avg timediff = 0.0000123534624044317750067014868853298992 sec = 12.35346240 usec
Command exited with non-zero status 82
Command being timed: "./micro-test.open"
User time (seconds): 0.68
System time (seconds):
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 2:53.94
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 104
Minor (reclaiming a frame) page faults: 13
Voluntary context switches: 0
Involuntary context switches: 0
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 82

38. Experimental Result (5)
Macro test
100,000 times per command
Command
Original libc & Kernel
(sec)
Secure libc & Secure Kernel
Increment ls
18.06 %
make
14.61 %
sysctl
18.65 %
tar
16.88 %
gcc
01.45 %

39. Conclusion
We propose a new method to protect system calls by registering valid int 80 on premise
Without recompile
Allow executable stack
Use ASLR optionally
Compatible with other protections

40. Future Work More secure
Implement “executable but non-readable” region in segment section on i386
The NX Bit chip
AMD 64 CPU

41. Thanks
Q & A