1. James Walden Northern Kentucky University
Buffer Overflows
James Walden
Northern Kentucky University

2. CSC 666: Secure Software Engineering
Topics
What is a Buffer Overflow?
Buffer Overflow Examples
Program Stacks
Smashing the Stack
Shellcode
Mitigations
CSC 666: Secure Software Engineering

3. CSC 666: Secure Software Engineering
Buffer Overflows
A program accepts too much input and stores it in a fixed length buffer that’s too small.
char A[8];
short B=3; A B 3
gets(A); A B o v e r f l w s
CSC 666: Secure Software Engineering

4. Buffer Overflow Examples
Morris Worm
Took down most of Internet in 1988.
Exploited a buffer overflow in fingerd.
Subsequent worms used overflow attacks too.
CVE : Adobe Shockwave
Overflow in Shockwave Player <
Allows arbitrary remote code execution.
Adobe released patch to fix with 4 other CVEs
CSC 666: Secure Software Engineering

5. Buffer Overflow Example #1
What’s the mistake in this program?
int main() {
int array[5] = {1, 2, 3, 4, 5};
printf("%d\n", array[5]); }
Program output:
> gcc -o buffer buffer.c
> ./buffer
CSC 666: Secure Software Engineering

6. Buffer Overflow Example #2
Writing beyond the buffer:
int main() {
int array[5] = {1, 2, 3, 4, 5};
int i;
for( i=0; i <= 255; ++i )
array[i] = 41; }
Program output:
> gcc -o bufferw bufferw.c
> ./bufferw
Segmentation fault (core dumped)
CSC 666: Secure Software Engineering

7. What happened to our program?
The buffer overflow:
Overwrote memory beyond buffer with 41.
Memory page was not writable by program.
OS terminated prog with segmentation fault.
Do overflows always produce a crash?
Most of the time, yes.
Careful attacker can access valid memory.
CSC 666: Secure Software Engineering

8. Why do we keep making the same mistake?
C/C++ inherently unsafe.
No bounds checking.
Unsafe library functions: strcpy(), sprintf(), gets(), scanf(), etc.
D, Java, Python, Ruby largely immune.
C/C++ gains performance by not checking.
CSC 666: Secure Software Engineering

9. CSC 666: Secure Software Engineering
Process Memory Layout
high mem
low mem
argv, env
stack
heap
bss
data
text
Argv/Env: CLI args and environment
Stack: generally grows downwards
Heap: generally grows upwards
BSS: unitialized global data
Data: initialized global data
Text: read-only program code
Memory segments have access protections: r, w, x.
Text is usually read-only and may be shared between processes.
CSC 666: Secure Software Engineering

10. CSC 666: Secure Software Engineering
Memory Layout Example
/* data segment: initialized global data */
int a[] = { 1, 2, 3, 4, 5 };
/* bss segment: uninitialized global data */
int b;
/* text segment: contains program code */
int main(int argc, char **argv) /* ptr to argv */ {
/* stack: local variables */
int *c;
/* heap: dynamic allocation by new or malloc */
c = (int *)malloc(5 * sizeof(int)); }
CSC 666: Secure Software Engineering

11. CSC 666: Secure Software Engineering
Program Stack Layout
Low Memory
b() { … }
a() {
b();
main() {
a();
Unallocated
Stack Frame
for b()
for a()
for main()
High Memory
CSC 666: Secure Software Engineering

12. CSC 666: Secure Software Engineering
Stack Frames
A function call produces a stack frame.
Return address of the caller.
Actual arguments used in function call.
Local variables.
Register EBP points to current frame.
Stack frame is deallocated on return.
CSC 666: Secure Software Engineering

13. CSC 666: Secure Software Engineering
C Calling Convention
Push params on stack in reverse order.
Parameter #N …
Parameter #1
Issue a call instruction.
Pushes address of next instruction (the return address) onto stack.
Modifies EIP to point to start of function.
CSC 666: Secure Software Engineering

14. Stack before Function Executes
old stack frame
parameter #N …
parameter #1
return address
Frame Pointer
Stack Pointer
FP points at top of old frame.
call statement places old PC onto stack
CSC 666: Secure Software Engineering

15. Function Initialization
Function pushes FP (%ebp) onto stack.
pushl %ebp
Sets FP to current SP.
Allows function to access params as fixed indexes from frame pointer.
movl %esp, %ebp
Reserves stack space for local vars.
subl $12, %esp
CSC 666: Secure Software Engineering

16. Stack at Function Start
Frame Pointer
Stack Pointer
old stack frame
parameter #N …
parameter #1
return address
old FP
local vars
FP still points at new frame.
SP points at current top of stack, moving with local var allocation/deallocation.
CSC 666: Secure Software Engineering

17. CSC 666: Secure Software Engineering
Function Return
Stores return value in %eax.
movl $1, %eax
Restores stack and frame pointers.
movl %esp, %ebp
popl %ebp
Pops return address off stack and transfers control to that address.
ret
CSC 666: Secure Software Engineering

18. Controlling Execution
Let’s make the program an infinite loop by changing the return value to start of main().
(gdb) disassemble main
Dump of assembler code for main:
0x080483c1 <main+0>: push %ebp
0x080483c2 <main+1>: mov %esp,%ebp
0x080483c4 <main+3>: call 0x804839c <printInput>
0x080483c9 <main+8>: leave
0x080483ca <main+9>: ret
CSC 666: Secure Software Engineering

19. Sending a non-ASCII Value
void main() {
char addr[44];
int i;
for( i=0; i<=40; i+=4 )
*(long *) &addr[i]=0x080483c1;
puts(addr); }
CSC 666: Secure Software Engineering

20. CSC 666: Secure Software Engineering
Does it work?
(./address; cat) | ./overflow
input1
input2
input3
Segmentation fault (core dumped)
CSC 666: Secure Software Engineering

21. CSC 666: Secure Software Engineering
Code and Data
What’s the difference between code and data?
CSC 666: Secure Software Engineering

22. CSC 666: Secure Software Engineering
Code Injection
Use a two step process:
Use buffer overflow to write machine code (shellcode) onto stack.
Rewrite return address to point to machine code on the stack.
Program will do whatever we tell it to do in those machine instructions.
CSC 666: Secure Software Engineering

23. CSC 666: Secure Software Engineering
Shellcode
Shellcode is machine code that starts a command shell. With a shell, you can run any command.
CSC 666: Secure Software Engineering

24. CSC 666: Secure Software Engineering
Shellcode
Shellcode in C.
int main() {
char *name[2];
name[0] = "/bin/sh";
name[1] = 0x0;
execve(name[0], name, 0x0); }
Running the program.
> gcc –ggdb –static –o shell shellcode.c
> ./shell
sh-3.00$ exit
CSC 666: Secure Software Engineering

25. From C to Machine Language
char shellcode[] =
"\xeb\x1f\x5e\x89\x76\x08\x31\xc0\x88\x46\x07\x89\x46\x0c\xb0\x0b"
"\x89\xf3\x8d\x4e\x08\x8d\x56\x0c\xcd\x80\x31\xdb\x89\xd8\x40\xcd"
"\x80\xe8\xdc\xff\xff\xff/bin/sh";
void main() {
int *ret;
ret = (int *)&ret + 2;
(*ret) = (int)shellcode; }
> gcc -o testsc2 testsc2.c
> ./testsc2
sh-3.00$ exit
CSC 666: Secure Software Engineering

26. CSC 666: Secure Software Engineering
Writing an Exploit
Construct shellcode to inject.
Find exploitable buffer in a program.
Estimate address of buffer.
Run program with an input that:
Injects shellcode into stack memory.
Overwrites return address with address of your shellcode.
CSC 666: Secure Software Engineering

27. CSC 666: Secure Software Engineering
Improving the Odds
Determining the correct address of your shellcode is difficult.
What if you could use multiple addrs?
Pad buffer with NOP instructions preceding the shellcode.
If function returns anywhere in NOP pad, it will continue executing until it executes the shellcode.
CSC 666: Secure Software Engineering

28. Overflow Exploits w/o the Stack
Overflows can exist in other segments.
Only the stack has return addresses.
Must rewrite other addresses
Function pointers
Longjmp buffers
Or alter security critical variables.
CSC 666: Secure Software Engineering

29. Buffer Overflow Mitigations
Use language with bounds checking.
Do your own bounds checking.
Avoid unsafe functions.
Use safe functions securely.
Operating system defenses.
Compiler-based defenses.
CSC 666: Secure Software Engineering

30. Languages with Bounds Checking
High-level languages do bounds checks
Java
JavaScript
OCaml
Python
Ruby
Scheme
Smalltalk
C variants
CCured
Cyclone D
Image from
CSC 666: Secure Software Engineering

31. CSC 666: Secure Software Engineering
Bounds Checking
Check input length before copying into buffer.
int myfunction(const char *str) {
char buf[1024];
/* strlen() doesn’t count NULL */
if (strlen(str) >= sizeof(buf)) {
/* str too long */
exit(1); }
If strlen(str) returns 1024, str is actually 1025 bytes long because of the NULL terminator, and thus the string is too large to be copied into buf, which has only 1024 bytes of space.
CSC 666: Secure Software Engineering

32. Unsafe Functions: Input
gets(char *s)
Always an overflow. Cannot be checked.
Use fgets(char *s, int size, FILE *stream);
scanf(const char *fmt, …)
Similar: _tscanf, wscanf, sscanf, fscanf, …
Use of “%s” format allows overflow.
Use “%Ns” to limit length of input.
CSC 666: Secure Software Engineering

33. Unsafe Functions: strcat
strcpy(char *dst, char *src)
Similar: wscpy, wcscpy, mbscpy
Overflow if dst smaller than src.
strncpy(char *dst, const char *src, size_t n)
Similar: _tcsncpy, wcscpyn, _mbsncpy, …
No NULL termination if src >= dst.
CSC 666: Secure Software Engineering

34. Bounded Function Pitfalls
Destination buffer overflows because bound depends on size of source data, not destination buffer.
Destination buffer left without null terminator, often as result of off-by-one error.
Destination buffer overflows because its bound is specified as the total size of the buffer, rather than space remaining.
Programs writes to arbitrary location in memory as destination buffer is not null-terminated and function begins writing at location of first null in destination buffer.
CSC 666: Secure Software Engineering

35. CSC 666: Secure Software Engineering
Safe String Libraries
UNIX Libraries C
Bstrlib
MT-Safe
SafeStr
strlcpy(), strlcat()
Vstr
C++
std::string (STL)
Windows Libraries C
Safe CRT
strlcpy(), strlcat()
StrSafe
C++
CString (MFC)
Safe C++
std::string (STL)
C++ can be used with C-style strings in dangerous ways (see example in a few slides.)
CSC 666: Secure Software Engineering

36. strlcpy() and strlcat()
size_t strlcpy (char *dst, const char *src, size_t size);
size_t strlcat (char *dst, const char *src, size_t size);
Size is max size of dest buffer (not maximum number of chars to copy), including NULL.
Destination buffer always NULL terminated
Return how much space would be required in destination buffer to perform operation.
BSD-style open source license.
strlcat() and strlcpy() exist.
CSC 666: Secure Software Engineering

37. CSC 666: Secure Software Engineering
Character Sets
Chars represented using encoding forms that map code points to printable chars.
Fixed Width
ISO
UTF-32
Variable Width
UTF-8
UTF-16 (Java, .NET)
Character
Encoding
Code Point s
ISO
UTF-8 73 ÿ FF
C3 BF
CSC 666: Secure Software Engineering

38. CSC 666: Secure Software Engineering
Wide Characters
C/C++
char contains 1-byte characters
wchar_t is 2-byte, 4-byte on some platforms
Java and .NET strings
UTF-16
Buffer Overflow issues
Mixing up different character-set string types.
Are sizes measured in bytes or characters?
CSC 666: Secure Software Engineering

39. CSC 666: Secure Software Engineering
C++ String Dangers
Using C-style strings with cin
char username[16];
cin >> username;
The [] operator does no bounds checking.
Converting from C++ to C-style strings:
string::data() output is not NULL terminated.
string::c_str() output is NULL terminated.
CSC 666: Secure Software Engineering

40. Dealing with Overly Long Inputs
Prevent operation with an error message.
Possibly exit program too.
Truncate input to allowed length.
Can lead to null-termination errors.
Can cause input to be misinterpreted later.
Resize the buffer to accomodate input.
Limit resizing to avoid memory DoS.
CSC 666: Secure Software Engineering

41. Dynamic vs. Static Allocation
Simple
Easy to track buffer size for bounds checks.
Inflexible.
Limited options if size is too small.
Waste memory if size is too large.
Dynamic
Complex
Manually tracking buffer sizes can lead to overflows due to stale size data.
Flexible
Can lead to memory exhaustion if no limits.
Possibility of use after free and double free errors.
CSC 666: Secure Software Engineering

42. CSC 666: Secure Software Engineering
Non-executable Stack
Memory protection prevents exploit code from being executed.
Some applications execute code on the stack/heap.
x86 arch doesn’t have exec bit in page tables.
Segment limits can divide memory into two parts: executable and non-executable.
Keep program code in low memory.
Keep data and stack in high memory.
Coarse-grained.
NX Technology
Exec bit for page tables.
Added in AMD64 and newer Intel P4 processors.
Only works in PAE 64-bit page table format.
CSC 666: Secure Software Engineering

43. Return-Oriented Programming
ROP transfers control to code that already exists in memory.
Put function arguments on stack.
Change return address to that function.
libc has functions to start a shell.
Allows exploit even if stack non-executable.
Sophisticated ROP attacks create multiple stack frames to run multiple functions that are in memory.
CSC 666: Secure Software Engineering

44. Address Randomization
Randomize layout of memory space
Stack location.
Shared library locations.
Heap location.
PIE: Position Independent Executable
Default format: binary compiled to work at an address selected when program was compiled.
Gcc can compile binaries to be freely relocatable throughout address space.
gcc flags: -fpie –pie
Program loaded at different address for each invocation.
CSC 666: Secure Software Engineering

45. CSC 666: Secure Software Engineering
Defence: Stackguard
Compiler extension for gcc
code must be compiled w/ Stackguard
Detects altered return address
before function returns
adds “canary” word to stack
must overwrite canary to change return addr
use random canary words for each function to avoid guessing attacks
CSC 666: Secure Software Engineering

46. Stackguard Stack Layout
Frame Pointer
Stack Pointer
old frame
param1
param2
old PC
canary word
old FP
local vars
FP still points at new frame.
SP points at current top of stack, moving with local var allocation/deallocation.
CSC 666: Secure Software Engineering

47. Stackguard Effectiveness
Code dependencies
are dynamic libraries stackguarded?
Compatibility
Recompiled entire RedHat Linux system.
Small performance cost
canary insert and check overhead on each call
Protects against future stack attacks.
Similar tools:
gcc -fstack-protector flag
Visual Studio 2005
Visual Studio 2003 had /gs flag, but 2005 better and on by default.
Gcc 4.1 will include Propolice (-fstack-protector) by default, but many Linux distributions include Propolice in their gcc already.
CSC 666: Secure Software Engineering

48. Buffer Overflow: Key Points
Buffer overflow attacks.
C/C++ perform no bounds checking.
There is no difference btw code and data.
Smashing the stack.
Mitigating buffer overflows.
Use a language with bounds checking.
Check your own bounds in C/C++.
Use safe functions, string libraries.
CSC 666: Secure Software Engineering

49. References
Aleph Null, “Smashing the Stack for Fun and Profit,” Phrack 49, 1996.
Brian Chess and Jacob West, Secure Programming with Static Analysis, Addison-Wesley, 2007.
Johnathan Bartlett, Programming from the Ground Up, Bartlett Publishing, 2004.
Matt Conover & w00w00 Security Team, “w00w00 on Heap Overflows,”
Mark Graff and Kenneth van Wyk, Secure Coding: Principles & Practices, O’Reilly, 2003.
Horizon, “Bypassing Non-executable Stack Protection on Solaris,”
Greg Hoglund and Gary McGraw, Exploiting Software: How to Break Code, Addison-Wesley, 2004.
Michael Howard and David LeBlanc, Writing Secure Code, 2nd edition, Microsoft Press, 2003.
Koziol, et. al, The Shellcoder’s Handbook: Discovering and Exploiting Security Holes, Wiley, 2004.
Robert C. Seacord, Secure Coding in C and C++, Addison-Wesley, 2006.
John Viega and Gary McGraw, Building Secure Software, Addison-Wesley, 2002.
David Wheeler, Secure Programming for UNIX and Linux HOWTO,