1. VM: Chapter 7 Buffer Overflows

2. csci5233 computer security & integrity (VM: Ch. 7) 2 Outline Impact of buffer overflows What is a buffer overflow? Types of buffer overflows heap overflow stack overflow Defense against buffer overflows Examples

3. csci5233 computer security & integrity (VM: Ch. 7) 3 Impact of buffer overflows [Wagner, 2000]: Contributed to as much as 50% of security problems (see Fig. 7-1, p.136) The problem is not getting better. (see Fig. 7-2) Why? Bad language design (e.g., C/C++) Poor programming

4. csci5233 computer security & integrity (VM: Ch. 7) 4 Impact of buffer overflows The unsafe C language: C is inherently unsafe. There is no bounds checking on array and pointer references. It is the programmer’s responsibility to do the checking. Unsafe string operations in the standard C lib.

5. csci5233 computer security & integrity (VM: Ch. 7) 5 What is buffer overflow? Buffer is a memory area where contiguous chunks of the same data type are allocated. Buffer overflow occurs when a program writes past the bounds of a buffer. P1’s buffer P2 The bound of P1’s buffer Overflow area P2’s memory area Buffer Overflow Q: What could be the impact of a buffer overflow? How would the impact be determined?

6. csci5233 computer security & integrity (VM: Ch. 7) 6 The impact of a buffer overflow Factors that determine the impact of a buffer overflow: 1. What data or code in P2 were over-written? 2. What data were written into the overflow area? 3. How would P1 access the overflow area? 4. How would P2 access the overflow area? Overall, the outcome is unpredictable!

7. csci5233 computer security & integrity (VM: Ch. 7) 7 Why are buffer overflows a security problem? Reason: A malicious program may exploit buffer overflow to gain privileged access to a system. The exploit is usually tied to specific architecture of particular operating systems. Example: stack smashing – An overflow overwrites the return address in a stack frame. Privilege escalation: A privilege is granted to a sequence of users through a chain of granting process In an OS, many applications and/or utilities are given super user privilege, usually only temporarily, to perform part of its job that requires the privilege.  regular mode versus privileged mode

8. csci5233 computer security & integrity (VM: Ch. 7) 8 Why are buffer overflows a security problem? Threat: When a malicious program (M) gain control of a process (P), it “inherits” its privileges. For example: In UNIX, applications such as lpr and xterm have been abused into giving up root privileges thru the exploit of buffer overflow in suid regions of the code. Another example: A malicious program finds a buffer overflow in an suid root program, and then exploits to trigger an interactive shell (with root privilege).

9. csci5233 computer security & integrity (VM: Ch. 7) 9 Buffer Overflow and C/C++ Functions C functions that do not check bounds: pp. 141-149 See also Table 7-1, p.152 Any solution? Defensive programming by 1.Set an upper bound of an argument, or use an function that allows a bound to be set Example: strncpy( ) in place of strcpy( ) 2.Check the length of an argument before passing it to a library function  To avoid internal buffer overflow (p.147) Example: relpath( ), syslog( ), getopt( ), getpass( ), etc. Are these the ultimate solutions? Probably not.

10. csci5233 computer security & integrity (VM: Ch. 7) 10 Types of Buffer Overflows Stack overflows versus heap overflows Compared to other parts of a process (such as data segment and program segment, which are static), the stack and the heap are dynamic. Stack is used for allocating the context of the current function call, such as non-static local variables, parameters passed by value, return address, …  activation record (or stack frame) Heap is for allocating data requested dynamically by a user program, such as via malloc( ) in C or new in C++. Heap overflows are generally much harder to exploit than stack overflows. Why?

11. csci5233 computer security & integrity (VM: Ch. 7) 11 Heap Overflows A less likely attack than stack overflows. The attacker needs to know many things: 1.Which variable(s) are security critical; 2.How the variables are allocated in the heap; 3.The number of bytes allocated to a variable; 4.A buffer that can overflow the target variable; 5.… Example: pp.155-159.

12. csci5233 computer security & integrity (VM: Ch. 7) 12 Stack Overflows Steps: p.159 1.Find a stack-allocated buffer that allows us to overwrite the return address in a stack frame; 2.Place some hostile code in memory to which we can jump when the function we’re attacking returns; 3.Write over the return address on the stack with a value that causes the program to jump to our hostile code. Examples: pp.160-177.

13. csci5233 computer security & integrity (VM: Ch. 7) 13 “Solutions” to Buffer Overflows 1. Software scanning tools (VM: Chapter 6) can help to find and remove buffer overflow problems. 2. Nonexecutable stacks: requires OS support or patch 3. Get a compiler that performs array bounds checking for C programs. 4. Stackguard tools adds a little bit of data at the end of stack-allocated data (called a canary), and later checks the canary to see whether the allocated data is still valid.

14. csci5233 computer security & integrity (VM: Ch. 7) 14 “Solutions” to Buffer Overflows 5. Memory integrity checking tools, such as Rational’s Purify 6. Replace vulnerable calls with “safe” versions. 7. Don’t use C. Use a type-safe language such as Java!

15. csci5233 computer security & integrity (VM: Ch. 7) 15 Summary Buffer overflow is the most common attack at C programs. It is difficult to write secure C programs. The best approach is to use a “safe” language. The challenge is there exist lots of C/C++ codes out there. Next: –Applying cryptography (VM: Ch 11) –CSCI5931 Web Security (Spring 2003)