1. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Stack allocation and buffer overflow CSCE 531 Presentation by Miao XU xum@engr.sc.edu

2. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Outline Stack allocation in Windows What is buffer overflow How to exploit buffer overflow Demo

3. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Stack allocation in TAM SB LB ST call frame SB = Stack base LB = Locals base ST = Stack top call frame Dynamic link globals

4. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Stack allocation in TAM (Contd.) A frame contains A dynamic link: to next frame on the stack (the frame of the caller) Return address Local variables for the current activation return address locals Link data Local data LB ST link

5. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering What’s going on inside Windows? Initial stack state EBP ESP EBP: Extended Base Pointer ESP: Extended Stack Pointer Current frame

6. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering What’s going on inside Windows? Before call f(arg1, arg2) –Push arguments EBP ESP EBP: Extended Base Pointer ESP: Extended Stack Pointer Current frame agr2 arg1

7. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering What’s going on inside Windows? Before call f(arg1, arg2) –Push next instruction address EBP ESP EBP: Extended Base Pointer ESP: Extended Stack Pointer Current frame agr2 arg1 Ret. Addr.

8. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering What’s going on inside Windows? Enter into f(arg1, arg2) –Push current EBP EBP ESP EBP: Extended Base Pointer ESP: Extended Stack Pointer Current frame agr2 arg1 Ret. Addr. Prev. EBP

9. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering What’s going on inside Windows? Enter into f(arg1, arg2) –Move EBP to ESP EBP ESP EBP: Extended Base Pointer ESP: Extended Stack Pointer Current frame agr2 arg1 Ret. Addr. Prev. EBP

10. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Enter into f(arg1, arg2) EBP ESP Previous frame agr2 arg1 Ret. Addr. Prev. EBP Current frame What’s going on inside Windows? EBP: Extended Base Pointer ESP: Extended Stack Pointer

11. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering What is buffer overflow? Related with stack allocation A buffer overflow, or buffer overrun, is an anomaly where a process stores data in a buffer outside the memory the programmer set aside for it. –Wikipedia

12. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Buffer overflow void function(char *str) { char buffer[8]; strcpy(buffer,str); } void main() { char large_string[256]; for( int i = 0; i < 255; i++) large_string[i] = 'A'; function(large_string); } void function(char *str) { char buffer[8]; strcpy(buffer,str); } void main() { char large_string[256]; for( int i = 0; i < 255; i++) large_string[i] = 'A'; function(large_string); }

13. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering An example

14. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering An example

15. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering An example

16. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering An example

17. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering An example

18. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering An example Return to 0x41414141

19. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Problems with buffer overflow A demo

20. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Acknowledgement The demo and part of this slides are from the training when the presenter was in Symantec, Chinese Development Center, Beijing The example comes from the following reference: –Aleph One, Smashing the stack for fun and profit, Phrack Magzine, Vol. 7 (49), 1996

21. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering Questions?

22. UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering