nd Joint Workshop between Security Research Labs in JAPAN and KOREA An Enhanced Buffer Separation Scheme to Protect Security Sensitive Data against Buffer Overflow Attacks Researcher : Lee, Seung Min (Presenter : Lee, Seung Ick) High Performance Computing Laboratory at POSTECH

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 2/14 Contents  Introduction  Related Works  Motivation  Problem Definition  Proposed Idea  Performance Analysis  Conclusion and Future Works

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 3/14 Introduction  Buffer overflow Occur when a program or process stores more data in buffer than the buffer size  Security sensitive data Data to be a target of buffer overflow attack for a successful change of control flow  Basic steps of buffer overflow attack The first step  To find, discover and identify vulnerability of buffer overflow The second step  To overflow and overwrite security sensitive data near buffer The third step  To change control flow of process The fourth step  To execute intended instruction or process

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 4/14  Protection Methods are classified depending on the step the prevention method is applied.  The first step To eliminate buffer overflow vulnerability  The second step To prevent overflow or overwrite data  The third step To prevent no intended control flow of a program or process  The fourth step To apply that data of buffer can’t be executed through hardware support Protection Methods Related Works ( 1/2 )

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 5/14 Problems of Previous Works  Problem of the prevention method on the first step Generate too many false warnings and miss errors in the code  Problem of the prevention method on the second step Performance degradation through array bounds checking Not to provide complete protection against vulnerabilities in user defined or non standard library code  Problem of prevention method on the third and fourth steps Occur an exception or termination because of attack failure Related Works ( 2/2 )

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 6/14 Motivation ( 1/2 )  The prevention methods applied in these steps but the second have the problem of process availability, since the process stops its execution because of false warnings, exceptions or termination.  Our focus is to provide a reliable process availability and more secure protection method.  Buffer separation approach is the method applied on the second step for prevention. Remove buffers from stack Allow the occurrence of buffer overflow but prevent security sensitive data from being overwritten Gemini and DYBOC

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 7/14  Shortcoming of previous works Gemini  Can generate heap overflow because of buffer using heap area instead of stack DYBOC  Has a memory overhead problem because of using write- protected page Motivation ( 2/2 )

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 8/14 Problem Definition  To prevent stack and heap overflow for using enhanced buffer separation approach  To have minimal performance penalty

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 9/14 Basic Idea  Assumption We can know the size of arrays at compile time.  Enhanced buffer separation schemes Buffer stack  Separate buffer from the security sensitive data on the stack area. Separated meta data  Separate meta data from buffer on the heap area. Proposed Idea ( 1/3 )

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 10/14  Buffer stack is to prevent against stack overflow.  Buffer stack will be determined as using heap or stack at the compile time. Buffer Stack Architecture Proposed Idea ( 2/3 )

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 11/14 Separated Meta Data Architecture  Separated meta data is to prevent against heap overflow. Proposed Idea ( 3/3 )

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 12/14 Performance Analysis  Performance comparison with Gemini, DYBOC and our solutions  Limitation If the security sensitive data exists inside buffers, it is very hard to split them from the buffers. Our solutions can’t prevent data from pointer operations that use the primitive type variables. Compiler has to know the size of stack. Data(Stack)Data(Heap)TimeMemory OriginalOverwrite -- GeminiPreventOverwriteIncrease DYBOCPreventOverwritePoor Our Solution Prevent A little Increase Poor

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 13/14 Experiments  We chose two vulnerabilities which is similar to the source of finger demon and PCT SSL vulnerability.  We applied single thread program but it can be applied in multi-threaded environment easily.  In both cases, the security sensitive data of the original program is overwritten and make an exception.  But, the program applied our approaches correctly executes because it protects the security sensitive data even though the buffer data is tainted.

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 14/14 Conclusion and Future Works  Conclusion Buffer stack prevents the stack overflow attack. Separated meta data prevents the heap overflow attack.  Future works We must find trade-off between performance and memory overhead on case 1 and 2 of the Buffer stack architecture, and then will choose one of cases. Throughout the implementation, we can provide more exact performance analysis. We can provide more powerful solutions to change the main assumption: security sensitive data is located near the buffer and also in the buffer.

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 15/14 Thank you! Questions?

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 16/14 Backup slide

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 17/14 References  [1] Hiroaki Etoh and Kunikazu Yoda. Protecting from Stack-Smashing Attacks. Published on World-Wide Web at URL June 2000http://  [2] Donaldson, Mark E. Inside The Buffer Overflow Attack: Mechanism, Method, & Prevention. April 3, URL:  [3] Bharath Madhusudan, John Lockwood. Design of a System for Real-Time Worm  [4] H. Wang, C. Guo, D. Simon, and A. Zugenmaier. Shield: Vulnerability-driven network filters for preventing known vulnerability exploits. In Proceedings of ACM SIGCOMM, Portland, OR, Aug Detection, 12th Annual Proceedings of IEEE Hot Interconnects 2004  [5] J. Newsome and D. Song. Dynamic taint analysis for automatic detection, analysis, and signature generation of exploits on commodity software. In Proceedings of the 12th Annual Network and Distributed System Security Symposium(NDSS05), Feb  [6] Rinard. M., Cada. C., Dumitran. D., Roy. D., Leu.T. A Dynamic Technique for Eliminating Buffer Overflow Vulnerabilities (and Other Memory Errors). In: Proceedings 20th Annual Computer Security Applications Conference (ACSAC), 2004  [7] StackShield.  [8] A. Baratloo, T. Tsai, and N. Singh. Transparent Run-Time Defense Against Stack Smashing Attacks. In Proceedings. of the USENIX Annual Technical Conference, June 2000

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 18/14 Reference (Cont.)  [9] Crispin Cowan, Steve Beattie, John Johansen, and Perry Wagle. Pointguard: Protecting pointers from buffer overflow vulnerabilities. In Proceedings of the 12th USENIX Security Symposium, Washington, D.C., August 2003  [10] BB. Madan, S. Phoha, G. NIST, KS Trivedi, StackOfence: A Technique for Defending Against Buffer Overflow Attacks, In Proceedings of the International COnference on Information Technology: Coding and Computing(ITCC05), 2005  [11] J. Xu, Z. Kalbarczyk, S. Patel, and R. K. Iyer. Architecture support for defending against buffer overflow attacks. In 2nd Workshop on Evaluating and Architecting Systems for Dependability, 2002  [12] S Bhatkar, DC DuVarney, R Sekar, Address obfuscation: An efficient approach to combat a broad range of memory error exploits, In Proceedings of the 12th USENIX Security Symposium, 2003  [13] Wilander, J. and M. Kamkar, Comparison of Publicly Available Tools for Dynamic Buffer Overflow Prevention, 10th Network and Distributed System Security Symposium, 2003  [14] Stelios Sidiroglou, Giannis Giovanidis, and Angelos D. Keromytis, A Dynamic Mechanism for Recovering from Buffer Overflow Attacks  [15] R Hieb, RK Dybvig, C Bruggeman, L Hall, Representing Control in the Presence of First-Class Continuations, In Proceedings of the ACM SIGPLAN 1990 conference on 1990 [16] EG Barrantes, DH Ackley, S Forrest, D Stefanovic, Randomized Instruction Set Emulation, ACM Transactions on Information and System Security, 2005  [16] Christopher Dahn, Spiros Mancoridis, Using Program Transformation to Secure C Programs Against Buffer Overflows, in Proceedings of the 10 th Working Conference on Reverse Engineering (WCRE03)

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 19/14 Scenario void f(char *a) { char c[10]; strcpy(c,a); } void main() { char a[20]; int i; a[5] = 4; f1(a); } Original Code push ebp mov ebp, esp sub esp, 10 push [ebp+8] push [ebp-10] call strcpy add esp,10 leave ret push ebp mov ebp, esp sub esp, 24 mov [ebp-15],4 push ebp-20 call f1 add esp,24 leave ret Original Pseudo Assembly Code Prologue Epilogue Return Address of main Old EBP a i c Return Address of f Old EBP EBP ESP address of a push ebp mov ebp, esp sub exp, 10 push [ebp+4] push exp call strcpy add exp,10 leave ret push ebp mov ebp, esp sub esp, 4 sub exp, 20 mov [exp+5],4 push exp call f1 add exp,20 add esp,4 leave ret Modified Pseudo Assembly Code Return Address of main Old EBP a i c Return Address of f Old EBP EBP ESP address of a EXP

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 20/14 Scenario (Cont.) void f(char *a) { char c[10]; strcpy(c,a); } void main() { char a[20]; int i; a[5] = 4; f1(a); } Original Code push ebp mov ebp, esp sub esp, 10 push [ebp+4+S] push ebp-10+S call strcpy add esp,10 leave ret push ebp mov ebp, esp sub esp, 24 mov [exp-15+S],4 push exp call f1 add esp,24 leave ret Modified Pseudo Assembly Code Prologue Epilogue Return Address of main Old EBP a i c Return Address of f Old EBP EBP ESP address of a S : Size of Stack

nd Joint Workshop between Security Research Labs in JAPAN and KOREA 21/14 function(char *packet, unsigned int N) { char buf[32]; unsigned int register i; if(N < 32) { memcpy(buf,packet,N); for(i = 0; i < N; i++) buf[i+N] = ~buf[i]; } } Vulnerable code of PCT SSL vulnerability