ANCS 2006 Scalable Network-based Buffer Overflow Attack Detection Fu-Hau Hsu Department of Computer Science and Information Engineering National Central.

Slides:



Advertisements
Similar presentations
Insertion, Evasion and Denial of Service: Eluding Network Intrusion Detection Aaron Beach Spring 2004.
Advertisements

Computer Security: Principles and Practice EECS710: Information Security Professor Hossein Saiedian Fall 2014 Chapter 10: Buffer Overflow.
Lecture 16 Buffer Overflow modified from slides of Lawrie Brown.
HTTP Cookies. CPSC Application Layer 2 User-server state: cookies Many major Web sites use cookies Four components: 1) cookie header line of HTTP.
Foundations of Network and Computer Security J J ohn Black Lecture #30 Nov 26 th 2007 CSCI 6268/TLEN 5831, Fall 2007.
Breno de MedeirosFlorida State University Fall 2005 Buffer overflow and stack smashing attacks Principles of application software security.
1 CHAPTER 8 BUFFER OVERFLOW. 2 Introduction One of the more advanced attack techniques is the buffer overflow attack Buffer Overflows occurs when software.
Computer Security Fundamentals by Chuck Easttom Chapter 4 Denial of Service Attacks.
 Firewalls and Application Level Gateways (ALGs)  Usually configured to protect from at least two types of attack ▪ Control sites which local users.
The Attack and Defense of Computers Dr. 許 富 皓 Attacking Program Bugs.
Intrusion Detection Systems and Practices
電腦攻擊與防禦 The Attack and Defense of Computers Dr. 許 富 皓.
FIREWALLS & NETWORK SECURITY with Intrusion Detection and VPNs, 2 nd ed. 6 Packet Filtering By Whitman, Mattord, & Austin© 2008 Course Technology.
1 RISE: Randomization Techniques for Software Security Dawn Song CMU Joint work with Monica Chew (UC Berkeley)
Control hijacking attacks Attacker’s goal: – Take over target machine (e.g. web server) Execute arbitrary code on target by hijacking application control.
Lecture 16 Buffer Overflow
File Transfer Protocol (FTP)
Address Obfuscation: An Efficient Approach to Combat a Broad Range of Memory Error Exploits Sandeep Bhatkar, Daniel C. DuVarney, and R. Sekar Stony Brook.
1 Advanced Application and Web Filtering. 2 Common security attacks Finding a way into the network Exploiting software bugs, buffer overflows Denial of.
Lucent Technologies – Proprietary Use pursuant to company instruction Learning Sequential Models for Detecting Anomalous Protocol Usage (work in progress)
The Impact of Programming Language Theory on Computer Security Drew Dean Computer Science Laboratory SRI International.
1 Intrusion Detection Systems. 2 Intrusion Detection Intrusion is any use or attempted use of a system that exceeds authentication limits Intrusions are.
Packet Filtering. 2 Objectives Describe packets and packet filtering Explain the approaches to packet filtering Recommend specific filtering rules.
A Statistical Anomaly Detection Technique based on Three Different Network Features Yuji Waizumi Tohoku Univ.
Address Space Layout Permutation
Security Exploiting Overflows. Introduction r See the following link for more info: operating-systems-and-applications-in-
Port Scanning 0x470~0x480 Presenter SangDuk Seo 1.
Computer Security and Penetration Testing
BLENDED ATTACKS EXPLOITS, VULNERABILITIES AND BUFFER-OVERFLOW TECHNIQUES IN COMPUTER VIRUSES By: Eric Chien and Peter Szor Presented by: Jesus Morales.
Jozef Goetz, Application Layer PART VI Jozef Goetz, Position of application layer The application layer enables the user, whether human.
UNIT IP Datagram Fragmentation Figure 20.7 IP datagram.
1 CHAPTER 3 CLASSES OF ATTACK. 2 Denial of Service (DoS) Takes place when availability to resource is intentionally blocked or degraded Takes place when.
Mitigation of Buffer Overflow Attacks
CIS 450 – Network Security Chapter 7 – Buffer Overflow Attacks.
Lecture slides prepared for “Computer Security: Principles and Practice”, 3/e, by William Stallings and Lawrie Brown, Chapter 10 “Buffer Overflow”.
IEEE Communications Surveys & Tutorials 1st Quarter 2008.
Fundamentals of Proxying. Proxy Server Fundamentals  Proxy simply means acting on someone other’s behalf  A Proxy acts on behalf of the client or user.
Chapter 15 – Part 2 Networks The Internal Operating System The Architecture of Computer Hardware and Systems Software: An Information Technology Approach.
1 Figure 4-1: Targeted System Penetration (Break-In Attacks) Host Scanning  Ping often is blocked by firewalls  Send TCP SYN/ACK to generate RST segments.
Buffer Overflow Attack-proofing by Transforming Code Binary Gopal Gupta Parag Doshi, R. Reghuramalingam The University of Texas at Dallas 11/15/2004.
Department of Computer Science and Engineering Applied Research Laboratory Architecture for a Hardware Based, TCP/IP Content Scanning System David V. Schuehler.
Buffer Overflow Attack Proofing of Code Binary Gopal Gupta, Parag Doshi, R. Reghuramalingam, Doug Harris The University of Texas at Dallas.
Lecture 9: Buffer Ovefflows and ROP EEN 312: Processors: Hardware, Software, and Interfacing Department of Electrical and Computer Engineering Spring 2014,
What is exactly Exploit writing?  Writing a piece of code which is capable of exploit the vulnerability in the target software.
A Tool for Pro-active Defense Against the Buffer Overrun Attack D. Bruschi, E. Rosti, R. Banfi Presented By: Warshavsky Alex.
EC week Review. Rules of Engagement Teams selected by instructor Host will read the entire questions. Only after, a team may “buzz” by raise of.
CSE 592 INTERNET CENSORSHIP (FALL 2015) LECTURE 22 PHILLIPA GILL - STONY BROOK U.
On the Effectiveness of Address-Space Randomization Hovav Shacham, Matthew Page, Ben Pfaff, Eu-Jin Goh, Nagendra Modadugu, Dan Boneh.
Buffer Overflows Taught by Scott Coté.-. _ _.-. / \.-. ((___)).-. / \ /.ooM \ / \.-. [ x x ].-. / \ /.ooM \ -/ \ /-----\-----/---\--\ /--/---\-----/-----\ / \-
Web Security Firewalls, Buffer overflows and proxy servers.
Foundations of Network and Computer Security J J ohn Black CSCI 6268/TLEN 5550, Spring 2013.
Security Attacks Tanenbaum & Bo, Modern Operating Systems:4th ed., (c) 2013 Prentice-Hall, Inc. All rights reserved.
Slides by Kent Seamons and Tim van der Horst Last Updated: Nov 11, 2011.
Lect 8 Tahani al jehain. Types of attack Remote code execution: occurs when an attacker exploits a software and runs a program that the user does not.
VM: Chapter 7 Buffer Overflows. csci5233 computer security & integrity (VM: Ch. 7) 2 Outline Impact of buffer overflows What is a buffer overflow? Types.
@Yuan Xue Worm Attack Yuan Xue Fall 2012.
Chapter 7: Using Network Clients The Complete Guide To Linux System Administration.
TCP/IP Protocol Suite 1 Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Internet Protocol Version4 (IPv4)
Shellcode COSC 480 Presentation Alison Buben.
Mitigation against Buffer Overflow Attacks
/50 /60 /40 /30 A Tale of Two Clients
CS 465 Buffer Overflow Slides by Kent Seamons and Tim van der Horst
Software Security Lesson Introduction
Format String.
Lecture 3: Secure Network Architecture
Crisis and Aftermath Morris worm.
ITIS 6167/8167: Network and Information Security
Format String Vulnerability
Return-to-libc Attacks
Presentation transcript:

ANCS 2006 Scalable Network-based Buffer Overflow Attack Detection Fu-Hau Hsu Department of Computer Science and Information Engineering National Central University Taoyuan, Taiwan, R.O.C. Fanglu Guo Symantec Research Laboratory Cupertino, CA, U.S.A. Tzi-cker Chiueh Computer Science Department Stony Brook University Stony Brook, NY, U.S.A.

ANCS 2006 Virulence of Buffer Overflow Attacks Buffer overflow attack is arguably the most widely used and thus most dangerous attack method used today. Most Internet Worms use it to proliferate themselves. It accounts for more than 50% of all the security vulnerabilities recorded by CERT.

ANCS 2006 Proposed Solutions Compiler Transformation Stack Guard, RAD, Address Obfuscation Library Rewriting OS Non-executable Stack Instruction Set Hardware AMD Athlon-64

ANCS 2006 Discrepancy between Theory and Practice In theory, these efforts have largely solved the buffer overflow attack problem. In practice, however, new buffer overflow vulnerabilities are still discovered and reported on a routine basis.

ANCS 2006 substantial modification substantial resistance Why?

ANCS 2006 A Solution to the above Dilemma -- Nebula Nebula A network-based buffer overflow attack detection mechanism Observe the network traffic only to detect BOAs Currently version is developed for Linux paltforms.

ANCS 2006 Existing Network-based Intrusion Detection System (NIDS) Misuse intrusion detection Zero-day BOAs Labor-Intensive Solution: automatically signature-generating approaches Anomaly intrusion detection False Positive

ANCS 2006 Generalized Signature

ANCS 2006 Two Factors for a Successful Buffer Overflow-style Attack A successful buffer overflow-style attack should be able to overflow the right place (e.g. the place to hold a return address with the correct value (e.g. the address of injected code entry point)).

ANCS 2006 Non-predicable Offset and Entry Point Address buffer where the overflow start injected code return address offset between the beginning of the overflowed buffer and the overflow target. address of injected code entry point. The offset and the entry point address are non-predicable. They can not decided by just looking the source code or local binary code.

ANCS 2006 Non-predicable Offset For performance concerns, most compilers don’t allocate memory for local variables in the order they appear in the source code, sometimes some space may be inserted between them. (Source Code doesn’t help) Different compiler/OS uses different allocation strategy. (Local binaries don’t help) Address obfuscation insert random number of space between local variables and return address. (Super good luck may help)

ANCS 2006 Non-predicable Entry Point Address 0xbfffffff system data environment variables argument strings env pointers argv pointers argc webserver –a –b security command line arguments and environment variables Function main()’s stack frame

ANCS 2006 Strategies Used by Attackers to Increase Their Success Chance Repeat address patterns. Insert NOP (0x90) operations before the entry point of injected code.

ANCS 2006 Indispensable Elements of BO- style Attacks ‘ The Address ’ For buffer overflow attacks, it is the address of the entry point of injected code.

ANCS 2006 Linux Process Memory Layout 0xc xffffffff kernel address space user stack 8M %esp for Shared libraries, including libc functions brk run-time heap data and code 0x address space of addresses of injected code and frame pointers (Stack Address Zone)

ANCS 2006 Size of Stack Address Zone The default maximum size of a process’s user space stack is 8 Mbytes. However, according to Ditzel et al., the average function frame size is 28 bytes. Therefore, the majority of program are not supposed to use a 2Mbyte stack. In our test, a 8k stack is enough to identify all 10 remote exploit strings.

ANCS 2006 Repeating Times and Values of Return Addresses 2k stack --- 0xbffffffff ~ 0xbfffe000

ANCS 2006 A Property of Stack Addresses The leading byte of any words that contain a stack address corresponds to a non- printable ASCII character.

ANCS 2006 Generalized Signature Signature of a stack smashing buffer overflow attack : If a sub-string of a traffic string could be interpreted as a stack address that repeats 3 or more times, it is alarmed as a buffer overflow attack string.

ANCS 2006 Contextual Analysis

ANCS 2006 Bypassing Detection Patient attackers could bypass detection based on repeating address signature by repeating addresses no more than 2 times. PS: All the 10 remote exploit code we tested repeat at least 4 times. Attackers repeat the addresses to increase their chance to success. In other words, it is very likely that without the repeat, attackers will fail many times before getting a successful one.

ANCS 2006 Unsuccessful Attacks Buffer overflow-style attacks will destroy targeted process’s address space which in turn usually will crash the attacked process. In order to recycle valuable system resources, OS will close the sockets opened by crash processes automatically. On both Linux and Windows, when a program is crashed, the OS will terminate all the program’s pending socket connections by sending out an RST packet to the communicating hosts on its behalf.

ANCS 2006 Server Termination Signature After forwarding a sub-string which could be interpreted as a single stack address, Nebula detects that the server closes the TCP connection without sending any data, then the traffic string is deemed as a buffer overflow attack string. Future traffic coming from the same hosts will be blocked or examined thoroughly.

ANCS 2006 Will Normal Traffic Behavior the Same Way? HTTP Protocol (RFC 2616) works in the request-reply way. (After the request, there will be a reply before the server close the connection) SMTP protocol (RFC 2821), for , and FTP protocol(RFC 959) use QUIT command to close a connection. (QUIT can not be interpreted as a stack address.)

ANCS 2006 Payload Bypassing

ANCS 2006 Payload Bypassing Payload bypassing tries to avoid packet analysis for as much traffic as possible. Because most buffer overflow attacks take place during the exchange of control messages, it is safe to ignore the bulk of data that is downloaded as uninterpreted bytes. For example, in an FTP session, data transferred over the data connection can never be used to mount a buffer overflow attack against the FTP program because the FTP program does not interpret them.

ANCS 2006 Internet Traffic Statistic From CacheLogic’s measurement on USA, Europe, and Asia backbone in June 2004, HTTP and P2P packets accounted for more than 70% of the total traffic.

ANCS 2006 Percentage of Payload Percentage of payload in the traffic when each of the four protocols that Nebula can recognize is used to transfer files of a total size of 1.22 Gbytes.

ANCS 2006 Number of False Positives without Payload Bypassing Number of false positives under the our sample as reported by Nebula. The minimal number of times the attack pattern is repeated is assumed to be 1, 2, 3 or 10, and the stack size tested is 2Mbytes, 16Kbytes, or 8Kbytes. In each entry the left is the number of false positives for RTL attacks, whereas the right is the number of false positives for CI attacks. The sample includes TCP connections and about Gbytes of data.

ANCS 2006 Number of False Positives with Payload Bypassing The number of false positives in the test traffic associated with different protocols after applying payload bypassing is negligible even when the attack pattern repetition count is 1.

ANCS 2006 Throughput Comparison The throughput of Nebula under a test HTTP connection when different options are turned on. With payload bypassing, Nebula can perform buffer overflow attack detection and still achieve a throughput higher than a generic Linux router.