1. CS703 - Advanced Operating Systems By Mr. Farhan Zaidi

2. Lecture No. 43

3. Overview of today’s lecture Types of buffer overflow attacks Methods for fighting buffer overflows StackGuard and PointGuard Libsafe Address obfuscation Viruses and worms Mobile code security & sandboxing Java security

4. Causing program to exec attack code Stack smashing attack:  Override return address in stack activation record by overflowing a local buffer variable. Function pointers: (used in attack on Linux superprobe )  Overflowing buf will override function pointer. Longjmp buffers: longjmp(pos) (used in attack on Perl 5.003)  Overflowing buf next to pos overrides value of pos. Heap or stack buf[128] FuncPtr

5. Finding buffer overflows Hackers find buffer overflows as follows:  Run web server on local machine.  Issue requests with long tags. All long tags end with “$$$$$”.  If web server crashes, search core dump for “$$$$$” to find overflow location. Some automated tools exist. (eEye Retina, ISIC).

6. Preventing buf overflow attacks Main problem:  strcpy(), strcat(), sprintf() have no range checking.  “Safe” versions strncpy(), strncat() are misleading strncpy() may leave buffer unterminated. Defenses:  Type safe languages (Java, ML). Legacy code?  Mark stack as non-execute.  Static source code analysis.  Run time checking: StackGuard, Libsafe, SafeC, (Purify).  Black box testing (e.g. eEye Retina, ISIC ).

7. Marking stack as non-execute Basic stack exploit can be prevented by marking stack segment as non-executable or randomizing stack location.  Code patches exist for Linux and Solaris. Problems:  Does not block more general overflow exploits: Overflow on heap: overflow buffer next to func pointer.  Some apps need executable stack (e.g. LISP interpreters). Patch not shipped by default for Linux and Solaris.

8. Run time checking: StackGuard Many many run-time checking techniques … Solutions 1: StackGuard (WireX)  Run time tests for stack integrity.  Embed “canaries” in stack frames and verify their integrity prior to function return. strretsfplocal top of stack canarystrretsfp local canary Frame 1Frame 2

9. Canary Types Random canary:  Choose random string at program startup.  Insert canary string into every stack frame.  Verify canary before returning from function.  To corrupt random canary, attacker must learn current random string. Terminator canary: Canary = 0, newline, linefeed, EOF  String functions will not copy beyond terminator.  Hence, attacker cannot use string functions to corrupt stack.

10. StackGuard (Cont.) StackGuard implemented as a GCC patch.  Program must be recompiled. Minimal performance effects: 8% for Apache. Newer version: PointGuard.  Protects function pointers and setjmp buffers by placing canaries next to them.  More noticeable performance effects. Note: Canaries don’t offer full protection.  Some stack smashing attacks can leave canaries untouched.

11. Run time checking: Libsafe Solutions 2: Libsafe (Avaya Labs)  Dynamically loaded library.  Intercepts calls to strcpy (dest, src) Validates sufficient space in current stack frame: |frame-pointer – dest| > strlen(src) If so, does strcpy. Otherwise, terminates application.

12. More methods … Address obfuscation. (Stony Brook ’03)  Encrypt return address on stack by XORing with random string. Decrypt just before returning from function.  Attacker needs decryption key to set return address to desired value. Randomize location of functions in libc.  Attacker cannot jump directly to exec function.

13. Viruses and worms External threat  code transmitted to target machine  code executed there, doing damage Goals of virus writer  quickly spreading virus  difficult to detect  hard to get rid of Virus = program can reproduce itself  attach its code to another program  additionally, do harm

14. How Viruses Work (1) Virus written in assembly language Inserted into another program  use tool called a “dropper” Virus dormant until program executed  then infects other programs  eventually executes its “payload”

15. How Viruses Work An executable program With a virus at the front With the virus at the end With a virus spread over free space within program

16. How Viruses Spread Virus placed where likely to be copied When copied  infects programs on hard drive, floppy  may try to spread over LAN Attach to innocent looking email  when it runs, use mailing list to replicate

17. Antivirus and Anti-Antivirus Techniques Signature based Integrity checkers Behavioral checkers Virus avoidance  good OS  install only shrink-wrapped software  use antivirus software  do not click on attachments to email  frequent backups Recovery from virus attack  halt computer, reboot from safe disk, run antivirus

18. Mobile Code (1) Sandboxing (a) Memory divided into 1-MB sandboxes (b) One way of checking an instruction for validity

19. Mobile Code (2) Applets can be interpreted by a Web browser

20. Mobile Code (3) How code signing works

21. Java Security (1) A type safe language  compiler rejects attempts to misuse variable Checks include … 1. Attempts to forge pointers 2. Violation of access restrictions on private class members 3. Misuse of variables by type 4. Generation of stack over/underflows 5. Illegal conversion of variables to another type