1. Intrusion Detection Host Vulnerability and Exploits, Common Attacks on Hosts

2. CS795-Fall 20102 Why Care About Hosts? Most Attacks/Intrusions Have Targeted Hosts –Break-in, penetration –Root privilege compromise –Steal, delete, modify and fabricate information in server Why? –Hosts are more interesting It has (almost) all the sensitive and useful information –Medical record –Payroll information –Classified information –Hosts have all the executables It has potentially more vulnerabilities It is easier for the intruders to exploit with many executables

3. CS795-Fall 20103 Common Attack on Hosts Gain Unauthorized Access to Host –User level Could impersonate that user, change, delete or forge information –Root level Could do everything to the host – worst possible scenario Denial of Service Provided by the Host –Denial of use of a host completely Disable the mail server –Denial of use of an application Disable the online stock trading –Denial of use of data Make the financial record inaccessible to users What Else?

4. CS795-Fall 20104 DoS: Web Server Attacks Many DoS Attacks Are Against Web Servers –Attacker sends enormous amount of bogus requests to the web server i.e. Syn-flood attack –Attacker sends a request consisting of thousands of ‘/’s. Some servers go belly up at this How to Detect? How To Handle This Kind of DoS? –Shutdown the web server?

5. CS795-Fall 20105 DoS: Mailbomb Exploits The Open-Door Nature of Email System –Mail server is supposed to receive emails –Attacker sends thousands of huge junk emails Fill up disks, overflow the quotas Deny access to emails Cause legitimate emails lost Usually done by some automated tools Mailbomb Is Different From Spam –There is no particular desire to have the email read, responded, or even necessarily received –The goal is to jam the email server and make is unusable How To Detect This? How To Handle This?

6. CS795-Fall 20106 DoS: Resource Hogging Resource Hogs –Programs that uses up the resources of the machine Fill up disks Use all the memory Use all the CPU cycle –Could be some executable downloaded –Code Wars was a game Let opponents write programs that would use up all the resources, until the opponent’s code was unable to run Detection Is Not Difficult –Except for memory leak How To Handle This?

7. CS795-Fall 20107 Unauthorized Access to Host Widely Considered –A much more serious threat than DoS All the sensitive information Impersonation –More difficult to detect than DoS Attack could come from inside Attacker could have the legitimate user name and password –Steal/crack password Attacker may gain access through backdoor

8. CS795-Fall 20108 Ways To Gain Unauthorized Access to Host Steal Account –Looking over one’s shoulder –Social engineering/Phishing –Sniffing –Spyware, key logger –Guessing the password –Cracking the password –What else?

9. CS795-Fall 20109 Ways To Gain Unauthorized Access to Host Abuse The Guest Account –Many systems have default guest account that requires no password Anonymous FTP Trojan Horse Exploit Race Conditions –Usually used for gaining root privilege from normal user privilege Exploit Buffer Overflow –Could gain root access remotely

10. CS795-Fall 201010 Social Engineering/Phishing Tricking People into Giving Access Example –“Hello, this Smith, the Vice President of marketing. I need to update my photo in the corporate directory, and I’ve fogot my password” –“Hello, I’m a customer support from Citibank, and we are upgrading the security mechanism of our customer account management. Please login into web site www.phishing.org to verify the status of your account”www.phishing.org How to detect this automatically?

11. CS795-Fall 201011 Account Guessing Attacker Can Simply Guess The Password –Knowing something about the user helps Kids/spouse/mother’s maiden name Birthdays Social security numbers –Sometimes just need to try a few obvious things to get in How to Detect This? –Count the unsuccessful login attempts! –What if the attacker is really lucky such that he/she gets the right password in the first try?

12. CS795-Fall 201012 Sniffing Wiretap the network Sniffer the wireless There Are Tools –That automatically look for user IDs and password from the sniffered trace How To Prevent? –Use encryption –Never send password in plaintext How To Detect?

13. CS795-Fall 201013 Trojan Horse A Program –That appears to do one thing but secretly does another Could Be Planted From Local –Running a trojan that simulate the login screen of a computer in a public lab –The trojan records the first login, and prompts the victim with error message –The trojan exits and let the real login program runs Could Be Planted From Remote –The attacker send an email with an attachment of a really cool screen saver –Once the victim is tricked into using it The screen saver opens a backdoor and allows the attacker access to the computer whenever the screen saver is running

14. CS795-Fall 201014 Trojan Horse Program with an expected and hidden effect –Appears normal/expected –hidden effect violates security policy User tricked into executing Trojan horse –Expects (and sees) expected behavior –Hidden effect performed with user’s authorization

15. CS795-Fall 201015 Cracking The Password In Unix –Password is not stored in plaintext, but in hashed form (user name, Hash(password)) –When a user enters the password, system computes Hash(password); a match on the hash would allow user to login Offline Guessing the Password –Steal the password file /etc/passwd Some cgi scripts let you do this through a web server –Run crack on the password file Try all possible passwords and compare the hashes of them with Hash(password) in the password file

16. CS795-Fall 201016 Cracking The Password Case Study Farmer and Venema performed the following study in 1993 –Tried to obtain password files from 656 hosts –Succeeded on 24 hosts –Tried to crack the passwords –Got 5 root passwords 259 passwords in total Access to 19 (out of 24) hosts

17. CS795-Fall 201017 Brute Force Password Cracking Assume the password consists of 6 characters from 80 possible choices Totally 2.62×10 11 possible passwords Assume a 1.7GHz computer can check 17,000,000 password per second It would take 15,420 seconds to check every possible password That’s a just a little over 4 hours!

18. CS795-Fall 201018 Brute Force Password Cracking Assume the password consists of 8 characters from 80 possible choices Totally 1.67×10 15 possible passwords Assume a 1.7GHz computer can check 17,000,000 password per second It would take 10 8 seconds to check every possible password That’s about 3 years Assume the computation power doubles every 18 months (Moore’s Law) –It would need another 19 years to be able to crack 8 character password in 4 hours

19. CS795-Fall 201019 Defense Against Password Cracking Use Strong Password –Longer –Not in dictionary Shadow the password –Only the root has access to the actual hashes of passwords Watch for Failed Login Attempts Make Sure There Are No Open Accounts Use Encryption for Access from Remote Is Password Cracking Detectable?

20. CS795-Fall 201020 Defense Against Password Cracking Question: –With increased number of accounts and passwords, how are you going to organize and remember them? Interesting approaches: –Site-Specific Password [HP Lab TR’03] –PwdHash [USENIX Security’05]

21. CS795-Fall 201021 Exploit Race Condition To Trick The Operating System into Letting You Access (Read, Write) a Root File, by Switching Files In the Middle of File Open –Create a temporary file –Open the temporary file to read or write –Between the permission check and the open, switch the file –Read or write the file Also Called Time-of-Check-to-Time-of-Use Flaw

22. CS795-Fall 201022 Exploit Race Condition Example –Create a temporary file /tmp/X –Open the temporary file to write. The OS will check if you have permission You do, since it is your file anyway! –Before the file is opened, but after the permission check, remove /tmp/X, and create a symbolic link to /etc/passwd! –/etc/password will be opened for writing –Write something to /etc/passwd –This would allow one to create new account (or modify existing account) in the password file!!!

23. CS795-Fall 201023 Exploit Race Condition / /tmp/etc /tmp/X/etc/passwd Create X in /tmp

24. CS795-Fall 201024 Exploit Race Condition / /tmp/etc /tmp/X/etc/passwd Access /tmp/X to write

25. CS795-Fall 201025 Exploit Race Condition / /tmp/etc /tmp/X/etc/passwd Link /tmp/X to /etc/passwd

26. CS795-Fall 201026 Exploit Race Condition / /tmp/etc /tmp/X/etc/passwd Write /tmp/X

27. CS795-Fall 201027 Buffer Overflow Buffer Overflow is Hacker’s Best Friend –The most common way of getting unauthorized access to a host Buffer Overflow Exists In Many Places –Many daemons (or server) run with root privilege –Once these root-privileged daemon is exploited with buffer over flow Attacker could get root access remotely!!! First Well-Known Buffer Overflow –Morris’s Worm in 1988

28. CS795-Fall 201028 Overview of Buffer Overflow Vulnerability Run-Time Memory Management of Executables –Use stack to hold local (or automatic) variables Return address of function calls (in x86) Programming Language C Emphasizes Performance –No check on buffer bounds strcpy() gets() etc. Overflow of Local Variables Could –Overwrite the return address –Control what to be run after current function

29. CS795-Fall 201029 Run-Time Stack Memory Organization Text (Initialized) Data (Uninitialized) Run-Time Stack Lower mem address Higher mem address text section Data-bss section SP Stack growth FP SP: Stack Pointer; FP: Frame Pointer

30. CS795-Fall 201030 Run-Time Stack Memory Management Procedure Prolog (Before A Function is Executed) –Advance SP to reserve space for parameters (if any) –Save the IP (Instruction Pointer) to stack as return address –Save current FP (so that is can be restored later) –Copy SP to FP to create a new FP –Advance SP to reserve space for local variables Procedure Epilog (Before A Function Exits) –Clean up the stack –Restore previous SP & FP

31. CS795-Fall 201031 Example C Program void function(int a, int b, int c) { char buffer1[5]; char buffer2[10]; } int main() { function(1,2,3); return 0; }

32. CS795-Fall 201032 Current Run-Time Stack Text (Initialized) Data (Uninitialized) Lower mem address Higher mem address text section Data-bss section SP FP SP: Stack Pointer; FP: Frame Pointer

33. CS795-Fall 201033 Reserve Space for Parameters Text (Initialized) Data (Uninitialized) c Lower mem address Higher mem address text section Data-bss section SP FP SP: Stack Pointer; FP: Frame Pointer b a

34. CS795-Fall 201034 Save Return Address & FP Text (Initialized) Data (Uninitialized) c Lower mem address Higher mem address text section Data-bss section SP FP SP: Stack Pointer; FP: Frame Pointer b a RET SFP

35. CS795-Fall 201035 Allocate New Stack Frame Text (Initialized) Data (Uninitialized) c Lower mem address Higher mem address text section Data-bss section SP FP SP: Stack Pointer; FP: Frame Pointer b a RET SFP

36. CS795-Fall 201036 Allocate Space for Local Variables Text (Initialized) Data (Uninitialized) c Lower mem address Higher mem address text section Data-bss section SP SP: Stack Pointer; FP: Frame Pointer b a RET SFP buffer1 buffer2 FP

37. CS795-Fall 201037 Overflow Buffer1 To Overwrite RET Text (Initialized) Data (Uninitialized) c Lower mem address Higher mem address text section Data-bss section SP SP: Stack Pointer; FP: Frame Pointer b a RET SFP buffer1 buffer2 FP My program somewhere

38. CS795-Fall 201038 Stack Smashing Attack Inject Attack Code –Overflow the buffer with string that is actually executable binary code native to the machine –Most common attack code is to spawn a shell Change the Return Address –Overwrite the return address with pointer to attack code Once the Current Function Returns –The attack code will get control with the same privilege of the original process

39. CS795-Fall 201039 StackGuard: Detection & Prevention of Buffer Overflow Attack Idea –When attacker overwrite the return address, the adjacent word is likely to be overwritten as well –Inject some special number, called canary word, adjacent to the return address –Before jump to the return address, check the canary word If it is changed, then there is buffer overflow If it is not changed, go ahead It can be done by compiler –No source code change is needed! –Just recompile the source code

40. CS795-Fall 201040 Code Injection Defense Security Extensions –Non-Executable Stack –MemGuard, StackGuard, … –Libsafe NX Protection –Hardware vs. Software Randomization –Address Space Layout Randomization –Instruction Set Randomization

41. Virus Checking Algorithms

42. CS795-Fall 201042 Problem Formulation The virus bit strings are collected {bs i | i =1,N}. Assume each string is K bits. Incoming string is I with a length of L bits. L > K Objective: Determine if I contains any bs i string. Stop at the first match. A match means that I is infected.

43. CS795-Fall 201043 Brute Force J=1 Compare I with bs J string. Slide bs J across I until match is found. Match found – done No match; J=J+1, repeat I: a b c f a b c c f d {L} bs J : c f a b c f {K} Worst case O(K.L)

44. CS795-Fall 201044 Random Compute signature of the K string Compute signature of each K substring of the L string Compare the two signature strings If signature match then virus Signature could be a simple mod operation O(K.L) – practical approach

45. CS795-Fall 201045 KMP Algorithm Pre-process the K string Compute repetition distance – this will correspond to a legit shift in case of mismatch c f a b c f {K} 5 5 0 0 0 0 a b c f a b c c f d {L} Pre processing O (K) Processing O (L)

46. CS795-Fall 201046 Automata Scheme Use automata to reduce computation Use K string to fix state transitions Correct sequence leads to match state

47. CS795-Fall 201047 Source: Cormen, Leiserson, Rivest, Stein, Introduction To Algorithms