1 Security Chapter The security environment 9.2 Basics of cryptography 9.3 User authentication 9.4 Attacks from inside the system 9.5 Attacks from outside the system 9.6 Protection mechanisms 9.7 Trusted systems

2 The Security Environment Threats Security goals and threats

3 Intruders Common Categories 1.Casual prying by nontechnical users 2.Snooping by insiders 3.Determined attempt to make money 4.Commercial or military espionage

4 Accidental Data Loss Common Causes 1.Acts of God -fires, floods, wars 2.Hardware or software errors -CPU malfunction, bad disk, program bugs 3.Human errors -data entry, wrong tape mounted

5 Basics of Cryptography Relationship between the plaintext and the ciphertext

6 Monoalphabetic substitution –each letter replaced by different letter Given the encryption key, –easy to find decryption key Secret-key crypto called symmetric-key crypto Secret-Key Cryptography

7 Public-Key Cryptography All users pick a public key/private key pair –publish the public key –private key not published Public key is the encryption key –private key is the decryption key

8 One-Way Functions Function such that given formula for f(x) –easy to evaluate y = f(x) But given y –computationally infeasible to find x

9 Digital Signatures Computing a signature block What the receiver gets (b)

10 User Authentication Basic Principles. Authentication must identify: 1.Something the user knows 2.Something the user has 3.Something the user is This is done before user can use the system

11 Authentication Using Passwords (a) A successful login (b) Login rejected after name entered (c) Login rejected after name and password typed

12 Authentication Using Passwords How a cracker broke into LBL –a U.S. Dept. of Energy research lab

13 Authentication Using Passwords The use of salt to defeat precomputation of encrypted passwords Salt Password,,,,

14 Authentication Using a Physical Object Magnetic cards –magnetic stripe cards –chip cards: stored value cards, smart cards

15 Authentication Using Biometrics A device for measuring finger length.

16 Countermeasures Limiting times when someone can log in Automatic callback at number prespecified Limited number of login tries A database of all logins Simple login name/password as a trap –security personnel notified when attacker bites

17 Operating System Security Trojan Horses Free program made available to unsuspecting user –Actually contains code to do harm Place altered version of utility program on victim's computer –trick user into running that program

18 Login Spoofing (a) Correct login screen (b) Phony login screen

19 Logic Bombs Company programmer writes program –potential to do harm –OK as long as he/she enters password daily –ff programmer fired, no password and bomb explodes

20 Trap Doors (a) Normal code. (b) Code with a trapdoor inserted

21 Buffer Overflow (a) Situation when main program is running (b) After program A called (c) Buffer overflow shown in gray

22 Generic Security Attacks Typical attacks Request memory, disk space, tapes and just read Try illegal system calls Start a login and hit DEL, RUBOUT, or BREAK Try modifying complex OS structures Try to do specified DO NOTs Convince a system programmer to add a trap door Beg admin's sec’y to help a poor user who forgot password

23 Famous Security Flaws The TENEX – password problem (a)(b)(c)

24 Design Principles for Security 1.System design should be public 2.Default should be n access 3.Check for current authority 4.Give each process least privilege possible 5.Protection mechanism should be -simple -uniform -in lowest layers of system 6.Scheme should be psychologically acceptable And … keep it simple

25 Network Security External threat –code transmitted to target machine –code executed there, doing damage Goals of virus writer –quickly spreading virus –difficult to detect –hard to get rid of Virus = program can reproduce itself –attach its code to another program –additionally, do harm

26 The Internet Worm Consisted of two programs –bootstrap to upload worm –the worm itself Worm first hid its existence Next replicated itself on new machines

27 Mobile Code (1) Sandboxing (a) Memory divided into 1-MB sandboxes (b) One way of checking an instruction for validity

28 Mobile Code (2) Applets can be interpreted by a Web browser

29 Mobile Code (3) How code signing works

30 Java Security (1) A type safe language –compiler rejects attempts to misuse variable Checks include … 1.Attempts to forge pointers 2.Violation of access restrictions on private class members 3.Misuse of variables by type 4.Generation of stack over/underflows 5.Illegal conversion of variables to another type

31 Java Security (2) Examples of specified protection with JDK 1.2

32 Protection Mechanisms Protection Domains (1) Examples of three protection domains

33 Protection Domains (2) A protection matrix

34 Protection Domains (3) A protection matrix with domains as objects

35 Access Control Lists (1) Use of access control lists of manage file access

36 Access Control Lists (2) Two access control lists

37 Capabilities (1) Each process has a capability list

38 Cryptographically-protected capability Generic Rights 1.Copy capability 2.Copy object 3.Remove capability 4.Destroy object Capabilities (2) ServerObjectRightsf(Objects, Rights, Check)

39 Trusted Systems Trusted Computing Base A reference monitor

40 Formal Models of Secure Systems (a) An authorized state (b) An unauthorized state

41 Multilevel Security (1) The Bell-La Padula multilevel security model

42 Multilevel Security (2) The Biba Model 1.Principles to guarantee integrity of data 1.Simple integrity principle process can write only objects at its security level or lower 2.The integrity * property process can read only objects at its security level or higher

43 Orange Book Security (1) Symbol X means new requirements Symbol -> requirements from next lower category apply here also

44 Orange Book Security (2)

45 Covert Channels (1) Client, server and collaborator processes Encapsulated server can still leak to collaborator via covert channels

46 Covert Channels (2) A covert channel using file locking

47 Covert Channels (3) Pictures appear the same Picture on right has text of 5 Shakespeare plays –encrypted, inserted into low order bits of color values Zebras Hamlet, Macbeth, Julius Caesar Merchant of Venice, King Lear

48 The access control mess Modify user ids using setuid, seteuid, setresuid, setreid –Real user id: identifies owner of process –Effective user id: used in access control decisions –Saved user id: stored a previous id for restoration Similarly for group ids fsuid – access control for file system in linux

49 Why effective user id? User process needs extra privileges Permission to read password file –e.g., passwd –-rwsr-xr-x. 1 root root Feb /usr/bin/passwd –The owner is root; –setuid program so that any user can run the program

50 Other setuid root programs -rwsr-xr-x 1 root root Feb Xorg -rwsr-xr-x. 1 root root Jan at -rwsr-xr-x. 1 root root Dec chage -rws--x--x. 1 root root Feb chfn -rws--x--x. 1 root root Feb chsh -rwsr-xr-x. 1 root root Jul crontab -rwsr-xr-x. 1 root root Dec gpasswd -rwsr-xr-x 1 root root Mar ksu -rwsr-xr-x. 1 root root Dec newgrp -rwsr-xr-x. 1 root root Feb passwd -rwsr-xr-x 1 root root Jun pkexec

51 Void undo_setuid (void) { int status; #ifdef _POSIX_SAVED_IDS status = seteuid (ruid); #else status = setreuid (euid, ruid); #endif if (status < 0) { fprintf (stderr, "Couldn't set uid.\n"); exit (status); } Void do_setuid (void) { int status; #ifdef _POSIX_SAVED_IDS status = seteuid (euid); #else status = setreuid (ruid, euid); #endif if (status < 0) { fprintf (stderr, "Couldn't set uid.\n"); exit (status); }

52 Usage of setuid root program int main (void) { /* Remember the real and effective user IDs. */ ruid = getuid (); euid = geteuid (); undo_setuid (); /* drop the privileges */ /* do unprivileged actions */ do_setuid(); /* step up privilege */ /* do privileged actions */ undo_setuid(); /* drop the privileges again */ Continue with unprivileged actions. s... }

53 The problem with setuid calls Poor design Insufficiently documented Widely misunderstood Result: many security vulnerabilities setuid demystified, Chen et al, USENIX Security 2002.

54 Early Unix Only two user ids, real and effective If effective uid is 0 Setuid can set both ruid and euid Otherwise Setuid can only set euid to ruid Cannot drop privileges temporarily

55 System V: seteuid If the euid was 0, seteuid can set euid to any userid Otherwise euid can be set to either ruid or suid

56 BSD: setreuid If the effective uid was zero, then the real uid and effective uid could be set to any user ID. Otherwise, either the real uid or the effective uid could be set to value of the other one.

57 Modern Unix: setresuid Direct manipulation of suid Setresuid accomplishes the task Preferred system call –Absence of side-effects –e.g., seteuid in System V has a side effects

58 Setuid spec

59 Appropriate privileges Solaris –Effective uid is 0 –POSIX_SAVED_IDS is defined. FreeBSD –Effective uid is 0 –Newuid is geteuid() –POSIX_SAVED_IDS is not defined Linux –SetUID capability; outside these three ids

60 Different system calls Setresuid –Euid is 0 –The new uid is one of the other three –Set all the uids or nothing –Preferred system call Seteuid –Changes euid and leaves other two as is –Linux or Solaris: neweuid to any of the three ids –FreeBSD: neweuid should be one of the other two uids 100, 200, 100

61 System calls Setreuid –Modifies reuid, euid –In some cases, modifies suid Linux, solaris: –Swap ruid and euid Freebsd: –Sometimes fails, (100,200,100)

62 Setuid Most confusing Linux/Solaris –Newuid equals ruid or suid, if euid is not 0 –setuid(geteuid()) can sometimes fail Always succeeds in freebsd Differs between privileged and unprivileged processes –Euid is 0 => all three uids to newuid –Otherwise, only euid is set to newuid

63 setfsuid: maintain invariant on fsuid and euid

64 Extract the model: run as root

65 Extract the model

66

67 Applications of formal model Verify accuracy of man pages Identifying implementation differences Detecting inconsistency with OS Kernel –Fsuid example Checking proper usage of uid-setting system calls

68 Misuse of setuid Sendmail installed sendmail binary as setuid root executable Non-root user: r, 0, 0 Dropped privileges permanently –setuid(getuid()) Bug in linux kernel –Changed only euid and not suid

69 Buffer overflow attack setuid(getuid()) –Does not change suid Run non-privileged actions Malicious user overflows the buffer Run setreuid(-1,0) –Process now has root privileges due to euid=0 –Spawn a shell

70 General Guidelines Select an appropriate system call –setresuid Proper order of system calls –Drop group privileges before user privileges Verifying proper execution of system calls –Checking return codes –Checking user ids –Verifying failures

71 Buffer overflow (Wenliang Du, Syracuse)

72 Exploiting buffer overflow vulnerability Injecting the malicious code –Overflowing the buffer Jumping to the malicious code Writing malicious code

73 Jumping to the code

74 Where does malicious code start? Copy setuid program and run it with own privileges –Get address of buffer –Address of buffer in setuid run will be closer to this value Multiple guesses Stack usually starts at same location Stack is not very deep –Add noops

75 Executing the malicious code char *name[2]; name[0] = ‘‘/bin/sh’’; name[1] = NULL; execve(name[0], name, NULL); –Where to store /bin/sh and how to locate it Push it onto stack –Null will cause strcpy to stop Use another operation that mimics noop

76 Counter measures Apply Secure Engineering Principles –Strong type languages – Java, C# –Use safe library functions – strncpy StackShield –Copy away return address to non-overflowable area StackGuard –Mark the boundary of the buffer

77 OS based approaches Address Space Randomization –Randomize start of stack and heap –# sysctl -w kernel.randomize_va_space=2 // Enable Randomization –# sysctl -w kernel.randomize_va_space=0 // Disable Randomization Non-executable stack –Configure stack to be non-executable –# sysctl -w kernel.exec-shield=1 // Enable ExecShield –# sysctl -w kernel.exec-shield=0 // Disable ExecShield

78 Return-to-libc attack Invoke shell without using injected code Copy of libc is already in memory –Accessed by all applications e.g., function system(“/bin/sh”); Change return address –Point it to system call

79 Challenges How to find location of system? –Libc always loaded at the same memory address. How to find location of /bin/sh? –Insert into stack and guess address –Create an environment variable –System uses /bin/sh in its own code How to pass the second location to the first function?

80

81 Safety property len(s) : length of some string, s alloc(s) : allocation to the string s len(s) <= alloc(s) If len(s) is [a,b] and alloc(s) is [c,d] If b < c, buffer overflow never occurs If a > d, buffer overflow always occurs If the ranges overflow, possibility of a buffer overflow.

82 Constraints

83 Privilege Escalation Attacks Cron daemon Allows users to schedule work to be done at periodic time intervals Runs as root Directory with commands to run User cannot write to directory Attacker Set working directory to cron directory Crash itself; core dumps in cron directory Dump made by system and hence written Valid set of commands in core

84 Malware Backdoor installed on machine Zombie Botnet – collection of zombies Install a keylogger Ability to make malware run on client machines is the key

85 Trojan Horse Trick users to download the application: Games Music players Program starts Malware is executed Delete, encrypt, modify files... No need to break into victim's machine Mistyped executable names

86 Virus Damage Scenarios Blackmail Denial of service as long as virus runs Permanently damage hardware Target a competitor's computer –do harm –espionage Intra-corporate dirty tricks –sabotage another corporate officer's files

87 How Viruses Work (1) Virus written in assembly language Inserted into another program –use tool called a “dropper” Virus dormant until program executed –then infects other programs –eventually executes its “payload”

88 Viruses Reproduce itself by attaching its code to another program Companion virus Run when program is run. Executable program viruses Parasitic viruses

89 How Viruses Work (2) Recursive procedure that finds executable files on a UNIX system Virus could infect them all

90 How Viruses Work (3) An executable program With a virus at the front With the virus at the end With a virus spread over free space within program

91 How Viruses Work (4) After virus has captured interrupt, trap vectors After OS has retaken printer interrupt vector After virus has noticed loss of printer interrupt vector and recaptured it

92 How Viruses Spread Virus placed where likely to be copied When copied –infects programs on hard drive, floppy –may try to spread over LAN Attach to innocent looking –when it runs, use mailing list to replicate

93 Antivirus and Anti-Antivirus Techniques (a) A program (b) Infected program (c) Compressed infected program (d) Encrypted virus (e) Compressed virus with encrypted compression code

94 Antivirus and Anti-Antivirus Techniques Examples of a polymorphic virus All of these examples do the same thing

95 Antivirus and Anti-Antivirus Techniques Integrity checkers Behavioral checkers Virus avoidance –good OS –install only shrink-wrapped software –use antivirus software –do not click on attachments to –frequent backups Recovery from virus attack –halt computer, reboot from safe disk, run antivirus