1. 15.1 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Chapter 15: Security The Security Problem Program Threats System and Network Threats Cryptography as a Security Tool User Authentication Implementing Security Defenses Firewalling to Protect Systems and Networks Computer-Security Classifications

2. 15.2 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Objectives To discuss security threats and attacks To explain the fundamentals of encryption, authentication, and hashing To examine the uses of cryptography in computing To describe the various countermeasures to security attacks

3. 15.3 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts The Security Problem Security must consider external environment of the system, and protect the system resources Intruders (crackers) attempt to breach security Threat is potential security violation Attack is an attempt to breach security Attack can be accidental or malicious Easier to protect against accidental than malicious misuse

4. 15.4 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Security Violations Categories of security violations Breach of confidentiality: unauthorized reading of data Breach of integrity: unauthorized modification of data Breach of availability: unauthorized destruction of data Theft of service: unauthorized use of resources. A user may install a daemon that acts as a file server. Denial of service: involves preventing legitimate use of the system. Denial-of- service attacks. Methods used for breaching security: Masquerading (breach authentication): one participant pretends to be someone else Replay attack : replay old messages  Message modification : replay attack used along with message modification to escalate privileges. Man-in-the-middle attack: attacker sits in the middle of communication flow and masquerades as sender or receiver. Session hijacking: active communication session is intercepted. This may precede man-in-the-middle attack.

5. 15.5 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Standard Security Attacks

6. 15.6 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Security Measure Levels Security must occur at four levels to be effective: Physical: sites containing the computer systems must be physically secured Human: authorizing users must be done carefully  Avoid social engineering – Phishing: a legitimate looking email or web page misleads a user to enter confidential information – Dumpster diving: attempting to gather information from trash, phonebooks, finding notes containing passwords, etc. to gain unauthorized access to computer Operating System: The system must protect itself from accidental or intentional security breaches. Should handle problems such as runaway processes, stack overflow, etc.  Runaway process is a process that enters an infinite loop and spawns new processes which can cause an overflow of the proc table. Network: Much of the computer data flows on private leased lines, shared lines like Internet, telephone lines and wireless connections. Intercepting this data could be just as harmful as breaking into computer. Security chain is as weak as the weakest link in the chain. Deployment of higher level security and escalating intruder techniques is a cat-and-mouse-game.

7. 15.7 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Program Threats Trojan Horse Code segment that misuses its environment Exploits mechanisms for allowing programs written by users to be executed by other users Some variations of Trojan horse:  Spyware: – Downloads adds to display on the user’s system, creates pop- up browser windows when certain sites are visited, or capture information from the user and send it to a central site which is known as covert channels, in which surreptitious communication occurs. – An example: when you download and install a program on your windows machine, it might come with a spyware to collect information but also send spam emails to addresses collected from a central site. 80% of the spam emails were generated like this in 2004.

8. 15.8 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Program Threats… Trap Door The programmer leaves a hole in the software that only he/she is capable of using. For example, the code might check a specific user ID or password and might circumvent normal security procedures Could be included in a compiler Programmers have been arrested for embezzling from banks by including rounding errors in their code and having occasional half-cent credited to their accounts. Logic Bomb Program that initiates a security incident under certain circumstances  For example, an employer could write code that includes code that detects if he/she is employed still; if not employed it launches some malicious attack on the employer’s system.

9. 15.9 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Program Threats… Stack and Buffer Overflow Common way for an outsider to gain access to the target system. This exploits a bug in an existing program (overflow of either the stack or memory buffers) The attacker determines the vulnerability and writes a program to do the following: 1. Overflow an input field, command-line argument, or input buffer – for example on a network daemon – until it writes into the stack. 2. Overwrite the current return address on the stack with the address of the exploit code in step 3 below 3. Write a sample set of code for the next space in the stack that includes the commands that the attacker wishes to execute – for instance, spawn a shell. The result of the attack program’s execution will be a root shell or other privileged command execution.

10. 15.10 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Buffer overflow… For instance, if a webpage form expects a user name to be entered into a field, the attacker could send the user name, plus extra characters to overflow the buffer and reach the stack, plus new return address to load onto the stack, plus the code the attacker wants to run. When the buffer reading subroutine returns from execution, the return address is the exploit code and the code is run.

11. 15.11 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts C Program with Buffer-overflow Condition #include #define BUFFER_SIZE 256 int main(int argc, char *argv[]) { char buffer[BUFFER_SIZE]; if (argc < 2) return -1; else { strcpy(buffer,argv[1]); return 0; } For the above program, if the user supplies an argument that is more than 255 characters (1 character for the null terminator \0), buffer overflow occurs. A solution to this problem: Use strncpy instead of strcpy : i.e., strncpy(buffer, argv[1], sizeof(buffer)-1)

12. 15.12 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Layout of Typical Stack Frame

13. 15.13 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Modified Shell Code An attacker first writes short code like the following and compiles it and tweeks the assembly code to fit into the stack frame #include int main(int argc, char *argv[]) { execvp(‘‘\bin\sh’’,‘‘\bin\sh’’, NULL); return 0; }

14. 15.14 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Hypothetical Stack Frame Before attack After attack

15. 15.15 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts A Solution to the buffer overflow problem A solution is to have the CPU a feature that disallows execution of code in a stack section of memory. Recent versions of Sun’s SPARC chip include this setting, and recent versions of Solaris OS enable this feature. The return address still can be modified. Recent versions of AMD and Intel x86 chips also have a feature to prevent this type of attack. The use of this feature is supported in several x86 OSes such as Linux and Windows XP. The hardware implementation involves the use of a new bit in the page table, this bit is used to mark the associated page as nonexecutable.

16. 15.16 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Program Threats (Cont.) Viruses A code fragment embedded in a legitimate program Viruses are self-replicating and are designed to infect other programs Very specific to CPU architecture, operating systems and applications Usually borne via email and downloads A common form of virus uses Microsoft Office files which contain macros (or visual basic programs) that programs in office suite will execute automatically. Since these programs are under user’s own account they can perform any operation that the user is allowed such as deleting files.

17. 15.17 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Program Threats (Cont.) Once a virus reaches a computer, a program called Virus dropper inserts virus onto the system. The virus dropper itself is a Trojan horse whose main purpose is to install the virus. Many categories of viruses, literally many thousands of viruses File: A standard file virus appends itself to an existing file Boot: A boot virus infects the boot sector of the system so it executes every time the systems is booted Macro: Are written usually in a high level language, such as Visual Basic. These viruses are triggered when a program capable of executing the macro is run. Source code: A source code virus looks for the source code and modifies it to include the virus and to help spread the virus. Polymorphic: This virus changes itself each time it is installed to avoid detection by antivirus software. Encrypted: An encrypted virus includes decryption code along with the encrypted virus to avoid detection. The virus first decrypts and then executes Stealth: This attempts to modify part of the system that could be used to detect it. And there are others.

18. 15.18 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts A Boot-sector Computer Virus

19. 15.19 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts System and Network Threats These threats involve abuse of services and network connections. Worms – use spawn mechanism; standalone program Internet worm - by Robert Morris (1988) Exploited UNIX networking features (remote access) and bugs in finger and sendmail programs The worm consisted of two programs  A Grappling hook program and a main program  The Grappling hook connected to the machine where it originated and uploaded a copy of the main worm on to the hooked system. The main program proceeded to search for other machines to which the newly infected system could connect easily. Port scanning Automated attempt to connect to a range of ports on one or a range of IP addresses. This is not an attack but rather a means for determining the vulnerabilities of a system so the attacker can exploit the vulnerabilities Denial of Service Overload the targeted computer preventing it from doing any useful work Distributed denial-of-service (DDOS) attacks come from multiple sites at once

20. 15.20 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts The Morris Internet Worm

21. 15.21 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Cryptography as a Security Tool Broadest security tool available Source and destination of messages cannot be trusted without cryptography Means to constrain potential senders (sources) and / or receivers (destinations) of messages Ensure the integrity of messages Based on secrets (keys)

22. 15.22 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Secure Communication over Insecure Medium

23. 15.23 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Encryption Encryption algorithm consists of Set K of keys Set M of Messages Set C of ciphertexts (encrypted messages) A function E : K → (M→C). That is, for each k  K, E(k) is a function for generating ciphertexts from messages.  Both E and E(k) for any k should be efficiently computable functions. A function D : K → (C → M). That is, for each k  K, D(k) is a function for generating messages from ciphertexts.  Both D and D(k) for any k should be efficiently computable functions. An encryption algorithm must provide this essential property: Given a ciphertext c  C, a computer can compute m from c such that E(k)(m) = c only if it possesses D(k) Thus, a computer holding D(k) can decrypt ciphertexts to the plaintexts used to produce them, but a computer not holding D(k) cannot decrypt ciphertexts. Since ciphertexts are generally exposed (for example, sent on the network), it is important that it be infeasible to derive D(k) from the ciphertexts

24. 15.24 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Symmetric Encryption Same key is used to encrypt and decrypt a message E(k) can be derived from D(k), and vice versa DES has been the most commonly used symmetric block- encryption algorithm (created by US Govt) Encrypts a block of data at a time This has been broken Advanced Encryption Standard (AES), has replaced DES. RC4 is most common symmetric stream cipher, but known to have vulnerabilities Encrypts/decrypts a stream of bytes Key is a input to psuedo-random-bit generator  Generates an infinite keystream

25. 15.25 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Asymmetric Encryption Public-key encryption based on each user having two keys: public key – published key (k e, N) used to encrypt data private key – key (k d N) known only to individual user and is used to decrypt data Must be an encryption scheme that can be made public without making it easy to figure out the decryption scheme Most common is RSA block cipher. Its features depend on the following facts:  Efficient algorithm for testing whether or not a number is prime is not known  No efficient algorithm is known for finding the prime factors of a large (100s of digits long) number

26. 15.26 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts RSA (Cont.) Formally, it is computationally infeasible to derive (k d, N) from (k e, N) and so the encryption algorithm E and the public key (k e,N) need not be kept secret and can be widely disseminated (k e, N) (or just k e ) is the public key (k d, N) (or just k d ) is the private key N is the product of two large, randomly chosen prime numbers p and q (for example, p and q are 512 bits each) Encryption algorithm is E(k e, N)(m) = m k e mod N, where k e satisfies k e k d mod (p−1)(q −1) = 1 The decryption algorithm is then D(k d, N)(c) = c k d mod N

27. 15.27 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts RSA-Asymmetric Encryption Example For example. make p = 7and q = 13 We then calculate N = 7 ∗ 13 = 91 and (p−1)(q−1) = 72 We next select k e relatively prime to 72 and< 72, yielding 5 Finally,we calculate k d such that k e k d mod 72 = 1, yielding 29 We how have our keys Public key, k e, N = 5, 91 Private key, k d, N = 29, 91 Encrypting the message 69 with the public key results in the cyphertext 62 Cyphertext can be decoded with the private key Public key can be distributed in cleartext to anyone who wants to communicate with holder of public key

28. 15.28 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Encryption and Decryption using RSA Asymmetric Cryptography

29. 15.29 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Authentication Constraining set of potential senders of a message Complementary and sometimes redundant to encryption Also can prove message was not modified Algorithm components A set K of keys A set M of messages A set A of authenticators A function S : K → (M→ A)  That is, for each k  K, S(k) is a function for generating authenticators from messages  Both S and S(k) for any k should be efficiently computable functions A function V : K → (M× A→ {true, false}). That is, for each k  K, V(k) is a function for verifying authenticators on messages  Both V and V(k) for any k should be efficiently computable functions

30. 15.30 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Authentication (Cont.) For a message m, a computer can generate an authenticator a  A such that V(k)(m, a) = true only if it possesses S(k) Thus, computer holding S(k) can generate authenticators on messages so that any other computer possessing V(k) can verify them Computer not holding S(k) cannot generate authenticators on messages that can be verified using V(k) Since authenticators are generally exposed (for example, they are sent on the network with the messages themselves), it must not be feasible to derive S(k) from the authenticators

31. 15.31 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Authentication – Hash Functions Basis of authentication Creates small, fixed-size block of data (message digest, hash value) from m Hash Function H must be collision resistant on m Must be infeasible to find an m’ ≠ m such that H(m) = H(m’) If H(m) = H(m’), then m = m’ The message has not been modified Common message-digest functions include MD5, which produces a 128-bit hash, and SHA-1, which outputs a 160-bit hash and SHA-2.

32. 15.32 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Authentication - MAC Symmetric encryption used in message-authentication code (MAC) authentication algorithm Simple example: MAC defines S(k)(m) = f (k, H(m))  Where f is a function that is one-way on its first argument – k cannot be derived from f (k, H(m))  Because of the collision resistance in the hash function, reasonably assured no other message could create the same MAC  A suitable verification algorithm is V(k)(m, a) ≡ ( f (k,H(m)) = a)  Note that k is needed to compute both S(k) and V(k), so anyone able to compute one can compute the other

33. 15.33 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Authentication – Digital Signature Based on asymmetric keys and digital signature algorithm Authenticators produced are digital signatures In a digital-signature algorithm, computationally infeasible to derive S(k s ) from V(k v ) V is a one-way function Thus, k v is the public key and k s is the private key Consider the RSA digital-signature algorithm Similar to the RSA encryption algorithm, but the key use is reversed Digital signature of message S(k s )(m) = H(m) k s mod N The key k s again is a pair d, N, where N is the product of two large, randomly chosen prime numbers p and q Verification algorithm is V(k v )(m, a) ≡ (a k v mod N = H(m))  Where k v satisfies k v k s mod (p − 1)(q − 1) = 1

34. 15.34 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Authentication (Cont.) If encryption can prove the identity of the sender of a message, then why need separate authentication algorithms? Authentication algorithms generally require fewer computations Authenticator of a message is usually shorter than the message itself Sometimes want authentication but not confidentiality. For example, a company may provide a software patch and could “sign” that patch to prove that it came from the company. Can be basis for non-repudiation

35. 15.35 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Key Distribution Delivery of symmetric key is huge challenge Sometimes done out-of-band Asymmetric keys can proliferate – stored on key ring Even asymmetric key distribution needs care – man-in-the- middle attack

36. 15.36 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Man-in-the-middle Attack on Asymmetric Cryptography

37. 15.37 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Encryption Example - SSL Insertion of cryptography at one layer of the ISO network model (the transport layer) SSL – Secure Socket Layer (also called TLS) Cryptographic protocol that limits two computers to securely exchange messages with each other Very complicated, with many variations Used between web servers and browsers for secure communication (credit card numbers) The server is verified with a certificate assuring client is talking to correct server Asymmetric cryptography used to establish a secure session key (symmetric encryption) for bulk of communication during session Communication between each computer then uses symmetric key cryptography

38. 15.38 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts User Authentication Crucial to identify user correctly, as protection of systems depend on user ID User identity most often established through passwords, can be considered a special case of either keys or capabilities Also can include something user has and /or a user attribute Passwords must be kept secret. Some ways to protect passwords: Frequent change of passwords Use of “non-guessable” passwords Do not use the same password for different systems Log all invalid access attempts Passwords may also either be encrypted or allowed to be used only once

39. 15.39 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Implementing Security Defenses Defense in depth is most common security theory – multiple layers of security Security policy describes what is being secured Vulnerability assessment compares real state of system / network compared to security policy Intrusion detection endeavors to detect attempted or successful intrusions Signature-based detection characterizes dangerous behaviors and tries to detect only those behaviors. Anomaly detection spots differences from normal behavior  Can detect zero-day (i.e., previously unknown) attacks False-positives and false-negatives, a problem in this approach Virus protection Auditing, accounting, and logging of all or specific system or network activities

40. 15.40 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Firewalling to Protect Systems and Networks A network firewall is placed between trusted and untrusted hosts The firewall limits network access between these two security domains Can be tunneled or spoofed Tunneling allows disallowed protocol to travel within allowed protocol (i.e. telnet inside of HTTP) Firewall rules typically based on host name or IP address which can be spoofed Personal firewall is software layer on given host Can monitor / limit traffic to and from the host Application proxy firewall understands application protocol and can control them (i.e. SMTP) System-call firewall monitors all important system calls and apply rules to them (i.e. this program can execute that system call)

41. 15.41 Silberschatz, Galvin and Gagne ©2005 Operating System Concepts Network Security Through Domain Separation Via Firewall DMZ – Demilitarized Zone (a semitrusted semisecure network)