Hashes and Message Digests Nick Feamster CS 6262: Network Security Spring 2009

Hashes Hash is also called message digest One-way function: d=h(m) but no h’(d)=m Cannot find the message given a digest Cannot find m1, m2, where d1=d2 Arbitrary-length message to fixed-length digest Randomness any bit in the outputs ‘1’ half the time each output: 50% ‘1’ bits

Hash Algorithm Structure Most important modern hash functions follow the basic structure shown in this figure, Stallings Figure 11.9. This has proved to be a fundamentally sound structure, and newer designs simply refine the structure and add to the hash code length. Within this basic structure, two approaches have been followed in the design of the compression function, as mentioned previously, which is the basic building block of the hash function.

Hash and MAC Algorithms Hash Functions condense arbitrary size message to fixed size by processing message in blocks through some compression function either custom or block cipher based Message Authentication Code (MAC) fixed sized authenticator for some message to provide authentication for message by using block cipher mode or hash function Now look at important examples of both secure hash functions and message authentication codes (MACs). Traditionally, most hash functions that have achieved widespread use rely on a compression function specifically designed for the hash function. Another approach is to use a symmetric block cipher as the compression function. MACs also fall into two categories: some use a hash algorithm such as SHA as the core of the MAC algorithm, others use a symmetric block cipher in a cipher block chaining mode.

Secure Hash Algorithm SHA originally designed by NIST & NSA in 1993 was revised in 1995 as SHA-1 US standard for use with DSA signature scheme standard is FIPS 180-1 1995, also Internet RFC3174 nb. the algorithm is SHA, the standard is SHS based on design of MD4 with key differences produces 160-bit hash values recent 2005 results on security of SHA-1 have raised concerns on its use in future applications The Secure Hash Algorithm (SHA) was developed by the National Institute of Standards and Technology (NIST) and published as a federal information processing standard (FIPS 180) in 1993; a revised version was issued as FIPS 180-1 in 1995 and is generally referred to as SHA-1. The actual standards document is entitled Secure Hash Standard. SHA is based on the hash function MD4 and its design closely models MD4. SHA-1 produces a hash value of 160 bits. In 2005, a research team described an attack in which two separate messages could be found that deliver the same SHA-1 hash using 2^69 operations, far fewer than the 2^80 operations previously thought needed to find a collision with an SHA-1 hash [WANG05]. This result should hasten the transition to newer, longer versions of SHA.

Revised Secure Hash Standard NIST issued revision FIPS 180-2 in 2002 adds 3 additional versions of SHA SHA-256, SHA-384, SHA-512 designed for compatibility with increased security provided by the AES cipher structure & detail is similar to SHA-1 hence analysis should be similar but security levels are rather higher In 2002, NIST produced a revised version of the standard, FIPS 180-2, that defined three new versions of SHA, with hash value lengths of 256, 384, and 512 bits, known as SHA-256, SHA-384, and SHA-512. These new versions have the same underlying structure and use the same types of modular arithmetic and logical binary operations as SHA-1, hence analyses should be similar. In 2005, NIST announced the intention to phase out approval of SHA-1 and move to a reliance on the other SHA versions by 2010. See Stallings Table12.1 for comparative details of these algorithms.

Collisions: Birthday Paradox Compute probability of different birthdays Random sample of n people (birthdays) taken from k (365) days kn samples with replacement (k)n=k(k-1)…(k-n+1) sample without replacement Probability of no repetition: p = (k)n/kn  1 - n(n-1)/2k

How Many Bits for Hash? m bits, takes 2m/2 to find two with the same hash 64 bits, takes 232 messages to search (doable) Need at least 128 bits

Application: Authentication Alice to Bob: challenge rA Bob to Alice: MD(KAB|rA) Bob to Alice: rB Alice to Bob: MD(KAB|rB) Only need to compare MD results

Application: MAC Cannot just compute MD(m) MD(KAB|m) Allows concatenation with additional message: MD(KAB|m|m’) MD through chunk n depends on MD through chunks n-1 and the data in chunk n Problem: Arbitrary messages can be concatenated to the original Solution: Put secret at the end of message: MD(m| KAB)

Application: Encryption One-time pad compute bit streams using MD, K, and IV b1=MD(KAB|IV), bi=MD(KAB|bi-1), …  with message blocks Or mixing in the plaintext similar to cipher feedback mode (CFB) b1=MD(KAB|IV), c1= p1  b1 b2=MD(KAB| c1), c2= p2  b2

Application: Secret Key Input Unix password algorithm Compute hash of user password, store the hash (not the password), and compare the hash of user-input password. First 8 bytes of password used to form a secret key. Encrypt 0 with a DES-like algorithm (why not use a “system” key to encrypt the password?). Salt 12-bit random number formed from time and process ID. Determine bits to duplicate in the mangler when expanding from 32 to 48 bits. Salt stored with hashed result.

MD2 128-bit message digest Arbitrary number of bytes of message First pad to multiple of 16 bytes Append MD2 checksum (16 bytes) to the end The checksum is almost a MD, but not cryptographically secure by itself. Process whole message

MD2 Checksum One byte at a time, k  16 steps mnk: byte nk of message cn=(mnk  cn-1)  cn  : 0  41, 1  46, … Substitution on 0-255 (value of the byte)

MD2 Final Pass Operate on 16-byte chunks 48-byte quantity q: (current digest|chunk|digestchunk) 18 passes of massaging over q, one byte at a time: cn=(cn-1)  cn for n = 0, … 47; c-1 = 0 for pass 0; c-1 = (c47 + pass #) mod 256 After pass 17, use first 16 bytes as new digest 16  8 = 128

MD5: Message Digest Version 5 input Message Output 128 bits Digest

MD5 Box 512-bit message chunks (16 words) Initial F: (xy)(~x  z) 128-bit vector F: (xy)(~x  z) G:(x  z) (y ~ z) H:xy z I: y(x  ~z) +: binary sum xy: x left rotate y bits 128-bit result

MD5: Padding 1 2 3 4 input Message 512 bit block Padding Initial Value Transformation block by block Final Output Output 128 bits Digest

Padding Twist Given original message M, add padding bits “1000…” such that resulting length is 64 bits less than a multiple of 512 bits. Append (original length in bits mod 264), represented in 64 bits to the padded message Final message is chopped 512 bits a block

MD5 Process As many stages as the number of 512-bit blocks in the final padded message Digest: 4 32-bit words: MD=A|B|C|D Every message block contains 16 32-bit words: m0|m1|m2…|m15 Digest MD0 initialized to: A=01234567,B=89abcdef,C=fedcba98, D=76543210 Every stage consists of 4 passes over the message block, each modifying MD

MD5 Blocks 512: B1 512: B2 MD5 512: B3 MD5 512: B4 MD5 MD5 Result

Processing of Block mi - 4 Passes MDi ABCD=fF(ABCD,mi,T[1..16]) A B C D ABCD=fG(ABCD,mi,T[17..32]) ABCD=fH(ABCD,mi,T[33..48]) ABCD=fI(ABCD,mi,T[49..64]) + + + + MD i+1

Different Passes... Different functions and constants are used Different set of mi is used Different set of shift amount is used

Functions and Random Numbers F(x,y,z) == (xy)(~x  z) selection function G(x,y,z) == (x  z) (y ~ z) H(x,y,z) == xy z I(x,y,z) == y(x  ~z) Ti = int(232 * abs(sin(i))), 0<i<65

Secure Hash Algorithm Developed by NIST, specified in the Secure Hash Standard (SHS, FIPS Pub 180), 1993 SHA is specified as the hash algorithm in the Digital Signature Standard (DSS), NIST

General Logic Input message must be < 264 bits not really a problem Message is processed in 512-bit blocks sequentially Message digest is 160 bits SHA design is similar to MD5, but a lot stronger

Basic Steps Step1: Padding Step2: Appending length as 64 bit unsigned Step3: Initialize MD buffer 5 32-bit words A|B|C|D|E A = 67452301 B = efcdab89 C = 98badcfe D = 10325476 E = c3d2e1f0

Basic Steps... Step 4: the 80-step processing of 512-bit blocks – 4 rounds, 20 steps each. Each step t (0 <= t <= 79): Input: Wt – a 32-bit word from the message Kt – a constant. ABCDE: current MD. Output: ABCDE: new MD.

Basic Steps... Only 4 per-round distinctive additive constants 0 <=t<= 19 Kt = 5A827999 20<=t<=39 Kt = 6ED9EBA1 40<=t<=59 Kt = 8F1BBCDC 60<=t<=79 Kt = CA62C1D6

Basic Steps A E B C D + ft + CLS5 Wt + CLS30 Kt + A E B C D

Basic Logic Functions Only 3 different functions Round Function ft(B,C,D) 0 <=t<= 19 (BC)(~B D) 20<=t<=39 BCD 40<=t<=59 (BC)(BD)(CD) 60<=t<=79 BCD

Twist With Wt’s Additional mixing used with input message 512-bit block W0|W1|…|W15 = m0|m1|m2…|m15 For 15 < t <80: Wt = Wt-16 Wt-14 Wt-8 Wt-3 XOR is a very efficient operation, but with multilevel shifting, it should produce very extensive and random mixing!

SHA Versus MD5 SHA is a stronger algorithm: Brute-force birthday attacks requires on the order of 280 operations vs. 264 for MD5 SHA’s 80 steps and 160-bit hash (vs. 128) requires a little more computation