1. Dictionaries Collection of pairs. Operations. (key, element)
Pairs have different keys.
Operations.
get(theKey)
put(theKey, theElement)
remove(theKey)
A linear list is an ordered sequence of elements. A dictionary is just a collection of pairs.
Interface Dictionary has these 3 methods. Note that Java has an abstract class Dictionary that has these three methods plus isEmpty and some others.

2. Application Collection of student records in this class.
(key, element) = (student name, linear list of assignment and exam scores)
All keys are distinct.
Get the element whose key is John Adams.
Update the element whose key is Diana Ross.
put() implemented as update when there is already a pair with the given key.
remove() followed by put().

3. Dictionaries AVL Trees vs. Hashtables
Method
AVL
Hashtable
Worst
Average
Not Bad
Worst
Average
Astounding!
put
O(Log N)
O(Log N)
O(N)
O(1)
get
O(Log N)
O(Log N)
O(N)
O(1)
remove
O(Log N)
O(Log N)
O(N)
O(1)

4. Hash Tables Worst-case time for get, put, and remove is O(size).
Expected time is O(1).

5. Simple Example
Insert data into the hashtable using characters as keys
The hashtable is an array of “items”
The hashtables’ capacity is 7
The hash function must take a character as input and convert it into a number between 0 and 6.
Use the following hash function: Let P be the position of the character in the English alphabet (starting with 1). The hash function h(K) = P
The function must be normalized in order to map into the appropriate range (0-6). The normalized hash function is h(K) % 7. 1 2 3 4 5 6

6. Example 1 2 3 4 5 6 put(B2, Data1) put(S19, Data2) put(J10, Data3)
put(N14, Data4)
put(X24, Data5)
put(W23, Data6)
put(B2, Data7)
get(X24)
get(W23) 1 2 3 4 5 6
(N14, Data4)
This is called a collision
Collisions are handled via a “collision resolution policy”
(B2, Data1)
(J10, Data3)
(X24, Data5) ???
(S19, Data2)

7. Ideal Hashing Uses a 1D array (or table) table[0:b-1].
Each position of this array is a bucket.
A bucket can normally hold only one dictionary pair.
Uses a hash function f that converts each key k into an index in the range [0, b-1].
f(k) is the home bucket for key k.
Every dictionary pair (key, element) is stored in its home bucket table[f[key]].
Bucket size equal to size of L1 cache line or of a disk track or sector may yield better performance in the case of non-ideal hashing.

8. Ideal Hashing Example
Pairs are: (22,a), (33,c), (3,d), (73,e), (85,f).
Hash table is table[0:7], b = 8.
Hash function is key/11.
Pairs are stored in table as below:
(3,d)
(22,a)
(33,c)
(73,e)
(85,f)
[0]
[1]
[2]
[3]
[4]
[5]
[6]
[7]
get, put, and remove take O(1) time.

9. What Can Go Wrong? (3,d) (22,a) (33,c) (73,e) (85,f)
[0]
[1]
[2]
[3]
[4]
[5]
[6]
[7]
Where does (26,g) go?
Keys that have the same home bucket are synonyms.
22 and 26 are synonyms with respect to the hash function that is in use.
The home bucket for (26,g) is already occupied.
Where does (100,h) go? There is no bucket with index 100/11 = 9. There is a problem with the hash function, because it is supposed to map all valid keys into a valid bucket.

10. What Can Go Wrong?
(85,f)
(22,a)
(33,c)
(3,d)
(73,e)
A collision occurs when the home bucket for a new pair is occupied by a pair with a different key.
An overflow occurs when there is no space in the home bucket for the new pair.
When a bucket can hold only one pair, collisions and overflows occur together.
Need a method to handle overflows.

11. Hash Table Issues Choice of hash function. Overflow handling method.
Size (number of buckets) of hash table.

12. Hash Functions Two parts: Map an integer into a home bucket.
Convert key into an integer in case the key is not an integer.
Done by the method hashCode().
Map an integer into a home bucket.
f(k) is an integer in the range [0, b-1], where b is the number of buckets in the table.
Object.hashCode() returns an int based on the reference value of this. So user defined classes must over ride Object.hashCode() for things to work right.
hashCode() is already defined properly for Java’s classes: Integer, Long, Float, Double, String, Character, etc. The hashCode() definition in these classes over rides the default definition in Object. Note that hashCode may return a negative int.

13. String To Integer Each Java character is 2 bytes long.
An int is 4 bytes.
A 2 character string s may be converted into a unique 4 byte int using the code:
int answer = s.charAt(0);
answer = (answer << 16) + s.charAt(1);
Strings that are longer than 2 characters do not have a unique int representation.

14. String To Nonnegative Integer
public static int integer(String s) {
int length = s.length();
// number of characters in s
int answer = 0;
if (length % 2 == 1)
{// length is odd
answer = s.charAt(length - 1);
length--; }

15. String To Nonnegative Integer
// length is now even
for (int i = 0; i < length; i += 2)
{// do two characters at a time
answer += s.charAt(i);
answer += ((int) s.charAt(i + 1)) << 16; }
return (answer < 0) ? -answer : answer;

16. Map Into A Home Bucket Most common method is by division. homeBucket =
(85,f)
[0]
[1]
[2]
[3]
[4]
[5]
[6]
[7]
Most common method is by division.
homeBucket =
Math.abs(theKey.hashCode()) % divisor;
divisor equals number of buckets b.
0 <= homeBucket < divisor = b
Note: hashCode maps into an integer, not into a nonnegative integer. % works properly only with nonnegative integers.

17. Uniform Hash Function Let keySpace be the set of all possible keys.
(3,d)
(22,a)
(33,c)
(73,e)
(85,f)
[0]
[1]
[2]
[3]
[4]
[5]
[6]
[7]
Let keySpace be the set of all possible keys.
A uniform hash function maps the keys in keySpace into buckets such that approximately the same number of keys get mapped into each bucket.

18. Uniform Hash Function
(3,d)
(22,a)
(33,c)
(73,e)
(85,f)
[0]
[1]
[2]
[3]
[4]
[5]
[6]
[7]
Equivalently, the probability that a randomly selected key has bucket i as its home bucket is 1/b, 0 <= i < b.
A uniform hash function minimizes the likelihood of an overflow when keys are selected at random.

19. Hashing By Division keySpace = all ints.
For every b, the number of ints that get mapped (hashed) into bucket i is approximately 232/b.
Therefore, the division method results in a uniform hash function when keySpace = all ints.
In practice, keys tend to be correlated.
So, the choice of the divisor b affects the distribution of home buckets.

20. Selecting The Divisor
Because of this correlation, applications tend to have a bias towards keys that map into odd integers (or into even ones).
When the divisor is an even number, odd integers hash into odd home buckets and even integers into even home buckets.
20%14 = 6, 30%14 = 2, 8%14 = 8
15%14 = 1, 3%14 = 3, 23%14 = 9
The bias in the keys results in a bias toward either the odd or even home buckets.

21. Selecting The Divisor
When the divisor is an odd number, odd (even) integers may hash into any home.
20%15 = 5, 30%15 = 0, 8%15 = 8
15%15 = 0, 3%15 = 3, 23%15 = 8
The bias in the keys does not result in a bias toward either the odd or even home buckets.
Better chance of uniformly distributed home buckets.
So do not use an even divisor.

22. Selecting The Divisor
Similar biased distribution of home buckets is seen, in practice, when the divisor is a multiple of prime numbers such as 3, 5, 7, …
The effect of each prime divisor p of b decreases as p gets larger.
Ideally, choose b so that it is a prime number.
Alternatively, choose b so that it has no prime factor smaller than 20.

23. Java.util.HashTable Simply uses a divisor that is an odd number.
This simplifies implementation because we must be able to resize the hash table as more pairs are put into the dictionary.
Array doubling, for example, requires you to go from a 1D array table whose length is b (which is odd) to an array whose length is 2b+1 (which is also odd).

24. Overflow Handling
An overflow occurs when the home bucket for a new pair (key, element) is full.
We may handle overflows by:
Search the hash table in some systematic fashion for a bucket that is not full.
Linear probing (linear open addressing).
Quadratic probing.
Random probing.
Eliminate overflows by permitting each bucket to keep a list of all pairs for which it is the home bucket.
Array linear list.
Chain.

25. Linear Probing – Get And Put
divisor = b (number of buckets) = 17.
Home bucket = key % 17. 4 8 12 16 34 45 6 23 7 28 12 29 11 30 33
Put in pairs whose keys are 6, 12, 34, 29, 28, 11, 23, 7, 0, 33, 30, 45

26. Linear Probing – Remove4 8 12 16 6 29 34 28 11 23 7 33 30 45
remove(0) 4 8 12 16 6 29 34 28 11 23 7 45 33 30
Search cluster for pair (if any) to fill vacated bucket. 4 8 12 16 6 29 34 28 11 23 7 45 33 30

27. Linear Probing – remove(34)4 8 12 16 6 29 34 28 11 23 7 33 30 45 4 8 12 16 6 29 28 11 23 7 33 30 45
Search cluster for pair (if any) to fill vacated bucket. 4 8 12 16 6 29 28 11 23 7 33 30 45 4 8 12 16 6 29 28 11 23 7 33 30 45

28. Linear Probing – remove(29)4 8 12 16 6 29 34 28 11 23 7 33 30 45 4 8 12 16 6 34 28 11 23 7 33 30 45
Search cluster for pair (if any) to fill vacated bucket. 4 8 12 16 6 11 34 28 23 7 33 30 45 4 8 12 16 6 11 34 28 23 7 33 30 45 4 8 12 16 6 11 34 28 23 7 33 30 45

29. Performance Of Linear Probing4 8 12 16 6 29 34 28 11 23 7 33 30 45
Worst-case get/put/remove time is Theta(n), where n is the number of pairs in the table.
This happens when all pairs are in the same cluster.

30. Expected Performance alpha = loading density = (number of pairs)/b.4 8 12 16 6 29 34 28 11 23 7 33 30 45
alpha = loading density = (number of pairs)/b.
alpha = 12/17.
Sn = expected number of buckets examined in a successful search when n is large
Un = expected number of buckets examined in a unsuccessful search when n is large
Time to put and remove governed by Un.
A put that increases the number of pairs in the table involves an unsuccessful search followed by the addition of an element. An unsuccessful remove is essentially an unsuccessful search. A successful remove must also go to the end of the cluster and so is like an unsuccessful search.

31. Expected Performance Alpha <= 0.75 is recommended.
Sn ~ ½(1 + 1/(1 – alpha))
Un ~ ½(1 + 1/(1 – alpha)2)
Note that 0 <= alpha <= 1.
Alpha <= 0.75 is recommended.

32. Hash Table Design
Performance requirements are given, determine maximum permissible loading density.
We want a successful search to make no more than 10 compares (expected).
Sn ~ ½(1 + 1/(1 – alpha))
alpha <= 18/19
We want an unsuccessful search to make no more than 13 compares (expected).
Un ~ ½(1 + 1/(1 – alpha)2)
alpha <= 4/5
So alpha <= min{18/19, 4/5} = 4/5.

33. Hash Table Design Dynamic resizing of table. Fixed table size.
Whenever loading density exceeds threshold (4/5 in our example), rehash into a table of approximately twice the current size.
Fixed table size.
Know maximum number of pairs.
No more than 1000 pairs.
Loading density <= 4/5 => b >= 5/4*1000 = 1250.
Pick b (equal to divisor) to be a prime number or an odd number with no prime divisors smaller than 20.

34. Linear List Of Synonyms
Each bucket keeps a linear list of all pairs for which it is the home bucket.
The linear list may or may not be sorted by key.
The linear list may be an array linear list or a chain.

35. [0]
[4]
[8]
[12]
[16] 12 6 34 29 28 11 23 7 33 30 45
Sorted Chains
Put in pairs whose keys are 6, 12, 34, 29, 28, 11, 23, 7, 0, 33, 30, 45
Home bucket = key % 17.
The chains may be sorted or unsorted (the text uses sorted chains).

36. Hashing - resolving collisions
Chaining a.k.a closed addressing
Idea : put all elements that hash to the same slot in a linked list (chain). The slot contains a pointer to the head of the list.
The load factor indicates the average number of elements stored in a chain. It could be less than, equal to, or larger than 1.

37. Hashing - resolving collisions
Chaining
Insert : O(1)
worst case
Delete : O(1)
assuming doubly-linked list
it’s O(1) after the element has been found
Search : ?
depends on length of chain.

38. Hashing - resolving collisions
Chaining
Assumption: simple uniform hashing
any given key is equally likely to hash into any of the m slots
Unsuccessful search:
average time to search unsuccessfully for key k = the average time to search to the end of a chain.
The average length of a chain is .
Total (average) time required : (1+ )

39. Hashing - resolving collisions
Chaining
Successful search:
expected number e of elements examined during a successful search for key k
=1 more than the expected number of elements examined when k was inserted.
it makes no difference whether we insert at the beginning or the end of the list.
Take the average, over the n items in the table, of 1 plus the expected length of the chain to which the ith element was added:

40. Hashing - resolving collisions
Chaining
Total time : (1+ )

41. Hashing - resolving collisions
Chaining
Both types of search take (1+ ) time on average.
If n=O(m), then =O(1) and the total time for Search is O(1) on average
Insert : O(1) on the worst case
Delete : O(1) on the worst case
Another idea: Link all unused slots into a free list

42. Expected Performance Note that alpha >= 0.
Expected chain length is alpha.
Sn ~ 1 + alpha/2.
Un <= alpha, when alpha < 1.
Un ~ 1 + alpha/2, when alpha >= 1.

43. Hashing - resolving collisions
Open addressing : linear probing
hash function: (h(k)+i)mod m for i=0, 1,...,m-1
Insert : start with the location where the key hashed and do a sequential search for an empty slot.
Search : start with the location where the key hashed and do a sequential search until you either find the key (success) or find an empty slot (failure).
Delete : (lazy deletion) follow same route but mark slot as DELETED rather than EMPTY, otherwise subsequent searches will fail (why?).

44. Hashing - resolving collisions
Open addressing : linear probing
Disadvantage: primary clustering
long sequences of used slots build up with gaps between them.
result : performance near sequential search

45. Hashing - resolving collisions
Open addressing : quadratic probing
Probe the table at slots (h(k)+i2) mod m for i=0, 1,2, 3, ..., m-1
Careful : for some values of m (hash table size), very few slots end up being probed.
Try m=16
It is possible to probe all slots for certain ms.
For prime m, we get pretty good results.
if the table is at least half-empty, an element can always be inserted

46. Hashing - resolving collisions
Open addressing : quadratic probing
Better than linear probing but may result in secondary clustering: if h(k1)==h(k2) the probing sequences for k1 and k2 are exactly the same
general hash function: (h(k)+c1i+c2i2) mod m

47. Hashing - resolving collisions
Open addressing : double hashing
The hash function is (h(k)+i h2(k)) mod m
In English: use a second hash function to obtain the next slot.
The probing sequence is:
h(k), h(k)+h2(k), h(k)+2h2(k), h(k)+3h3(k), ...
Performance :
Much better than linear or quadratic probing.
Does not suffer from clustering
BUT requires computation of a second function

48. Hashing - resolving collisions
Open addressing : double hashing
The choice of h2(k) is important
It must never evaluate to zero
consider h2(k)=k mod 9 for k=81
The choice of m is important
If it is not prime, we may run out of alternate locations very fast.
If m and h2(k) are relatively prime, we’ll end up probing the entire table.
A good choice for h2 is h2(k)=p- (k mod p) where p is a prime less than m.

49. Double Hashing Example
h1(K) = K mod 13
h2 (K) = 8 - K mod 8
we want h2 to be an offset to add

50. Double Hashing Example (cont.)

51. Hashing - resolving collisions
Open addressing : random probing
Use a pseudorandom number generator to obtain the sequence of slots that are probed.

52. Hashing - resolving collisions
Open addressing : expected # of probes
Assuming uniform hashing...
Insert/Unsuccessful search : 1/(1-)
Successful search : (1+ln(1/(1- ))/ 

53. Hashing - resolving collisions
Example
m = 13
sequence of keys:
h1(k) = k mod 13
Insert the sequence into a hash table using
linear probing
quadratic probing
double hashing with h2(k) = k mod 7 + 6

54. Hashing - rehashing
If the table becomes too full, its performance falls.
The O(1) property is lost
Solution:
build a bigger table (e.g. approximately twice as big) and rehash the keys of the old table.
When should we rehash?
when the table is half full?
when an insertion fails?
when a certain load factor has been reached?

55. Theoretical Results Let  = N/M
the load factor: average number of keys per array index
Analysis is probabilistic, rather than worst-case
Expected Number of Probes
Not found
found

56. Expected Number of Probes vs. Load Factor

57. Summary Dictionaries may be ordered or unordered
Unordered can be implemented with
lists (array-based or linked)
hashtables (best solution)
Ordered can be implemented with
trees (avl (best solution), splay, bst)