1. Chapter 2.5: Dictionaries and Hash Tables 1 2 3  4

2. Dictionary ADT (§2.5.1)
The dictionary ADT models a searchable collection of key-element items
The main operations of a dictionary are searching, inserting, and deleting items
Multiple items with the same key are allowed
Applications:
address book
credit card authorization
mapping host names (e.g., cs16.net) to internet addresses (e.g., )
Dictionary ADT methods:
findElement(k): if the dictionary has an item with key k, returns its element, else, returns the special element NO_SUCH_KEY
insertItem(k, o): inserts item (k, o) into the dictionary
removeElement(k): if the dictionary has an item with key k, removes it from the dictionary and returns its element, else returns the special element NO_SUCH_KEY
size(), isEmpty()
keys(), Elements()

3. Log File (§2.5.1)
A log file is a dictionary implemented by means of an unsorted sequence
We store the items of the dictionary in a sequence (based on a doubly-linked lists or a circular array), in arbitrary order
Performance:
insertItem takes O(1) time since we can insert the new item at the beginning or at the end of the sequence
findElement and removeElement take O(n) time since in the worst case (the item is not found) we traverse the entire sequence to look for an item with the given key
The log file is effective only for dictionaries of small size or for dictionaries on which insertions are the most common operations, while searches and removals are rarely performed (e.g., historical record of logins to a workstation)

4. Lookup Table
A lookup table is a dictionary implemented with a sorted sequence
We store the items of the dictionary in an array-based sequence, sorted by key
We use an external comparator for the keys
Performance:
findElement takes O(log n) time, using binary search
insertItem takes O(n) time since in the worst case we have to shift O(n) items to make room for the new item
removeElement take O(n) time since in the worst case we have to shift O(n) items to compact the items after the removal
Effective for small dictionaries or for dictionaries where searches are common but inserts and deletes are rare (e.g. credit card authorizations)

5. Hashing (2.5.2) Application: word occurrence statistics
Operations: insert, find
Dictionary: insert, delete, find
Are O(log n) comparisons necessary? (no)
Hashing basic plan:
create a big array for the items to be stored
use a function to figure out storage location from key (hash function)
a collision resolution scheme is necessary

6. Hash Table Example Simple Hash function: Example:
Treat the key as a large integer K
h(K) = K mod M, where M is the table size
let M be a prime number.
Example:
Suppose we have 101 buckets in the hash table.
‘abcd’ in hex is 0x
Converted to decimal it’s
% 101 = 11
Thus h(‘abcd’) = 11. Store the key at location 11.
“dcba” hashes to 57.
“abbc” also hashes to 57 – collision. What to do?
If you have billions of possible keys and hundreds of buckets, lots of collisions are possible!

7. Hash Functions (§ 2.5.3)
A hash function is usually specified as the composition of two functions:
Hash code map: h1: keys  integers
Compression map: h2: integers  [0, N - 1]
The hash code map is applied first, and the compression map is applied next on the result, i.e., h(x) = h2(h1(x))
The goal of the hash function is to “disperse” the keys in an apparently random way

8. Hash Code Maps (§2.5.3)
Polynomial accumulation: like component sum, but multiply each term by 1, z, z2, z3, ...
p(z) = a0 + a1 z + a2 z2 + … … + an-1zn-1
at a fixed value z, ignoring overflows
Can be evaluated in O(n) time using Horner’s rule:
Each term is computed from the previous in O(1) time
p0(z) = an-1
pi (z) = an-i-1 + zpi-1(z) (i = 1, 2, …, n -1)
Memory address: interpret the memory address of the key as an integer
Integer cast: interpret the bits of the key as an integer (for short keys)
Component sum: partition the bits of the key into chunks (e.g., 16 or 32 bits) and sum, ignoring overflows (for long keys)

9. Compression Maps (§2.5.4) Multiply, Add and Divide (MAD): Division:
h2 (y) = (ay + b) mod N
a and b are nonnegative integers such that a mod N  0
Otherwise, every integer would map to the same value b
Division:
h2 (y) = y mod N
The size N of the hash table is usually chosen to be a prime
The reason has to do with number theory and is beyond the scope of this course

10. Hashing Strings h(‘aVeryLongVariableName’)? Horner’s method example:
256 * = % 101 = 72
256 * = % 101 = 50
256 * = % 101 = 87
Scramble by replacing 256 with 117
int hash(char *v, int M)
{ int h, a=117;
for (h=0; *v; v++)
h = (a*h + *v) % M;
return h; }

11. Collisions (§2.5.5) How likely are collisions? Birthday paradox
M sqrt(p M/2) (about 1.25 sqrt(M))
[1.25 sqrt(365) is about 24]
Experiment: generate random numbers …
Collision at 13th number, as predicted
What to do about collisions?

12. Collision Resolution: Chaining
Build a linked list for each bucket
Linear search within each list
Simple, practical, widely used
Cuts search time by a factor of M over sequential search
But, requires extra memory outside of table  1 2 3 4

13. Chaining 2 Insertion time? Average search cost, successful search?
O(N/2M)
Average search cost, unsuccessful?
O(N/M)
M large: CONSTANT average search time
Worst case: N (“probabilistically unlikely”)
Keep lists sorted?
insert time O(N/2M)
unsuccessful search time O(N/2M)

14. Linear Probing (§2.5.5) Or, we could keep everything in the same table
Insert: upon collision, search for a free spot
Search: same (if you find one, fail)
Runtime?
Still O(1) if table is sparse
But: as table fills, clustering occurs
Skipping c spots doesn’t help…

15. Clustering Long clusters tend to get longer Precise analysis difficult
Theorem (Knuth):
Insert cost: approx. (1 + 1/(1-N/M)2)/2
(50% full  2.5 probes; 80% full  13 probes)
Search (hit) cost: approx. (1 + 1/(1-N/M))/2
(50% full  1.5 probes; 80% full  3 probes)
Search (miss): same as insert
Too slow when table gets 70-80% full
How to reduce/avoid clustering?

16. Double Hashing Use a second hash function to compute increment seq.
Analysis extremely difficult
About like ideal (random probe)
Thm (Guibas-Szemeredi):
Insert: approx 1+1/(1-N/M)
Search hit: ln(1+N/M)/(N/M)
Search miss: same as insert
Not too slow until the table is about 90% full

17. Example of Double Hashing1 2 3 4 5 6 7 8 9 10 11 12 31 41 18 32 59 73 22 44
Consider a hash table storing integer keys that handles collision with double hashing
N = 13
h(k) = k mod 13
d(k) = 7 - k mod 7
Insert keys 18, 41, 22, 44, 59, 32, 31, 73, in this order

18. Dynamic Hash Tables
Suppose you are making a symbol table for a compiler. How big should you make the hash table?
If you don’t know in advance how big a table to make, what to do?
Could grow the table when it “fills” (e.g. 50% full)
Make a new table of twice the size.
Make a new hash function
Re-hash all of the items in the new table
Dispose of the old table

19. Table Growing Analysis
Worst case insertion: Q(n), to re-hash all items
Can we make any better statements?
Average case?
O(1), since insertions n through 2n cost O(n) (on average) for insertions and O(2n) (on average) for rehashing  O(n) total (with 3x the constant)
Amortized analysis?
The result above is actually an amortized result for the rehashing.
Any sequence of j insertions into an empty table has O(j) average cost for insertions and O(2j) for rehashing.
Or, think of it as billing 3 time units for each insertion, storing 2 in the bank. Withdraw them later for rehashing.

20. Separate Chaining vs. Double Hashing
Assume the same amount of space for keys, links (use pointers for long or variable-length keys)
Separate chaining:
1M buckets, 4M keys
4M links in nodes
9M words total; avg search time 2
Double hashing in same space:
4M items, 9M buckets in table
average search time: 1/(1-4/9) = 1.8: 10% faster
Double hashing in same time
4M items, average search time 2
space needed: 8M words (1/(1-4/8) = 2) (11% less space)

21. Deletion How to implement delete() with separate chaining?
Simply unlink unwanted item
Runtime?
Same as search()
How to implement delete() with linear probing?
Can’t just erase it. (Why not?)
Re-hash entire cluster
Or mark as deleted?
How to delete() with double hashing?
Re-hashing cluster doesn’t work – which “cluster”?
Mark as deleted
Every so often re-hash entire table to prune “dead-wood”

22. Comparisons Separate chaining advantages: Why use hashing?
Idiot-proof (degrades gracefully)
No large chunks of memory needed
Why use hashing?
Fastest dictionary implementation
Constant time search and insert, on average
Easy to implement; built into many environments
Why not use hashing?
No performance guarantees
Uses extra space
Doesn’t support pred, succ, sort, etc. – no notion of order
Where did perl “hashes” get their name?

23. Hashing Summary Separate chaining: easiest to deploy
Linear probing: fastest (but takes more memory)
Double hashing: least memory (but takes more time to compute the second hash function)
Dynamic (grow): handles any number of inserts at < 3x time
Curious use of hashing: early unix spell checker (back in the days of 3M machines…)
Construction Search Miss
Chain Probe Dbl Grow Chain Probe Dbl Grow 5k
50k
100k
190k
200k

24. File tamper test
Problem: you want to guarantee that a file hasn’t been tampered with. How?

25. Password verification
Problem: you run a website. You need to verify people’s logins. How?
Possible techniques?
Ethics?

27. Cache filenames Problem: caching FlexScores $outfileName =
$outDirectory . "/" . $cfg->hymn . "-" . $cfg->instrument;
if ($custom)
$outfileName .= "-" . substr(md5($cfg->asXML()),0,6);
$outfileNameLy = $outfileName . ".ly";

28. Hash Tables in C# System.Collections: basic data structures Dictionary
ArrayList
(See handout)

29. GUI Programming in C#
Microsoft has provided a number of different GUI programming environments over the years:
MFC: Microsoft Foundation Classes
C++, legacy
Windows Forms
.net wrapper for access to native Windows interface
WPF: Windows Presentation Foundation
.net, Managed code, primarily C# and VB
XML file defines user interface
Better support for media, animation, etc.
MFC: old C++-based library fro GUI programming.
Windows Forms:.Net GUI api. Replaces MFC. (Not the most recent, though.) Like Java’s AWT.
Obsolete, but still used.
WPF previously known as Avalon. Released as part of .Net 3.0. Uses DirectX.
Uses XML-oriented object model, XAML, to define a user interface and link various UI elements.
Runtime libraries are included with all recent versions of windows.
Sliverlight is mostly a subset of WPF for embedded Web controls.

31. Turing Test
You want to generate random text that sounds like it makes sense. How?

32. Google ngrams Bag-of-words context word2vec
Word2vec: how frequently the word appears in different contexts. Row, with hundreds of columns for different contexts
Maybe context is a bag of words before and after
Big matrix: rows for words, columns for contexts
Would have enormous millions x millions matrix, but decompose into much smaller dimensions
Result is vector for each word with hundreds of entries, encoding something about meaning
If two words have similar vector, then similar meaning
Can add+subtract vectors to add and subtract meanings
[king] – [man] + [woman] is very similar to [queen]
Paris is to France as ?? Is to Germany: how to figure out?
[Paris] – [France] + [England]
Should give a vector very similar to the one for “London”
Run billions of words through; each time a pair of words co-occurs in the same context, move those vectors a hair closer
[effectively a stochastic singular value decomposition of the PMI matrix, but faster]
Could ask questions like:
Aaron is the brother of Moses. Who are the brothers of Jesus?

33. Babble
You want to generate natural-sounding random text. How?