Applications Data Structures and Algorithms (60-254)

Encoding and Decoding Messages 2

Fixed-length coding Code for each character has the same number of bits. Question How many bits do we need to encode uniquely each character in a message made up of characters from an n-letter alphabet? Answer  log n  bits at least. 3

Variable-length coding 4

Another encoding scheme 5

Prefix codes Code word of a character  is NOT prefix of the code for a character , where   . Also, called prefix-free because the term “prefix” is misleading… Meaning of prefix Examples: 00 is prefix of is prefix of is not prefix of

What is interesting in a prefix code? 7

Lossless Data Compression M = a d d d Encoding using scheme 1 C 1 = Encoding using scheme 2 C 2 = Length(C 1 )=6 Length(C 2 )=8 Lossless data compression is possible using prefix codes 8

Huffman coding 9

10

Huffman coding 11

Huffman coding 12

Huffman coding 13

Huffman coding 14

Huffman coding 15

Huffman coding 16

Huffman coding 17

Huffman coding 18

Graphs 19

Graphs We denote a graph by G=(V, E) where V = Set of vertices and E = Set of edges |V| = Number of vertices and |E| = Number of edges G is directed if the edges are, otherwise G is undirected. 20

Graph representation 21

Graph representation 22

Graph Representation 23

Breadth-first search Also called: level order search What does it do? Given: G=(V, E), a directed or undirected graph and a “source” vertex s  V It systematically explores vertices reachable from s, to construct a breadth-first search tree 24

Shortest Path Length of the path from s to each reachable vertex is the shortest path from s to this vertex Example graph 25

Expand node s 26

Expand node w 27

Expand node r 28

Expand node t 29

Expand node x 30

Expand node v 31

Expand node u 32

Breath-first Search Tree 33

Formal Algorithm Colour terminology: Gray nodes make up the “frontier” (red line) White nodes (vertices) are the unexplored ones Black nodes are the completely explored ones, lying on the other side of the “frontier” 34

Breadth-first search algorithm BFS(G, s) For each vertex u  V –{s} {// Setup loop O(|V|) u.color  WHITE; u.distance   ; u.parent  NULL; // parent of u } s.color  GRAY; s.distance  0; s.parent  NULL; Q  Q  {s}; (Cont’d.) 35

Breadth-first search algorithm While (Q   ) {// Processing loop O(|E|) u  head (Q); for each v adjacent to u { if (v.color = WHITE) { v.color  GRAY; v.distance  u.distance+1; v.parent  u; Q  Q  {v}; } Delete u from Q; u.color  BLACK; } 36

Analysis Thm.Running time of BFS is O (|V|+|E|) PfQueuing operations take O (|V|) time. Each edge in the graph is looked at most once. Hence O (|E|) time and the theorem follows. 37

We can prove the following facts Fact1:Breath-first-search (BFS) discovers every vertex reachable from s Fact2:The length of the path from s to a reachable vert ex v, is a shortest path from s to v. 38

Weighted Graph Each edge has a weight (a number). 39

Minimum Spanning Tree (MST) Green edges form a minimum spanning tree of the example graph 40

Minimum Spanning Tree Weight of the MST is: W(T) = = 37 41

Formally Given a connected, weighed, undirected graph G=(V, E), Find an acyclic subset T  E that connects all the vertices such that is minimum, where weight(u,v) is the weight of the edges connecting u to v. 42

Growing an MST, greedily Let A be a subset of edges of some MST Def. An edge e=(u,v) of G is safe for A if A  {(u,v)} is also a subset of an MST. 43

A generic MST algorithm GENERIC_MST(G,w) A   ; while (A does not form an MST) find a safe edge (u,v) for A; A  A  {(u,v)}; return A; 44

Cuts and safe edges Definition: A cut of G is a partition (S, V-S) of V 45

A few definitions Definition: An edge (u,v)  E crosses the cut (S,V-S) if one of its end points is in S and the other in V-S Definition: A cut respects the set A if no edge in A crosses the cut Definition: An edge is a light edge crossing a cut if its weight is the minimum of any edge crossing the cut. 46

Safe Edges The following theorem characterizes a safe edge for A Theorem Let A be a subset of E that is included in some MST for G. Let (S,V-S) be any cut of G that respects A, and Let (u,v) be a light edge crossing (S,V-S). Then, edge (u,v) is safe for A. 47

Prim’s algorithm Select vertex a S={a} 48

(a, b) is a light edge for (S, V-S) Select vertex b new S={a, b} 49

(b,c) is a light edge for (S,V-S) Select vertex c new S={a, b, c} 50

(c, i) is a light edge for (S,V-S) where S={a, b, c} Select vertex i new S={a, b, c, i} 51

(c, f) is a light edge for (S,V-S) Select vertex f new S={a, b, c, f, i} 52

(f, g) is a light edge for (S,V-S) Select vertex g new S={a, b, c, g, f, i} 53

(g, h) is a light edge for (S,V-S) Select vertex h new V-S={d, e} 54

(c, d) is a light edge for (S,V-S) Select vertex d new V-S={e} 55

(d, e) is a light edge for (S,V-S) Select vertex e new V-S=  56

Formal Algorithm MST_Prim(G, W, r) Q  V; for (each u  Q) u.key   ; r.key  0; r.parent  NULL; while ( Q   ) u  extract_min(Q); for (each v adjacent to u) if v  Q and weight(u,v) < v.key v.parent  u; v.key  weight(u,v) 57

Important Note on Shortest Path: Dijkstra’s algorithm The algorithm incrementally constructs a set S of vertices to which the shortest path is known. It adds a vertex v (from the remaining vertices) to S, whose distance from S (source) is the shortest of the remaining vertices. The assumption of non-negative weights guarantees that the shortest path to v passes only through vertices in S. 58

Priority Queues Let S be a set of keys, a priority queue supports the following operations on S: 1.Insert (S, x) 2.Minimum (S)-- returns min key 3. Extract_Min (S)--returns and removes min key 59

Heap as Priority Queue A priority queue can be implemented using a heap A heap is a complete binary tree, satisfying the heap property: For a node v, v.key  v.left.key v.key  v.right.key 60

Single-source shortest paths Let G be a weighted, directed graph. A path in the graph from u to v is a sequence of edges that connects u to v. The weight of this path is the sum of the weights of its constituent edges. Let  (u, v) denote a minimum weight path from u to v Let s be a source vertex of G 61

Single-source shortest path problem Compute a shortest path from s to each vertex v. Remark:A shortest path is a path of minimum weight. Dijkstra’s algorithm: We assume that the weights of the edges are non-negative. 62

Relaxation 63

Relax Relax(u,v,w) { If (v.distance > u.distance +w(u,v)) { v.distance  u.distance +w(u,v); v.parent  u; } Initialize_Simple_Source (G, v) { For each vertex v  V [G]; v.distance   ; v.parent  NULL; } 64

Dijkstra’s algorithm by example The algorithm maintains a set X of vertices whose final shortest-path weights from s have already been determined 65

Add s to X Relax vertices adjacent to s 66

Add x to X Relax vertices adjacent to x 67

Add y to X Relax vertices adjacent to y 68

Add u to X Relax vertices adjacent to u 69

Add v to X Done 70

Formal Algorithm Dijkstra(G,w,s) Initialize_single_source(G,s) X  Q  V [G] while (Q   ) u  extract_min(Q) X  X  {u} For each vertex v adjacent to u Relax(u,v,w) 71