1. Applications Data Structures and Algorithms (60-254)

2. Encoding and Decoding Messages 2

3. Fixed-length coding Code for each character has the same number of bits. Question How many bits do we need to encode uniquely each character in a message made up of characters from an n-letter alphabet? Answer  log n  bits at least. 3

4. Variable-length coding 4

5. Another encoding scheme 5

6. Prefix codes Code word of a character  is NOT prefix of the code for a character , where   . Also, called prefix-free because the term “prefix” is misleading… Meaning of prefix Examples: 00 is prefix of 001 110 is prefix of 110111 111101 is not prefix of 1111001 6

7. What is interesting in a prefix code? 7

8. Lossless Data Compression M = a d d d Encoding using scheme 1 C 1 = 000111 Encoding using scheme 2 C 2 =00111111 Length(C 1 )=6 Length(C 2 )=8 Lossless data compression is possible using prefix codes 8

9. Huffman coding 9

10. 10

11. Huffman coding 11

12. Huffman coding 12

13. Huffman coding 13

14. Huffman coding 14

15. Huffman coding 15

16. Huffman coding 16

17. Huffman coding 17

18. Huffman coding 18

19. Graphs 19

20. Graphs We denote a graph by G=(V, E) where V = Set of vertices and E = Set of edges |V| = Number of vertices and |E| = Number of edges G is directed if the edges are, otherwise G is undirected. 20

21. Graph representation 21

22. Graph representation 22

23. Graph Representation 23

24. Breadth-first search Also called: level order search What does it do? Given: G=(V, E), a directed or undirected graph and a “source” vertex s  V It systematically explores vertices reachable from s, to construct a breadth-first search tree 24

25. Shortest Path Length of the path from s to each reachable vertex is the shortest path from s to this vertex Example graph 25

26. Expand node s 26

27. Expand node w 27

28. Expand node r 28

29. Expand node t 29

30. Expand node x 30

31. Expand node v 31

32. Expand node u 32

33. Breath-first Search Tree 33

34. Formal Algorithm Colour terminology: Gray nodes make up the “frontier” (red line) White nodes (vertices) are the unexplored ones Black nodes are the completely explored ones, lying on the other side of the “frontier” 34

35. Breadth-first search algorithm BFS(G, s) For each vertex u  V –{s} {// Setup loop O(|V|) u.color  WHITE; u.distance   ; u.parent  NULL; // parent of u } s.color  GRAY; s.distance  0; s.parent  NULL; Q  Q  {s}; (Cont’d.) 35

36. Breadth-first search algorithm While (Q   ) {// Processing loop O(|E|) u  head (Q); for each v adjacent to u { if (v.color = WHITE) { v.color  GRAY; v.distance  u.distance+1; v.parent  u; Q  Q  {v}; } Delete u from Q; u.color  BLACK; } 36

37. Analysis Thm.Running time of BFS is O (|V|+|E|) PfQueuing operations take O (|V|) time. Each edge in the graph is looked at most once. Hence O (|E|) time and the theorem follows. 37

38. We can prove the following facts Fact1:Breath-first-search (BFS) discovers every vertex reachable from s Fact2:The length of the path from s to a reachable vert ex v, is a shortest path from s to v. 38

39. Weighted Graph Each edge has a weight (a number). 39

40. Minimum Spanning Tree (MST) Green edges form a minimum spanning tree of the example graph 40

41. Minimum Spanning Tree Weight of the MST is: W(T) = 8 + 2 + 4 + 7 + 4 + 2 + 9 + 1 = 37 41

42. Formally Given a connected, weighed, undirected graph G=(V, E), Find an acyclic subset T  E that connects all the vertices such that is minimum, where weight(u,v) is the weight of the edges connecting u to v. 42

43. Growing an MST, greedily Let A be a subset of edges of some MST Def. An edge e=(u,v) of G is safe for A if A  {(u,v)} is also a subset of an MST. 43

44. A generic MST algorithm GENERIC_MST(G,w) A   ; while (A does not form an MST) find a safe edge (u,v) for A; A  A  {(u,v)}; return A; 44

45. Cuts and safe edges Definition: A cut of G is a partition (S, V-S) of V 45

46. A few definitions Definition: An edge (u,v)  E crosses the cut (S,V-S) if one of its end points is in S and the other in V-S Definition: A cut respects the set A if no edge in A crosses the cut Definition: An edge is a light edge crossing a cut if its weight is the minimum of any edge crossing the cut. 46

47. Safe Edges The following theorem characterizes a safe edge for A Theorem Let A be a subset of E that is included in some MST for G. Let (S,V-S) be any cut of G that respects A, and Let (u,v) be a light edge crossing (S,V-S). Then, edge (u,v) is safe for A. 47

48. Prim’s algorithm Select vertex a S={a} 48

49. (a, b) is a light edge for (S, V-S) Select vertex b new S={a, b} 49

50. (b,c) is a light edge for (S,V-S) Select vertex c new S={a, b, c} 50

51. (c, i) is a light edge for (S,V-S) where S={a, b, c} Select vertex i new S={a, b, c, i} 51

52. (c, f) is a light edge for (S,V-S) Select vertex f new S={a, b, c, f, i} 52

53. (f, g) is a light edge for (S,V-S) Select vertex g new S={a, b, c, g, f, i} 53

54. (g, h) is a light edge for (S,V-S) Select vertex h new V-S={d, e} 54

55. (c, d) is a light edge for (S,V-S) Select vertex d new V-S={e} 55

56. (d, e) is a light edge for (S,V-S) Select vertex e new V-S=  56

57. Formal Algorithm MST_Prim(G, W, r) Q  V; for (each u  Q) u.key   ; r.key  0; r.parent  NULL; while ( Q   ) u  extract_min(Q); for (each v adjacent to u) if v  Q and weight(u,v) < v.key v.parent  u; v.key  weight(u,v) 57

58. Important Note on Shortest Path: Dijkstra’s algorithm The algorithm incrementally constructs a set S of vertices to which the shortest path is known. It adds a vertex v (from the remaining vertices) to S, whose distance from S (source) is the shortest of the remaining vertices. The assumption of non-negative weights guarantees that the shortest path to v passes only through vertices in S. 58

59. Priority Queues Let S be a set of keys, a priority queue supports the following operations on S: 1.Insert (S, x) 2.Minimum (S)-- returns min key 3. Extract_Min (S)--returns and removes min key 59

60. Heap as Priority Queue A priority queue can be implemented using a heap A heap is a complete binary tree, satisfying the heap property: For a node v, v.key  v.left.key v.key  v.right.key 60

61. Single-source shortest paths Let G be a weighted, directed graph. A path in the graph from u to v is a sequence of edges that connects u to v. The weight of this path is the sum of the weights of its constituent edges. Let  (u, v) denote a minimum weight path from u to v Let s be a source vertex of G 61

62. Single-source shortest path problem Compute a shortest path from s to each vertex v. Remark:A shortest path is a path of minimum weight. Dijkstra’s algorithm: We assume that the weights of the edges are non-negative. 62

63. Relaxation 63

64. Relax Relax(u,v,w) { If (v.distance > u.distance +w(u,v)) { v.distance  u.distance +w(u,v); v.parent  u; } Initialize_Simple_Source (G, v) { For each vertex v  V [G]; v.distance   ; v.parent  NULL; } 64

65. Dijkstra’s algorithm by example The algorithm maintains a set X of vertices whose final shortest-path weights from s have already been determined 65

66. Add s to X Relax vertices adjacent to s 66

67. Add x to X Relax vertices adjacent to x 67

68. Add y to X Relax vertices adjacent to y 68

69. Add u to X Relax vertices adjacent to u 69

70. Add v to X Done 70

71. Formal Algorithm Dijkstra(G,w,s) Initialize_single_source(G,s) X  Q  V [G] while (Q   ) u  extract_min(Q) X  X  {u} For each vertex v adjacent to u Relax(u,v,w) 71