1. 241-303 Discrete Maths: Trees/6 1 Discrete Maths Objective – –introduce more unusual tree algorithms and techniques you already know about binary (search) trees 241-303, Semester 1 2014-2015 6. Trees

2. 241-303 Discrete Maths: Trees/6 2 Overview 1. Uses of Trees 2. Huffman Codes 3.(Rooted) Tree Terminology 4.Spanning Trees 5.Minimal Spanning Trees 6.Game Trees 7.More Information

3. 241-303 Discrete Maths: Trees/6 3 1. Uses of Trees Davenport S. Williams Bartoli V. Williams S. Williams V. Williams S. Williams Wimbledon Womens Tennis

4. 241-303 Discrete Maths: Trees/6 4 Organizational Chart President Vice-President for Academics Vice-President for Admin. Dean of Engineering Dean of Business Head of CoEHead of EEHead of AC. Planning Officer Purchases Officer..

5. 241-303 Discrete Maths: Trees/6 5 Saturated Hydrocarbons H HH H C HH C HH C HH C H C H H HH C H H C H H C H Butane Isobutane Non-rooted (free) trees – –a free tree is a graph with no cycles

6. 241-303 Discrete Maths: Trees/6 6 A Computer File System / usrbintmp binadspoollsmailwhojunk edviexsopruucp printer

7. 241-303 Discrete Maths: Trees/6 7 2. Huffman Codes A Huffman code represents characters by variable-length bit strings – –there are many different Huffman codes By comparison, ASCII uses fixed-length codes. – –CharacterASCII code A1000001 B1000010 : : S1010011 : :

8. 241-303 Discrete Maths: Trees/6 8 A Huffman Code in Tree Form Root 10 01 01 01 A O R ST Character Code A1 O00 R010 S0110 T0111 where do these come from?

9. 241-303 Discrete Maths: Trees/6 9 Huffman uses Less Bits ASCII coding of "RATS": – –1010010 1000001 1010100 1010011 – –4*7 = 28 bits Huffman Coding (as in the tree): – –010 1 0111 0110 – –12 bits useful for network comms, text compression, etc.

10. 241-303 Discrete Maths: Trees/6 10 Constructing a Huffman Code A Huffman code is based on a table giving the frequency of the characters in the 'language'. The 'language' can be a subset of a full language – –e.g. business English, computer language keywords, the months of the year, a language made up of only {A, O, R, S, T}

11. 241-303 Discrete Maths: Trees/6 11 Letter Frequencies for English The most common letters, in full English, in order of frequency, are usually ETAOINSHRDLU. This varies depending on the data source – –e.g. novels, textbooks, cartoons

12. 241-303 Discrete Maths: Trees/6 12 Example: The Haddock Language The Haddock language contains only 5 characters: – –!, @, #, $, % – –we can make words like: !!!@@$%$#@@@@@@@@@ We must create a frequency table by analysing existing examples of Haddock writing. continued

13. 241-303 Discrete Maths: Trees/6 13 The frequency table we create: – –CharacterFrequency ! 2/32 @ 3/32 # 7/32 $ 8/32 % 12/32 We work with the numerators of the fractions.

14. 241-303 Discrete Maths: Trees/6 14 Informal Algorithm 1. Make a sequence of the frequencies: { 2, 3, 7, 8, 12} 2. Repeatedly replace the smallest two frequencies by a sum until only 2 elements are left: – –{2,3,7,8,12} ==> {2+3, 7, 8, 12} {5,7,8,12}==> {5+7, 8, 12} {8,12,12}==> {8+12, 12} {12,20} continued

15. 241-303 Discrete Maths: Trees/6 15 3. Build a tree by expanding the sequence back to its full size. – –Start with the 2 elements as a 2 branch tree. 01 2012 ==> 10 01 12 8 ==> 1 0 01 128 01 75 ==> continued

16. 241-303 Discrete Maths: Trees/6 16 4. Replace each frequency by the character having that frequency: ==> 1 0 01 128 01 701 32 1 0 01 %$ 01 #01 @! continued

17. 241-303 Discrete Maths: Trees/6 17 Huffman Code for Haddock CharacterCode !111 @110 #10 $01 %00 The higher frequency letters have shorter Huffman codes.

18. 241-303 Discrete Maths: Trees/6 18 Another Possible Answer Why? There are two "12" nodes (see slide 12), either of which can be expanded back to "5" and "7". 1 0 0112 80 1 701 32 continued

19. 241-303 Discrete Maths: Trees/6 19 Huffman Code for Haddock v.2 CharacterCode !0011 @0010 #000 $01 %1 The higher frequency letters have shorter Huffman codes.

20. 241-303 Discrete Maths: Trees/6 20 Huffman Algorithm Tree huffman(FreqSeq f, int size) { if (size == 2){ let f1, f2 be the frequencies in f return baseTree; else { let fi, fj be the smallest freqs in f; replace fi and fj in f with fi+fj; Tree t1 = huffman(f, size-1); Tree t = replace "fi+fj" vertex in t1 with extTree return t; } } 01 f2f1 baseTree 01 fjfi extTree call: Tree t = huffman( {2,3,7,8,12}, 5);

21. 241-303 Discrete Maths: Trees/6 21 3. (Rooted) Tree Terminology e.g. Part of the ancient Greek god family: Uranus AphroditeKronosAtlasPrometheus ErosZeusPoseidonHadesAres ApolloAthenaHermesHeracles levels 0 1 2 3 ::::

22. 241-303 Discrete Maths: Trees/6 22 Some Definitions Let T be a tree with root v 0. Suppose that x, y, z are verticies in T. (v 0, v 1,..., v n ) is a simple path in T (no loops). a) v n-1 is the parent of v n. b) v 0,..., v n-1 are ancestors of v n c) v n is a child of v n-1 continued

23. 241-303 Discrete Maths: Trees/6 23 d) If x is an ancestor of y, then y is a descendant of x. e) If x and y are children of z, then x and y are siblings. f) If x has no children, then x is a terminal vertex (or a leaf). g) If x is not a terminal vertex, then x is an internal (or branch) vertex. continued

24. 241-303 Discrete Maths: Trees/6 24 h) The subtree of T rooted at x is the graph with vertex set V and edge set E – –V contains x and all the descendents of x – –E = {e | e is an edge on a simple path from x to some vertex in V} i) The length of a path is the number of edges it uses, not verticies. continued

25. 241-303 Discrete Maths: Trees/6 25 j) The level of a vertex x is the length of the simple path from the root to x. k) The height of a vertex x is the length of the simple path from x to the farthest leaf – –the height of a tree is the height of its root l) A tree where every internal vertex has exactly m children is called a full m-ary tree.

26. 241-303 Discrete Maths: Trees/6 26 Applied to the Example The root is Uranus. A simple path is {Uranus, Aphrodite, Eros} The parent of Eros is Aphrodite. The ancestors of Hermes are Zeus, Kronos, and Uranus. The children of Zeus are Apollo, Athena, Hermes, and Heracles. continued

27. 241-303 Discrete Maths: Trees/6 27 The descendants of Kronos are Zeus, Poseidon, Hades, Ares, Apollo, Athena, Hermes, and Heracles. The leaves (terminal verticies) are Eros, Apollo, Athena, Hermes, Heracles, Poseidon, Hades, Ares, Atlas, and Prometheus. The branches (internal verticies) are Uranus, Aphrodite, Kronos, and Zeus. continued

28. 241-303 Discrete Maths: Trees/6 28 The subtree rooted at Kronos: Kronos ZeusPoseidonHadesAres ApolloAthenaHermesHeracles continued

29. 241-303 Discrete Maths: Trees/6 29 The length of the path {Uranus, Aphrodite, Eros} is 2 (not 3). The level of Ares is 2. The height of the tree is 3.

30. 241-303 Discrete Maths: Trees/6 30 Some Properties of Trees 1) A tree with n verticies has n-1 edges. – –e.g. Proof: – –ignore the root; – –pair each vertex with the edge above it; – –so n-1 verticies use n-1 edges (all of them)

31. 241-303 Discrete Maths: Trees/6 31 2) A full m-ary tree with i internal verticies has a total of m*i + 1 verticies. – –e.g. A full 2-ary tree (binary tree) 4 internal verticies A total of 9 verticies continued

32. 241-303 Discrete Maths: Trees/6 32 Proof – –ignore the root; – –every remaining vertex is the child of an internal vertex; – –each internal vertex has m children; – –there are i internal verticies, so there is a total of m*i verticies in the tree; – –add back the root, to get m*i + 1

33. 241-303 Discrete Maths: Trees/6 33 4. Spanning Trees A spanning tree T is a subgraph of a graph G which contains all the verticies of G. Example graph G: a b dc ef h g continued

34. 241-303 Discrete Maths: Trees/6 34 One possible spanning tree: a b dc ef h g the tree is drawn with thick lines

35. 241-303 Discrete Maths: Trees/6 35 4.1. Example: IP Multicasting A network of computers and routers: source computer router continued

36. 241-303 Discrete Maths: Trees/6 36 How can a packet (message) be sent from the source computer to every other computer? The inefficient way is to use broadcasting – –send a copy along every link, and have each router do the same – –each router and computer will receive many copies of the same packet – –loops may mean the packet never disappears! continued

37. 241-303 Discrete Maths: Trees/6 37 IP multicasting is an efficient solution – –send a single packet to one router – –have the router send it to 1 or more routers in such a way that a computer never receives the packet more than once This behaviour can be represented by a spanning tree. continued

38. 241-303 Discrete Maths: Trees/6 38 The spanning tree for the network: source computer router the tree is drawn with thick lines

39. 241-303 Discrete Maths: Trees/6 39 4.2. Finding a Spanning Tree There are two main types of algorithms: – –breadth-first search – –depth-first search

40. 241-303 Discrete Maths: Trees/6 40 4.3. Breadth-first Search Process all the verticies at a given level before moving to the next level. Example graph G (again): a b dc ef h g

41. 241-303 Discrete Maths: Trees/6 41 Informal Algorithm 1) Put the verticies into an ordering – –e.g. {a, b, c, d, e, f, g, h} 2) Select a vertex, add it to the spanning tree T: e.g. a 3) Add to T all edges (a,X) and X verticies that do not create a cycle in T – –i.e. (a,b), (a,c), (a,g) T = {a, b, c, g} continued a bc g

42. 241-303 Discrete Maths: Trees/6 42 Repeat step 3 on the verticies just added, these are on level 1 – –i.e. b: add (b,d) c: add (c,e) g: nothing T = {a,b,c,d,e} Repeat step 3 on the verticies just added, these are on level 2 – –i.e. d: add (d,f) e: nothing T = {a,b,c,d,e,f} a bc g de a bc g de f continued level 1 level 2

43. 241-303 Discrete Maths: Trees/6 43 Repeat step 3 on the verticies just added, these are on level 3 – –i.e. f: add (f,h) T = {a,b,c,d,e,f,h} Repeat step 3 on the verticies just added, these are on level 4 – –i.e. h: nothing, so stop a bc g de f h continued level 3

44. 241-303 Discrete Maths: Trees/6 44 Resulting spanning tree: a b dc ef h g a different spanning tree from the earlier solution

45. 241-303 Discrete Maths: Trees/6 45 Breadth-first Algorithm Tree bfs(Verts vsG, Edges edsG) // input ordered verts and edges of graph G { Vert v1 = firstVert(VsG); // first vert in G Seq levelVs = ( v1 ); // verts at curr level Verts vsT = { v1 }; // verts in tree Edges edsT = {}; // edges in tree while (1) { for each x in levelVs for each y in vsG - vsT if (x,y) is an edge in edsG add (x,y) to edsT; add y to vsT; if no edges added to vsT return tree made from vsT and edsT; levelVs = childrenOf( levelVs ); } } continued

46. 241-303 Discrete Maths: Trees/6 46 Typical call: Verts vsG = {a, b, c, d, e, f, g, h}; Edges edsG = { (a,b), (a,c), (a,g), (b,d), (b,g), (c,d), (c,e), (d,f), (e,g), (f,h) }; Tree spanTree = bfs( vsG, edsG ); :

47. 241-303 Discrete Maths: Trees/6 47 4.4. Depth-first Search Process all the verticies down one path, then backtrack (go back) to verticies along other paths. Example graph G (again): a b dc ef h g

48. 241-303 Discrete Maths: Trees/6 48 Informal Algorithm 1) Put the verticies into an ordering – –e.g. {a, b, c, d, e, f, g, h} 2) Select a vertex, add it to the spanning tree T: e.g. a 3) Add the edge (a,X) where X is the smallest vertex in the ordering, and does not make a cycle in T – –i.e. (a,b), T = {a, b} continued a b

49. 241-303 Discrete Maths: Trees/6 49 4) Repeat step 3 with the new vertex, until there is no possible new vertex – –i.e. add the edges (b,d) (d,c) (c,e) (e,f) (f,h) T = {a,b,d,c,e,f,h} 5) At this point, there is no (h,X), so backtrack to a vertex that does have another edge: – –parent of h == f but there is no new (f,X) to add, so backtrack continued a b d c e f h

50. 241-303 Discrete Maths: Trees/6 50 – –parent of f == e – –there is an (e,g) to add, so repeat step 3 with e 6) After g is added, there are no further verticies to add, so stop. a b d c e f h g continued

51. 241-303 Discrete Maths: Trees/6 51 Resulting spanning tree: a b dc ef h g a different spanning tree from the breadth-first solution

52. 241-303 Discrete Maths: Trees/6 52 4.5. Another Backtracking Problem Find a subset of {31,27,15,11,7,5} with a sum that equals 39. – –one answer is {27,7,5} How?: build up the sum by adding integers. An integer is included if the sum remains less than 39. – –backtracking is required to try different integers when the sum doesn't reach 39. Summing a Subset

53. 241-303 Discrete Maths: Trees/6 53 Backtracking Diagram {} sum = 0 {31} sum = 31 {31,7} sum = 38 {31,5} sum = 36 {27} sum = 27 {27,11} sum = 38 {27,7} sum = 34 {27,7,5} sum = 39

54. 241-303 Discrete Maths: Trees/6 54 Prolog Code and Query sum(0, _, []). sum(Sum, List, [X|Used]) :- Sum > 0, select(List, X, List1), Sum1 is Sum - X, sum(Sum1, List1, Used). select([X|L], X, L). select([Y|L], X, [Y|L1]) :- select(L, X, L1). ?- sum(39, [31,27,15,11,7,5], Used). Used = [27,7,5] It can calculate sums of any set.

55. 241-303 Discrete Maths: Trees/6 55 Amzi-Prolog Execution ; means "give another answer"

56. 241-303 Discrete Maths: Trees/6 56 5. Minimal Spanning Tree A minimal spanning tree T is a subgraph of a weighted graph G which contains all the verticies of G and whose edges have the minimum summed weight. Example weighted graph G: A B C D E F 3 4 5 1 63 2 6 2

57. 241-303 Discrete Maths: Trees/6 57 A minimal spanning tree (weight = 12): A non-minimal spanning tree (weight = 20): A B C D E F 3 4 5 1 63 2 2 6 A B C D E F 3 4 5 1 63 2 6 2

58. 241-303 Discrete Maths: Trees/6 58 5.1. Prim's Algorithm Prim's algorithm finds a minimal spanning tree T by iteratively adding edges to T. At each iteration, a minimum-weight edge is added that does not create a cycle in the current T.

59. 241-303 Discrete Maths: Trees/6 59 Informal Algorithm For the graph G. 1) Add any vertex to T – –e.g A,T = {A} 2) Examine all the edges leaving {A} and add the vertex with the smallest weight. – –edgeweight (A,B) 4 (A,C) 2 (A,E) 3 – –add edge (A,C), T becomes {A,C} continued A B C D E F 3 4 5 1 63 2 6 2

60. 241-303 Discrete Maths: Trees/6 60 3) Examine all the edges leaving {A,C} and add the vertex with the smallest weight. – –edgeweightedgeweight (A,B) 4(C,D) 1 (A,E) 3(C,E) 6 (C,F) 3 – –add edge (C,D), T becomes {A,C,D} continued

61. 241-303 Discrete Maths: Trees/6 61 4) Examine all the edges leaving {A,C,D} and add the vertex with the smallest weight. – –edgeweightedgeweight (A,B) 4(D,B) 5 (A,E) 3(C,E) 6 (C,F) 3(D,F) 6 – –add edge (A,E) or (C,F), it does not matter – –add edge (A,E), T becomes {A,C,D,E} continued

62. 241-303 Discrete Maths: Trees/6 62 5) Examine all the edges leaving {A,C,D,E} and add the vertex with the smallest weight. – –edgeweightedgeweight (A,B) 4(D,B) 5 (C,F) 3(D,F) 6 (E,F) 2 – –add edge (E,F), T becomes {A,C,D,E,F} continued

63. 241-303 Discrete Maths: Trees/6 63 6) Examine all the edges leaving {A,C,D,E,F} and add the vertex with the smallest weight. – –edgeweightedgeweight (A,B) 4(D,B) 5 – –add edge (A,B), T becomes {A,B,C,D,E,F} All the verticies of G are now in T, so we stop. continued

64. 241-303 Discrete Maths: Trees/6 64 Resulting minimum spanning tree (weight = 12): A B C D E F 3 4 5 1 63 2 2 6

65. 241-303 Discrete Maths: Trees/6 65 Pseudocode Tree prim(Graph G, int numVerts) { Tree T = anyVert(G); for i = 1 to numVerts-1{ Edge e = an edge of minimum weight to a vertex in T and not forming a cycle in T when added to T; T = T with e added; } return T }

66. 241-303 Discrete Maths: Trees/6 66 5.2. A Greedy Algorithm Prim's algorithm is an example of a greedy algorithm – –its choice at each iteration does not depend on any previous choices A greedy algorithm does the best thing at this time without considering the past (or the future).

67. 241-303 Discrete Maths: Trees/6 67 Greed is not Always Best An example where greed is not the best approach: – –a greedy shortest-path algorithm can lead to a non-optimal solution (not the best) – –the algorithm selects an edge with the minimum weight connected to the last vertex added continued

68. 241-303 Discrete Maths: Trees/6 68 Find the shortest path a --> z. The greedy algorithm produces (a,c,z), but the shortest is (a,b,z) a b c z 2 4 6 1 8

69. 241-303 Discrete Maths: Trees/6 69 5.3. Kruskal's Minimum Spanning Tree Algorithm At the start, the minimum spanning tree T consists of all the verticies of the weighted graph G, but no edges. At each iteration, add an edge e to T having minimum weight that does not create a cycle in T. When T has n-1 edges, stop.

70. 241-303 Discrete Maths: Trees/6 70 Pseudocode Tree kruskal(Graph G, int numVerts) { Tree T = allVerts(G); for i = 1 to numVerts-1 { Edge e = an edge of minimum weight in G and not forming a cycle in T when added to T; T = T with e added; } return T }

71. 241-303 Discrete Maths: Trees/6 71 Example Graph G: a bc d e f g h i j k l 231 3 4 3 42 1 3 2 3 4 3 31 3 iteredge 1(c,d) 2(k,l) 3(b,f) 4(c,g) 5(a,b) 6(f, j) 7(b,c) 8(j,k) 9(g,h) 10(i, j) 11(a,e) continued

72. 241-303 Discrete Maths: Trees/6 72 Minimum spanning tree (weight = 24): a bc d e f g h i j k l 231 3 4 3 42 1 3 2 3 4 3 31 3

73. 241-303 Discrete Maths: Trees/6 73 5.4. Difference between Prim and Kruskal Prim's algorithm chooses an edge that must already be connected to a vertex in the minimum spanning tree T. Kruskal's algorithm can choose an edge that may not already be connected to a vertex in T.

74. 241-303 Discrete Maths: Trees/6 74 6. Game Trees Trees can be used to develop game-playing strategies – –used in many AI (Artificial Intelligence) game programs where the human is competing against the 'clever' machine

75. 241-303 Discrete Maths: Trees/6 75 6.1. Nim In Nim there are two piles of stones. Each player takes 1 or more stones from one pile. The player who removes the last stone loses. For example, the pile (3,2):

76. 241-303 Discrete Maths: Trees/6 76 Part of a Nim Game Tree (3,2) (2,2)(1,2)(0,2)(3,1)(3,0) (1,2)(0,2)(2,1)(2,0)(0,2)(1,1)(1,0) (1,1)(0,1)(2,0) (1,0)(0,1) (0,0) (1,0)(0,0) (0,1)(1,0) (0,0) (2,0)(1,0)(0,0) (1,0)(0,0).. 1st player goes now 2nd player goes now

77. 241-303 Discrete Maths: Trees/6 77 Analysing the Game Tree Label each terminal vertex with the first player's score: – –if the first player won, write "1" – –if the first player lost, write "0" continued

78. 241-303 Discrete Maths: Trees/6 78 – –means that it is the 1st player’s turn – –means 1st player wins since there are no stones left – –so assign it – –means that it is the 2nd player’s turn – –means 2nd player wins since there are no stones left. So the 1st player loses. – –so assign it Graphically ( 0, 0 ) 1 0

79. 241-303 Discrete Maths: Trees/6 79 Case 1. Label a box: The box is the first player, who will move to the most valuable child (to win). So label the box with the maximum child value (e.g. 1). 010 continued max Moving up the Tree

80. 241-303 Discrete Maths: Trees/6 80 Case 2. Label an oval The oval is the second player, who will move to the least valuable child (so the first player loses). So label the box with the minimum child value (e.g. 0). 110 min

81. 241-303 Discrete Maths: Trees/6 81 The Minimax Algorithm In many games: – –the first player chooses the most valuable position to move to; and – –the second player chooses the least valuable position for the first player The minimax algorithm: – –first player seeks the max of the child values – –second player seeks the min of the child values

82. 241-303 Discrete Maths: Trees/6 82.. Nim Game Tree Analysis (part) 1 10000 1111110 010 00 00 101 0 11 11 101 100 1.. 1st player 2nd player continued max min max min max

83. 241-303 Discrete Maths: Trees/6 83 The root value (the starting position) contains a "1" – –this means that the first player will definitely win if he/she chooses the most valuable child position to move to

84. 241-303 Discrete Maths: Trees/6 84 Larger Game Trees Games like Chess or Go generate game trees that are too large to analyse using a simple minimax algorithm. Instead the game tree is generated to a certain level, and then an evaluation function is used to 'calculate' the scores at that level for the first player. Minimax is then used to work back to the root.

85. 241-303 Discrete Maths: Trees/6 85 Tic-Tac-Toe.. X X X XXXX X O O O O O Game tree generated to level 2 (with symmetric positions ignored).

86. 241-303 Discrete Maths: Trees/6 86 The Evaluation Function For a given position P: – –eval(P) = numWinsX - numWinsO; – –numWinsX = number of rows, columns, and diagonals containing an X that the X player might be able to complete in the future – –numWinsO = same, but for O continued

87. 241-303 Discrete Maths: Trees/6 87 Example position: eval(P) = numWinsX - numWinsO = 2 - 1 = 1 If eval(P) is positive, then X has more ways to win than O. XO

88. 241-303 Discrete Maths: Trees/6 88 Analyse the Tic-Tac-Toe Tree 0100 -20012 1 -21 This means that the best move for the first player (X) is the center square. max min

89. 241-303 Discrete Maths: Trees/6 89 Alpha-Beta Pruning Evaluating a game tree with a large level takes too much time. Alpha-beta pruning reduces the number of positions (verticies) that need to be evaluated, and still gives the right answer. There are two parts to alpha-beta pruning: – –alpha cutoff – –beta cutoff

90. 241-303 Discrete Maths: Trees/6 90 Alpha Cutoff An example: 3264 1 ??? x 2 y? ? A BCD MLKJIHGFE alpha cutoff continued Evaluate children left-to-right. max min max

91. 241-303 Discrete Maths: Trees/6 91 In A, x must be >= 2 – –the lower bound for A, the alpha value for A In C, y must be <= 1 – –since y <= x, it cannot affect x, so there is no need to evaluate J – –also the C subtree can be ignored (cutoff) Alpha cutoff occurs when a grandchild of A (e.g. I) has a value <= the alpha value of A.

92. 241-303 Discrete Maths: Trees/6 92 Beta Cutoff An example: 326 8 ????? y? 6 x max min A1 B1C1D1 E1F1G1H1I1J1K1L1M1 beta cutoff continued max min

93. 241-303 Discrete Maths: Trees/6 93 In A1, x must be <= 6 – –the upper bound for A1, the beta value for A1 In C1, y must be >= 8 – –since y >= x, it cannot affect x, so there is no need to evaluate I1 or J1 – –also the C1 subtree can be ignored (cutoff) Beta cutoff occurs when a grandchild of A1 (e.g. H1) has a value >= the beta value of A1.

94. 241-303 Discrete Maths: Trees/6 94 7. More Information DM: chapter 7 Rosen: chapter 8