Discrete Maths: Trees/6 1 Discrete Maths Objective – –introduce more unusual tree algorithms and techniques you already know about binary (search) trees , Semester Trees

Discrete Maths: Trees/6 2 Overview 1. Uses of Trees 2. Huffman Codes 3.(Rooted) Tree Terminology 4.Spanning Trees 5.Minimal Spanning Trees 6.Game Trees 7.More Information

Discrete Maths: Trees/ Uses of Trees Davenport S. Williams Bartoli V. Williams S. Williams V. Williams S. Williams Wimbledon Womens Tennis

Discrete Maths: Trees/6 4 Organizational Chart President Vice-President for Academics Vice-President for Admin. Dean of Engineering Dean of Business Head of CoEHead of EEHead of AC. Planning Officer Purchases Officer..

Discrete Maths: Trees/6 5 Saturated Hydrocarbons H HH H C HH C HH C HH C H C H H HH C H H C H H C H Butane Isobutane Non-rooted (free) trees – –a free tree is a graph with no cycles

Discrete Maths: Trees/6 6 A Computer File System / usrbintmp binadspoollsmailwhojunk edviexsopruucp printer

Discrete Maths: Trees/ Huffman Codes A Huffman code represents characters by variable-length bit strings – –there are many different Huffman codes By comparison, ASCII uses fixed-length codes. – –CharacterASCII code A B : : S : :

Discrete Maths: Trees/6 8 A Huffman Code in Tree Form Root A O R ST Character Code A1 O00 R010 S0110 T0111 where do these come from?

Discrete Maths: Trees/6 9 Huffman uses Less Bits ASCII coding of "RATS": – – – –4*7 = 28 bits Huffman Coding (as in the tree): – – – –12 bits useful for network comms, text compression, etc.

Discrete Maths: Trees/6 10 Constructing a Huffman Code A Huffman code is based on a table giving the frequency of the characters in the 'language'. The 'language' can be a subset of a full language – –e.g. business English, computer language keywords, the months of the year, a language made up of only {A, O, R, S, T}

Discrete Maths: Trees/6 11 Letter Frequencies for English The most common letters, in full English, in order of frequency, are usually ETAOINSHRDLU. This varies depending on the data source – –e.g. novels, textbooks, cartoons

Discrete Maths: Trees/6 12 Example: The Haddock Language The Haddock language contains only 5 characters: – #, $, % – –we can make words like: We must create a frequency table by analysing existing examples of Haddock writing. continued

Discrete Maths: Trees/6 13 The frequency table we create: – –CharacterFrequency ! 3/32 # 7/32 $ 8/32 % 12/32 We work with the numerators of the fractions.

Discrete Maths: Trees/6 14 Informal Algorithm 1. Make a sequence of the frequencies: { 2, 3, 7, 8, 12} 2. Repeatedly replace the smallest two frequencies by a sum until only 2 elements are left: – –{2,3,7,8,12} ==> {2+3, 7, 8, 12} {5,7,8,12}==> {5+7, 8, 12} {8,12,12}==> {8+12, 12} {12,20} continued

Discrete Maths: Trees/ Build a tree by expanding the sequence back to its full size. – –Start with the 2 elements as a 2 branch tree ==> ==> ==> continued

Discrete Maths: Trees/ Replace each frequency by the character having that frequency: ==> %$ 01 continued

Discrete Maths: Trees/6 17 Huffman Code for Haddock CharacterCode #10 $01 %00 The higher frequency letters have shorter Huffman codes.

Discrete Maths: Trees/6 18 Another Possible Answer Why? There are two "12" nodes (see slide 12), either of which can be expanded back to "5" and "7" continued

Discrete Maths: Trees/6 19 Huffman Code for Haddock v.2 CharacterCode #000 $01 %1 The higher frequency letters have shorter Huffman codes.

Discrete Maths: Trees/6 20 Huffman Algorithm Tree huffman(FreqSeq f, int size) { if (size == 2){ let f1, f2 be the frequencies in f return baseTree; else { let fi, fj be the smallest freqs in f; replace fi and fj in f with fi+fj; Tree t1 = huffman(f, size-1); Tree t = replace "fi+fj" vertex in t1 with extTree return t; } } 01 f2f1 baseTree 01 fjfi extTree call: Tree t = huffman( {2,3,7,8,12}, 5);

Discrete Maths: Trees/ (Rooted) Tree Terminology e.g. Part of the ancient Greek god family: Uranus AphroditeKronosAtlasPrometheus ErosZeusPoseidonHadesAres ApolloAthenaHermesHeracles levels ::::

Discrete Maths: Trees/6 22 Some Definitions Let T be a tree with root v 0. Suppose that x, y, z are verticies in T. (v 0, v 1,..., v n ) is a simple path in T (no loops). a) v n-1 is the parent of v n. b) v 0,..., v n-1 are ancestors of v n c) v n is a child of v n-1 continued

Discrete Maths: Trees/6 23 d) If x is an ancestor of y, then y is a descendant of x. e) If x and y are children of z, then x and y are siblings. f) If x has no children, then x is a terminal vertex (or a leaf). g) If x is not a terminal vertex, then x is an internal (or branch) vertex. continued

Discrete Maths: Trees/6 24 h) The subtree of T rooted at x is the graph with vertex set V and edge set E – –V contains x and all the descendents of x – –E = {e | e is an edge on a simple path from x to some vertex in V} i) The length of a path is the number of edges it uses, not verticies. continued

Discrete Maths: Trees/6 25 j) The level of a vertex x is the length of the simple path from the root to x. k) The height of a vertex x is the length of the simple path from x to the farthest leaf – –the height of a tree is the height of its root l) A tree where every internal vertex has exactly m children is called a full m-ary tree.

Discrete Maths: Trees/6 26 Applied to the Example The root is Uranus. A simple path is {Uranus, Aphrodite, Eros} The parent of Eros is Aphrodite. The ancestors of Hermes are Zeus, Kronos, and Uranus. The children of Zeus are Apollo, Athena, Hermes, and Heracles. continued

Discrete Maths: Trees/6 27 The descendants of Kronos are Zeus, Poseidon, Hades, Ares, Apollo, Athena, Hermes, and Heracles. The leaves (terminal verticies) are Eros, Apollo, Athena, Hermes, Heracles, Poseidon, Hades, Ares, Atlas, and Prometheus. The branches (internal verticies) are Uranus, Aphrodite, Kronos, and Zeus. continued

Discrete Maths: Trees/6 28 The subtree rooted at Kronos: Kronos ZeusPoseidonHadesAres ApolloAthenaHermesHeracles continued

Discrete Maths: Trees/6 29 The length of the path {Uranus, Aphrodite, Eros} is 2 (not 3). The level of Ares is 2. The height of the tree is 3.

Discrete Maths: Trees/6 30 Some Properties of Trees 1) A tree with n verticies has n-1 edges. – –e.g. Proof: – –ignore the root; – –pair each vertex with the edge above it; – –so n-1 verticies use n-1 edges (all of them)

Discrete Maths: Trees/6 31 2) A full m-ary tree with i internal verticies has a total of m*i + 1 verticies. – –e.g. A full 2-ary tree (binary tree) 4 internal verticies A total of 9 verticies continued

Discrete Maths: Trees/6 32 Proof – –ignore the root; – –every remaining vertex is the child of an internal vertex; – –each internal vertex has m children; – –there are i internal verticies, so there is a total of m*i verticies in the tree; – –add back the root, to get m*i + 1

Discrete Maths: Trees/ Spanning Trees A spanning tree T is a subgraph of a graph G which contains all the verticies of G. Example graph G: a b dc ef h g continued

Discrete Maths: Trees/6 34 One possible spanning tree: a b dc ef h g the tree is drawn with thick lines

Discrete Maths: Trees/ Example: IP Multicasting A network of computers and routers: source computer router continued

Discrete Maths: Trees/6 36 How can a packet (message) be sent from the source computer to every other computer? The inefficient way is to use broadcasting – –send a copy along every link, and have each router do the same – –each router and computer will receive many copies of the same packet – –loops may mean the packet never disappears! continued

Discrete Maths: Trees/6 37 IP multicasting is an efficient solution – –send a single packet to one router – –have the router send it to 1 or more routers in such a way that a computer never receives the packet more than once This behaviour can be represented by a spanning tree. continued

Discrete Maths: Trees/6 38 The spanning tree for the network: source computer router the tree is drawn with thick lines

Discrete Maths: Trees/ Finding a Spanning Tree There are two main types of algorithms: – –breadth-first search – –depth-first search

Discrete Maths: Trees/ Breadth-first Search Process all the verticies at a given level before moving to the next level. Example graph G (again): a b dc ef h g

Discrete Maths: Trees/6 41 Informal Algorithm 1) Put the verticies into an ordering – –e.g. {a, b, c, d, e, f, g, h} 2) Select a vertex, add it to the spanning tree T: e.g. a 3) Add to T all edges (a,X) and X verticies that do not create a cycle in T – –i.e. (a,b), (a,c), (a,g) T = {a, b, c, g} continued a bc g

Discrete Maths: Trees/6 42 Repeat step 3 on the verticies just added, these are on level 1 – –i.e. b: add (b,d) c: add (c,e) g: nothing T = {a,b,c,d,e} Repeat step 3 on the verticies just added, these are on level 2 – –i.e. d: add (d,f) e: nothing T = {a,b,c,d,e,f} a bc g de a bc g de f continued level 1 level 2

Discrete Maths: Trees/6 43 Repeat step 3 on the verticies just added, these are on level 3 – –i.e. f: add (f,h) T = {a,b,c,d,e,f,h} Repeat step 3 on the verticies just added, these are on level 4 – –i.e. h: nothing, so stop a bc g de f h continued level 3

Discrete Maths: Trees/6 44 Resulting spanning tree: a b dc ef h g a different spanning tree from the earlier solution

Discrete Maths: Trees/6 45 Breadth-first Algorithm Tree bfs(Verts vsG, Edges edsG) // input ordered verts and edges of graph G { Vert v1 = firstVert(VsG); // first vert in G Seq levelVs = ( v1 ); // verts at curr level Verts vsT = { v1 }; // verts in tree Edges edsT = {}; // edges in tree while (1) { for each x in levelVs for each y in vsG - vsT if (x,y) is an edge in edsG add (x,y) to edsT; add y to vsT; if no edges added to vsT return tree made from vsT and edsT; levelVs = childrenOf( levelVs ); } } continued

Discrete Maths: Trees/6 46 Typical call: Verts vsG = {a, b, c, d, e, f, g, h}; Edges edsG = { (a,b), (a,c), (a,g), (b,d), (b,g), (c,d), (c,e), (d,f), (e,g), (f,h) }; Tree spanTree = bfs( vsG, edsG ); :

Discrete Maths: Trees/ Depth-first Search Process all the verticies down one path, then backtrack (go back) to verticies along other paths. Example graph G (again): a b dc ef h g

Discrete Maths: Trees/6 48 Informal Algorithm 1) Put the verticies into an ordering – –e.g. {a, b, c, d, e, f, g, h} 2) Select a vertex, add it to the spanning tree T: e.g. a 3) Add the edge (a,X) where X is the smallest vertex in the ordering, and does not make a cycle in T – –i.e. (a,b), T = {a, b} continued a b

Discrete Maths: Trees/6 49 4) Repeat step 3 with the new vertex, until there is no possible new vertex – –i.e. add the edges (b,d) (d,c) (c,e) (e,f) (f,h) T = {a,b,d,c,e,f,h} 5) At this point, there is no (h,X), so backtrack to a vertex that does have another edge: – –parent of h == f but there is no new (f,X) to add, so backtrack continued a b d c e f h

Discrete Maths: Trees/6 50 – –parent of f == e – –there is an (e,g) to add, so repeat step 3 with e 6) After g is added, there are no further verticies to add, so stop. a b d c e f h g continued

Discrete Maths: Trees/6 51 Resulting spanning tree: a b dc ef h g a different spanning tree from the breadth-first solution

Discrete Maths: Trees/ Another Backtracking Problem Find a subset of {31,27,15,11,7,5} with a sum that equals 39. – –one answer is {27,7,5} How?: build up the sum by adding integers. An integer is included if the sum remains less than 39. – –backtracking is required to try different integers when the sum doesn't reach 39. Summing a Subset

Discrete Maths: Trees/6 53 Backtracking Diagram {} sum = 0 {31} sum = 31 {31,7} sum = 38 {31,5} sum = 36 {27} sum = 27 {27,11} sum = 38 {27,7} sum = 34 {27,7,5} sum = 39

Discrete Maths: Trees/6 54 Prolog Code and Query sum(0, _, []). sum(Sum, List, [X|Used]) :- Sum > 0, select(List, X, List1), Sum1 is Sum - X, sum(Sum1, List1, Used). select([X|L], X, L). select([Y|L], X, [Y|L1]) :- select(L, X, L1). ?- sum(39, [31,27,15,11,7,5], Used). Used = [27,7,5] It can calculate sums of any set.

Discrete Maths: Trees/6 55 Amzi-Prolog Execution ; means "give another answer"

Discrete Maths: Trees/ Minimal Spanning Tree A minimal spanning tree T is a subgraph of a weighted graph G which contains all the verticies of G and whose edges have the minimum summed weight. Example weighted graph G: A B C D E F

Discrete Maths: Trees/6 57 A minimal spanning tree (weight = 12): A non-minimal spanning tree (weight = 20): A B C D E F A B C D E F

Discrete Maths: Trees/ Prim's Algorithm Prim's algorithm finds a minimal spanning tree T by iteratively adding edges to T. At each iteration, a minimum-weight edge is added that does not create a cycle in the current T.

Discrete Maths: Trees/6 59 Informal Algorithm For the graph G. 1) Add any vertex to T – –e.g A,T = {A} 2) Examine all the edges leaving {A} and add the vertex with the smallest weight. – –edgeweight (A,B) 4 (A,C) 2 (A,E) 3 – –add edge (A,C), T becomes {A,C} continued A B C D E F

Discrete Maths: Trees/6 60 3) Examine all the edges leaving {A,C} and add the vertex with the smallest weight. – –edgeweightedgeweight (A,B) 4(C,D) 1 (A,E) 3(C,E) 6 (C,F) 3 – –add edge (C,D), T becomes {A,C,D} continued

Discrete Maths: Trees/6 61 4) Examine all the edges leaving {A,C,D} and add the vertex with the smallest weight. – –edgeweightedgeweight (A,B) 4(D,B) 5 (A,E) 3(C,E) 6 (C,F) 3(D,F) 6 – –add edge (A,E) or (C,F), it does not matter – –add edge (A,E), T becomes {A,C,D,E} continued

Discrete Maths: Trees/6 62 5) Examine all the edges leaving {A,C,D,E} and add the vertex with the smallest weight. – –edgeweightedgeweight (A,B) 4(D,B) 5 (C,F) 3(D,F) 6 (E,F) 2 – –add edge (E,F), T becomes {A,C,D,E,F} continued

Discrete Maths: Trees/6 63 6) Examine all the edges leaving {A,C,D,E,F} and add the vertex with the smallest weight. – –edgeweightedgeweight (A,B) 4(D,B) 5 – –add edge (A,B), T becomes {A,B,C,D,E,F} All the verticies of G are now in T, so we stop. continued

Discrete Maths: Trees/6 64 Resulting minimum spanning tree (weight = 12): A B C D E F

Discrete Maths: Trees/6 65 Pseudocode Tree prim(Graph G, int numVerts) { Tree T = anyVert(G); for i = 1 to numVerts-1{ Edge e = an edge of minimum weight to a vertex in T and not forming a cycle in T when added to T; T = T with e added; } return T }

Discrete Maths: Trees/ A Greedy Algorithm Prim's algorithm is an example of a greedy algorithm – –its choice at each iteration does not depend on any previous choices A greedy algorithm does the best thing at this time without considering the past (or the future).

Discrete Maths: Trees/6 67 Greed is not Always Best An example where greed is not the best approach: – –a greedy shortest-path algorithm can lead to a non-optimal solution (not the best) – –the algorithm selects an edge with the minimum weight connected to the last vertex added continued

Discrete Maths: Trees/6 68 Find the shortest path a --> z. The greedy algorithm produces (a,c,z), but the shortest is (a,b,z) a b c z

Discrete Maths: Trees/ Kruskal's Minimum Spanning Tree Algorithm At the start, the minimum spanning tree T consists of all the verticies of the weighted graph G, but no edges. At each iteration, add an edge e to T having minimum weight that does not create a cycle in T. When T has n-1 edges, stop.

Discrete Maths: Trees/6 70 Pseudocode Tree kruskal(Graph G, int numVerts) { Tree T = allVerts(G); for i = 1 to numVerts-1 { Edge e = an edge of minimum weight in G and not forming a cycle in T when added to T; T = T with e added; } return T }

Discrete Maths: Trees/6 71 Example Graph G: a bc d e f g h i j k l iteredge 1(c,d) 2(k,l) 3(b,f) 4(c,g) 5(a,b) 6(f, j) 7(b,c) 8(j,k) 9(g,h) 10(i, j) 11(a,e) continued

Discrete Maths: Trees/6 72 Minimum spanning tree (weight = 24): a bc d e f g h i j k l

Discrete Maths: Trees/ Difference between Prim and Kruskal Prim's algorithm chooses an edge that must already be connected to a vertex in the minimum spanning tree T. Kruskal's algorithm can choose an edge that may not already be connected to a vertex in T.

Discrete Maths: Trees/ Game Trees Trees can be used to develop game-playing strategies – –used in many AI (Artificial Intelligence) game programs where the human is competing against the 'clever' machine

Discrete Maths: Trees/ Nim In Nim there are two piles of stones. Each player takes 1 or more stones from one pile. The player who removes the last stone loses. For example, the pile (3,2):

Discrete Maths: Trees/6 76 Part of a Nim Game Tree (3,2) (2,2)(1,2)(0,2)(3,1)(3,0) (1,2)(0,2)(2,1)(2,0)(0,2)(1,1)(1,0) (1,1)(0,1)(2,0) (1,0)(0,1) (0,0) (1,0)(0,0) (0,1)(1,0) (0,0) (2,0)(1,0)(0,0) (1,0)(0,0).. 1st player goes now 2nd player goes now

Discrete Maths: Trees/6 77 Analysing the Game Tree Label each terminal vertex with the first player's score: – –if the first player won, write "1" – –if the first player lost, write "0" continued

Discrete Maths: Trees/6 78 – –means that it is the 1st player’s turn – –means 1st player wins since there are no stones left – –so assign it – –means that it is the 2nd player’s turn – –means 2nd player wins since there are no stones left. So the 1st player loses. – –so assign it Graphically ( 0, 0 ) 1 0

Discrete Maths: Trees/6 79 Case 1. Label a box: The box is the first player, who will move to the most valuable child (to win). So label the box with the maximum child value (e.g. 1). 010 continued max Moving up the Tree

Discrete Maths: Trees/6 80 Case 2. Label an oval The oval is the second player, who will move to the least valuable child (so the first player loses). So label the box with the minimum child value (e.g. 0). 110 min

Discrete Maths: Trees/6 81 The Minimax Algorithm In many games: – –the first player chooses the most valuable position to move to; and – –the second player chooses the least valuable position for the first player The minimax algorithm: – –first player seeks the max of the child values – –second player seeks the min of the child values

Discrete Maths: Trees/ Nim Game Tree Analysis (part) st player 2nd player continued max min max min max

Discrete Maths: Trees/6 83 The root value (the starting position) contains a "1" – –this means that the first player will definitely win if he/she chooses the most valuable child position to move to

Discrete Maths: Trees/6 84 Larger Game Trees Games like Chess or Go generate game trees that are too large to analyse using a simple minimax algorithm. Instead the game tree is generated to a certain level, and then an evaluation function is used to 'calculate' the scores at that level for the first player. Minimax is then used to work back to the root.

Discrete Maths: Trees/6 85 Tic-Tac-Toe.. X X X XXXX X O O O O O Game tree generated to level 2 (with symmetric positions ignored).

Discrete Maths: Trees/6 86 The Evaluation Function For a given position P: – –eval(P) = numWinsX - numWinsO; – –numWinsX = number of rows, columns, and diagonals containing an X that the X player might be able to complete in the future – –numWinsO = same, but for O continued

Discrete Maths: Trees/6 87 Example position: eval(P) = numWinsX - numWinsO = = 1 If eval(P) is positive, then X has more ways to win than O. XO

Discrete Maths: Trees/6 88 Analyse the Tic-Tac-Toe Tree This means that the best move for the first player (X) is the center square. max min

Discrete Maths: Trees/6 89 Alpha-Beta Pruning Evaluating a game tree with a large level takes too much time. Alpha-beta pruning reduces the number of positions (verticies) that need to be evaluated, and still gives the right answer. There are two parts to alpha-beta pruning: – –alpha cutoff – –beta cutoff

Discrete Maths: Trees/6 90 Alpha Cutoff An example: ??? x 2 y? ? A BCD MLKJIHGFE alpha cutoff continued Evaluate children left-to-right. max min max

Discrete Maths: Trees/6 91 In A, x must be >= 2 – –the lower bound for A, the alpha value for A In C, y must be <= 1 – –since y <= x, it cannot affect x, so there is no need to evaluate J – –also the C subtree can be ignored (cutoff) Alpha cutoff occurs when a grandchild of A (e.g. I) has a value <= the alpha value of A.

Discrete Maths: Trees/6 92 Beta Cutoff An example: ????? y? 6 x max min A1 B1C1D1 E1F1G1H1I1J1K1L1M1 beta cutoff continued max min

Discrete Maths: Trees/6 93 In A1, x must be <= 6 – –the upper bound for A1, the beta value for A1 In C1, y must be >= 8 – –since y >= x, it cannot affect x, so there is no need to evaluate I1 or J1 – –also the C1 subtree can be ignored (cutoff) Beta cutoff occurs when a grandchild of A1 (e.g. H1) has a value >= the beta value of A1.

Discrete Maths: Trees/ More Information DM: chapter 7 Rosen: chapter 8