Pruning 24-Feb-19

Exponential growth How many leaves are there in a complete binary tree of depth N? depth = 0, count = 1 depth = 1, count = 2 depth = 3, count = 4 depth = 4, count = 8 depth = 5, count = 16 depth = N, count = 2N This is easy to demonstrate: Count “going left” as a 0 Count “going right” as a 1 Each leaf represents one of the 2N possible N-bit binary numbers This observation turns out to be very useful in certain kinds of problems

Pruning Suppose the binary tree represents a problem that we have to explore to find a solution (or goal node) If we can prune (decide we can ignore) a part of the tree, we save effort saves 15 saves 7 saves 3 The higher up in the tree we can prune, the more effort we can save The advantage is exponential

Sum of subsets Problem: Example: There are n positive integers, and a positive integer W Find a subset of the integers that sum to exactly W Example: The numbers are 2, 5, 7, 8, 13 Find a subset of numbers that sum to exactly 25 We can multiply each number by 1 if it is in the sum, 0 if it is not 2 5 7 8 13 0 0 0 0 0  0 0 0 0 0 1  13 0 0 0 1 0  8 0 0 0 1 1  21 0 0 1 0 1  20 0 0 1 1 0  15 0 0 1 1 1  28 0 1 0 0 0  5 0 1 0 0 1  18 0 1 0 1 0  13 0 1 0 1 1  26 0 1 1 0 0  12 0 1 1 0 1  25

Brute force We have a brute-force method for solving the sum of subsets problem For N numbers, count in binary from 0 to 2N For each 1, include the corresponding number; for each 0, exclude the corresponding number Stop if we get lucky This is clearly an exponential-time algorithm It seems like, with a little cleverness, we could do better It turns out that we can use pruning to do somewhat better But we are still left with an exponential-time algorithm

Binary tree representation Suppose our numbers are 3, 8, 9, 17, 26, 39, 43, 56 and our goal is 100 We can describe this as a binary tree search 3 (yes or no) 8 (yes or no) 9 (yes or no) 17 (yes or no) etc. As we search the binary tree, A node is promising if we might be able to get to a solution from it A node is nonpromising if we know we can’t get to a solution When we detect a nonpromising node, we can prune (ignore) the entire subtree rooted at that node How do we detect nonpromising nodes?

Detecting nonpromising nodes Suppose we work from left to right in the sequence 3 8 9 17 26 39 43 56 That is, we try things in the order 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 ... When we get to 0 0 0 0 0 0 1 0  43 we notice that even if we include all the remaining numbers (in this case, there is only one), we can’t get to 100 There is no need to try the 0 0 0 0 0 0 1 x numbers When we get to 1 1 1 1 1 1 0 0  101 we notice that we have overshot, hence no solution is possible with what we have so far We don’t need to try any of the 1 1 1 1 1 1 x x numbers

Still exponential Even with pruning, the sum of subsets typically requires exponential time However, in some cases, pruning can save significant amounts of time Consider trying to find a subset of {23, 29, 35, 41, 43, 46, 48, 51} that sums to 100 Here, pruning can save substantial effort Sometimes, common sense can be a big help Consider trying to find a subset of {16, 20, 28, 34, 44, 48} that sums to 75

Boolean satisfaction Suppose you have n boolean variables, a, b, c, ..., that occur in a logical expression such as (a or c or not f) and (not b or not d or a) and ... The problem is to assign true/false values to each of the boolean variables in such a way as to satisfy (make true) the logical expression The brute-force algorithm is the same as before (0 is false, 1 is true, try all n binary numbers) Again, you can do significant pruning, if you think about the problem Anything you do, you will still end up with a solution that is exponential in n

Intractable problems The technical term for a problem that takes exponential time is intractable Intractable problems can only be solved for small input sizes Faster computer speeds will not help much—exponential growth is fast Bottom line: Avoid these problems if at all possible!

The End