1. G ENERATE AND T EST F RAMEWORK Nattee Niparnan

2. O PTIMIZATION E XAMPLE : F INDING M AX V ALUE IN AN A RRAY 2523443490-5870561 There are N possible answers The first element The second element 3 rd, 4 th … Try all of them Remember the best one

3. G ENERATE AND T EST F RAMEWORK

4. C OMBINATION AND P ERMUTATION In many case, the set of the admissible solutions is a set of “combination” or “permutation” of something We need to knows how to generate all permutations and combinations

5. G ENERATING ALL POSSIBLE ANSWER

6. C OMBINATION Given N things Generate all possible selections of K things from N things Ex. N = 3, k = 2

7. C OMBINATION WITH REPLACEMENT Given N things Generate all possible selections of K things from N things When something is selected, we are permit to select that things again (we replace the selected thing in the pool) Ex. N = 3, k = 2

8. B REAKING THE P ADLOCK

9. Assuming we have four rings Assuming each ring has following mark    We try         ….        Backtracking Undone the second step, switch to another value

10. K EY I DEA A problem consists of several similar steps Choosing a things from the pool We need to remember the things we done so far

11. G ENERAL F RAMEWORK Storage Step that have been done Engine Initial Step 1. Get a step that is not complete 2. Try any possible next step 3. Store each newly generated next step

12. C OMBINATION Generate all combinations of lock key Represent key by int  =0  =1  =2 =3 Step = sequence of selected symbols E.g., ’01’    ‘0003’    

13. G ENERAL S EARCH Storage s S  ‘’ While s is not empty Curr  Storage.get If Curr is the last step evaluate Else Generate all next step from Curr Push them to S For all symbol i New = curr+I Storage.push(new) If length(curr) == 4

14. S EARCH S PACE Set of all admissible solutions E.g., Combination Padlock Search space = 0000  4444 For a task with distinct steps it is easy to enumerate the search space with “backtracking” for most of the case, we can do “backtracking”

15. B ACKTRACKING A tool to enumerate the search space Usually using the “stack” to implement the storage Employ the processor stack i.e., using the “recursive” paradigm

16. C OMBINATION E XAMPLE BY B ACKTRACKING The process automatically remember the step void backtracking(int step,int *sol) { if (step < num_step) { for (int i = 0; i < num_symbol; i++) { sol[step] = i; backtracking(step + 1,sol); } } else { check(sol); }

17. S EARCH T REE Sols in each step 0 00 000 0000  check 000 0001  check 000 0002  check 000 0003  check 000 00 001 0010  check

18. S EARCH T REE 0123 01020304 … … …

19. 8- QUEEN Given a chess board 8 queensXXXXXX XXXX XXX XXX

20. 8- QUEENS PROBLEM Try to place the queens so that they don’t get in the others’ ways Q Q Q Q Q Q Q Q

21. S OLVING THE P ROBLEM Define the search space What is the search space of this problem? How large it is? Choose an appropriate representation

22. E XAMPLE Every possible placement of queens Size: 64 8 Representation: a set of queens position E.g., (1,1) (1,2) (2,5) (4,1) (1,2) (3,4) (8,8) (7,6) This includes overlapping placement!!!

23. E XAMPLE Another representation Try to exclude overlapping Use combination without replacement This is a combination Selecting 8 positions out of 64 positions Size: (64)! / (64 – 8)! * 8! Implementation: in the “generating next step”, check for overlapping

24. C OMBINATION WITHOUT REPLACEMENT We go over all position For each position, we either “choose” or “skip” that position for the queen 1234 5678 9101112 13141516 1234 5678 9101112 13141516

25. C OMBINATION WITHOUT REPLACEMENT void e_queen(int step,int *mark_on_board) { if (step < 64) { mark_on_board[step] = 0; e_queen(step + 1,mark_on_board); mark_on_board[step] = 1; e_queen(step + 1,mark_on_board); } else { check(mark_on_board); } Also has to check whether we mark exactly 8 spots Mark_on_board is a binary string indicate that whether the position is selected

26. C OMBINATION WITHOUT REPLACEMENT The generated mark_on_board includes 000000000000  select no position (obviously not the answer) 111111000000  select 6 positions (obviously not the answer) We must limit our selection to be exactly 4

27. C OMBINATION WITHOUT REPLACEMENT void e_queen(int step,int *mark_on_board,int chosen) { if (step < 64) { if ((64 – 8) – (step – chosen) > 0) { mark_on_board[step] = 0; e_queen(step + 1,mark_on_board,chosen); } if (8 - chosen > 0) { mark_on_board[step] = 1; e_queen(step + 1,mark_on_board,chosen+1); } } else { check(mark_on_board); } Number of possible 0 Number of possible 1

28. E XAMPLE Any better way? For each row, there should be only one queen Size: 8 8 Representation: sequence of columns E.g., (1,2,3,4,5,6,7,8)

29. U SING B ACKTRACKING ? The problem consists of 8 step Placing each queen We never sure whether the queen we place would lead to a solution Backtracking is an appropriate way

30. S OLVING THE P ROBLEM : E NUMERATING S EARCH S PACE There are eight possible ways in each step There are eight steps Very similar to the combination problem void e_queen(int step,int *queen_pos) { if (step < 8) { for (int i = 0; i < 8; i++) { queen_pos[step] = i; e_queen(step + 1, queen_pos); } } else { check(queen_pos); }

31. 8- QUEEN BY PERMUTATION Queen should not be in the same column The solution should never have any column repeated E.g., (1,2,3,4,5,6,7,1) is bad (column collision (1,1,3,4,5,6,7,5) is bad as well…. (1,2,3,4,5,6,7,8) is good There should be no duplicate column index!!!

32. P ERMUTATION Given N symbols A permutation is the element arrange in any order E.g., 1 2 3 4 Shows 1 2 3 4 1 2 4 3 1 3 2 4 1 3 4 2 … 4 3 2 1 For each step, we have to known which one is used

33. P ERMUTATION BY B ACKTRACKING The problem consists of several similar steps Special condition Symbols never repeat How to do? Easy way: Generate all combination (as done before) Check for ones that symbols do not repeat Better way: Remember what symbols are used

34. P ERMUTATION

35. More proper ways void backtracking(int step,int *sol,bool *used) { if (step < num_step) { for (int i = 0; i < num_symbol; i++) { if (!used[i]) { used[i] = true; sol[step] = i; backtracking(step,sol,used); used[i] = false; } } else { check(sol); }

36. B ACKTRACKING P ROBLEMS Given N Find any sequence of (a 1,a 2,a 3,…) such that a 1 +a 2 + a 3 +… + a k = N a i > 0 a i <= a j for all i < j a i is an integer

37. E XAMPLE N = 4 1 + 1 + 1 + 1 1 + 1 + 2 1 + 3 2 + 2 4

38. S OLVING WITH B ACKTRACKING Representation Array of a i Step Choosing the next value for a i

39. B RANCH & B OUND Technique to reduce enumeration

40. M AIN I DEA We should not enumerate solution that will never produce a solution We have done that!!! 8-queens By naïve combination, we will have to do all 64 8 But, by each improvement, we further reduce what we have to do

41. P ERMUTATION 012 101112 202122 000102 000 001 002 010 011 012 020 021 022 100 101 102 110 111 112 120 121 122 200 201 202 210 211 212 220 221 222

42. P ERMUTATION 012 101112 202122 000102 000 001 002 010 011 012 020 021 022 100 101 102 110 111 112 120 121 122 200 201 202 210 211 212 220 221 222

43. K EY If we know, at any step, that the solution is not feasible Then, it is futile to further search along that path

44. O PTIMIZATION P ROBLEM All previous examples are decision problems Asking whether the solution satisfy the criteria Now, we consider broader set of problem, the optimization problem Example For all students, find one with maximum height

45. E VALUATION Representation: x = id of student “goodness evaluation” evaluate(x) For all x in the search space, we have to find one with maximum evaluate(x)

46. S OLVING O PTIMIZATION P ROBLEM Enumerate all possible solution Calculate its value Remember the max void backtracking(int step,int *sol) { if (step < num_step) { for (int i = 0; i < num_symbol; i++) { sol[step] = i; backtracking(step,sol); } } else { value = evaluate(sol); if (value > max) remember(value,sol); }

47. B RANCH & B OUND IN O PTIMIZATION P ROBLEM For many problems, it is possible to assert its goodness even the solution is not complete If we can predict the best value for the remaining step, then we can use that value to “bound” our search

48. E XAMPLE Assuming that we have 10 steps At step 7, the goodness of the partial solution is X Assuming that we know that the remaining step could not produce a solution better than Y If we have found a solution better than X+Y We can simply “bound” the search

49. K EYS We must know the so-called “upper bound” of the remaining step It should be compute easily

50. E XAMPLE 23352 Let value at this point be 10 If we know that this path never bet higher than 13 (which make 10 + 13 < 35) We can neglect it

51. K NAPSACK P ROBLEM

52. Given a sack, able to hold K kg Given a list of objects Each has a weight and a value Try to pack the object in the sack so that the total value is maximized

53. V ARIATION Rational Knapsack Object is like a gold bar, we can cut it in to piece with the same value/weight 0-1 Knapsack Object cannot be broken, we have to choose to take (1) or leave (0) the object E.g. K = 50 Objects = (60,10) (100,20) (120,30) Best solution = choose second and third

54. R ATIONAL K NAPSACK Can be solved by greedy Sort object according to value/weight ratio Pick objects by that ratio If object is larger than the remaining capacity, just divide it

55. 0-1 K NAPSACK WITH B&B 0-1 knapsack is very suitable for B&B We can calculate the goodness of the partial solution Just sum the value of the selected objects We have fast, good upper bounds (several one) The sum of remaining unselected objects The sum of remaining unselected object that don’t exceed the capacity The solution of the “rational knapsack” of the remaining objects with the remaining capacity

56. T ASK Implement 0-1 Knapsack