1. CS 312: Algorithm Analysis Lecture #35: Branch and Bound Design Options - State Spaces Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, and Sean Warnick This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.Creative Commons Attribution-Share Alike 3.0 Unported License

2. Announcements  Homework #25 due now  Homework #26 optional  Project #7: TSP  We continue discussing ideas today  Early: Wednesday  Due: next Friday

3. Objectives  Consider an alternate way to structure the state space.  i.e., an alternate way to generate successor states  Provide more options to complete the project.

4. Review the Ingredients of B&B

5. Review: Branch and Bound function BandB(v) BSSF  quick-solution(v) // BSSF.cost holds cost Agenda.clear() v.b  bound(v) Agenda.add(v, v.b) while !Agenda.empty() and time remains and BSSF.cost != v.b do u  Agenda.first() Agenda.remove_first() children = generate_children_ascending(u) for each w in children do if ! time remains then break w.b  bound(w) if (w.b < BSSF.cost) then if criterion(w) then BSSF  w Agenda.prune(BSSF.cost) else if partial_criterion(w) then Agenda.add(w, w.b) return BSSF Compute bound when constructing state rather than as separate step.

6. Review: Branch and Bound function BandB(v) BSSF  quick-solution(v) // BSSF.cost holds cost Agenda.clear() Agenda.add(v, v.b) while !Agenda.empty() and time remains and BSSF.cost != v.b do u  Agenda.first() Agenda.remove_first() children = generate_children_ascending(u) for each w in children do if ! time remains then break if (w.b < BSSF.cost) then if criterion(w) then BSSF  w Agenda.prune(BSSF.cost) else if partial_criterion(w) then Agenda.add(w, w.b) return BSSF Compute bound when constructing state rather than as separate step.

7. Review: Branch and Bound function BandB(v) BSSF  quick-solution(v) // BSSF.cost holds cost Agenda.clear() Agenda.add(v, v.b) while !Agenda.empty() and time remains and BSSF.cost != v.b do u  Agenda.first() Agenda.remove_first() children = generate_children_ascending(u) for each w in children do if ! time remains then break if (w.b < BSSF.cost) then if criterion(w) then BSSF  w Agenda.prune(BSSF.cost) else if partial_criterion(w) then Agenda.add(w, w.b) return BSSF Apply partial_criterion() when generating children.

8. Simplify: Branch and Bound function BandB(v) BSSF  quick-solution(v) // BSSF.cost holds cost Agenda.clear() Agenda.add(v, v.b) while !Agenda.empty() and time remains and BSSF.cost != v.b do u  Agenda.first() Agenda.remove_first() children = generate_children_ascending(u) for each w in children do if ! time remains then break if (w.bound < BSSF.cost) then if criterion(w) then BSSF  w Agenda.prune(BSSF.cost) else Agenda.add(w, w.b) return BSSF Apply partial_criterion() when generating children.

9. Simplify: Branch and Bound function BandB(v) BSSF  quick-solution(v) // BSSF.cost holds cost Agenda.clear() Agenda.add(v, v.b) while !Agenda.empty() and time remains and BSSF.cost != v.b do u  Agenda.first() Agenda.remove_first() children = generate_children_ascending(u) for each w in children do if ! time remains then break if (w.bound < BSSF.cost) then if criterion(w) then BSSF  w Agenda.prune(BSSF.cost) else Agenda.add(w, w.b) return BSSF Use init_state() function. Compute bound as part of initial state.

10. Simplify: Branch and Bound function BandB() s  init_state() BSSF  quick-solution(s) // BSSF.cost holds cost Agenda.clear() Agenda.add(s, s.bound) while !Agenda.empty() and time remains and BSSF.cost != s.bound do u  Agenda.first() Agenda.remove_first() children = generate_children_ascending(u) for each w in children do if ! time remains then break if (w.bound < BSSF.cost) then if criterion(w) then BSSF  w Agenda.prune(BSSF.cost) else Agenda.add(w, w.b) return BSSF Use init_state() function. Compute bound as part of initial state. Requirement for Proj. #7: Use B&B pseudo-code.

11. Review: A Bound on TSP Tour 1 2 34 5 8 6 7 4 4 3 2 19 10 12

12. 1.Reduce Rows 1 2 34 5 0 0 1 2 1 0 0 01 9 6

13. 2. Reduce Columns 1 2 34 5 0 0 0 2 1 0 0 01 9 6

14. Review: Pruning  Under what conditions can a search state be pruned?  Lower bound for state  BSSF (least upper bound)  Could be true when the state is created  Could happen when the BSSF is updated and the agenda is pruned  The state is not feasible  Where feasibility may include any test you might think of:  Already occupied  Cycle among sub-set of cities  No options remaining ……

15. Focus on State Space function BandB() s  init_state() BSSF  quick-solution(s) // BSSF.cost holds cost Agenda.clear() Agenda.add(s, s.bound) while !Agenda.empty() and time remains and BSSF.cost != s.bound do u  Agenda.first() Agenda.remove_first() children = generate_children_ascending(u) for each w in children do if ! time remains then break if (w.bound < BSSF.cost) then if criterion(w) then BSSF  w Agenda.prune(BSSF.cost) else Agenda.add(w, w.b) return BSSF

16. Structure of State Space  Recall the structure of the state space from last time?

17. Review: State Space #1  Generate all children one step away  each child state is generated by including one of the edges leaving city j, where j is the destination of the last edge in the parent state  Pro: Does not include premature cycles. Why not?  Pro: Simple algorithm: pick all neighbors that remain from vertex j, (leave the starting vertex for the end)  Con: On many instances, while searching with B&B, will result in excessively deep agendas. Why?  Can B&B in this state space be optimal? Include edge (j,1) (j,2)(j,3) … (j,N) (i,j)

18. Structure of State Space  Could we structure the space another way?  How else could we generate children states?

19. State Space #2 Include edge (i,j)Exclude edge (i,j) Use the same method for representing states computing bounds. New method for generating children: Why can branch and bound in this state space be optimal?

20. Excluding an Edge 1 2 34 5 0 0 0 2 1 0 0 01 9 6 What happens to a state’s cost matrix when an edge is excluded? Exclude 4  2

21. State Space: Strategy #2

23. Which edge to select?  What strategies for edge selection can you think of?  Idea: try to reduce the size of the part of the state-space you actually need to explore!

24. 0 Try #1: Consider edge (3,2) include (3,2)exclude (3,2) 1 2 34 5 0 0 1 0 0 9 6 1 2 34 5 0 0 0 1 1 0 0 01 9 6 bound = 22bound = 21 Parent: bound = 21 Both children similarly competetive.

25. Try #2: Consider edge (2,3) include (2,3)exclude (2,3) bound = 21bound = 28 bound = 21 1 2 34 5 0 0 1 1 0 01 9 6 1 2 34 5 0 0 0 1 1 0 0 01 9 6 Parent: Left child more likely to be pruned.

26. Try #3: Consider edge (5,1) include (5,1)exclude (5,1) bound = 30bound = 21 1 2 34 5 0 0 0 1 1 0 0 01 9 6 1 2 34 5 0 0 0 1 1 0 0 01 6 Parent: Right child more likely to be pruned.

27. State Space so far include (5,1)exclude (5,1) bound = 30bound = 21

28. Consider edge (2,5) include (2,5)exclude (2,5) bound = 28bound = 21 1 2 34 5 0 0 0 1 0 0 1 1 2 34 5 0 0 0 1 1 0 0 1 6 Parent: Again, right child more likely to be pruned.

29. Search Space so far include (5,1)exclude (5,1) 3021 include (2,5)exclude (2,5) 2821

30. Consider edge (3,2) include (3,2)exclude (3,2) bound = 22bound = 21 1 2 34 5 0 0 0 1 2 34 5 0 0 0 0 0 1 Parent:

31. Search Space so far include (5,1)exclude (5,1) 3021 include (2,5)exclude (2,5) 2821 include (3,2) 21 exclude (3,2) 22

32. Search Space so far include (5,1)exclude (5,1) 3021 include (2,5)exclude (2,5) 2821 include (3,2) 21 exclude (3,2) 22 1 2 34 5 0 0... 21 include (4,3),(1,4)

33. Summarize: State Space #2  Pick an edge e = (i,j) and generate two children: 1.Include edge e 2.Exclude edge e  One might pick an edge by:  maximizing the difference between the included edge child and the excluded edge child  maximizing the bound on the excluded edge child  minimizing the bound on the included edge child  One must be careful not to create a cycle prematurely  How to prevent? Include edge (i,j) Exclude edge (i,j)

34. Sizes of State Spaces  Start with Complete Graph

35. Sizes of State Spaces  Strategy #1  Strategy #2  See ch. 6: DP strategy  n 2 2 n

36. Conclusions  You have options!  Having some control without sacrificing optimality is important.  You may be able to prune more aggressively without sacrificing optimality.

37. Assignment  HW #26: Optional  Include-exclude state-space generation  Kill two birds with one stone:  Read the project instructions and lecture notes  Do HW #26 on whiteboard with a partner  Have your “whiteboard experience” for Project #7  Consider your options  Design a B&B algorithm  Work and talk through an example  Read the assigned selection from Russell & Norvig  Use your homework key credentials  Happy General Conference weekend!