1. Constraint Optimization Presentation by Nathan Stender Chapter 13 of Constraint Processing by Rina Dechter 3/25/20131Constraint Optimization

2. Motivation 3/25/20132Constraint Optimization Real-life problems often have both hard and soft constraints – Hard constraints must be satisfied – Soft constraints would like to be satisfied Goal is to – optimize soft constraints – while satisfying hard constraints Applications – Planning – Scheduling – Design

3. Example: Combinatorial Auction Bidders place a bid on a set of items Hard constraints: no two bids sharing an item can be selected Soft constraints: cost of selecting each bid Auctioneer must select bids to maximize revenue 3/25/2013Constraint Optimization3

4. Outline 3/25/2013Constraint Optimization4 Motivation Constraint Optimization and Cost Networks Section 13.1 Branch-and-Bound Search Section 13.2 Bucket Elimination for Optimization Section 13.3 Mini-bucket Elimination Section 13.4 Search with Mini-bucket Heuristics Section 13.5 Summary

5. Constraint Optimization Problem COP = Constraint network + a cost function Global cost function – Real-valued functional components: F 1,…,F l – F 1,…,F l defined over scopes: Q 1,…,Q l, Q j ∈X – Assignment: ā = (a 1,…, a n ), a i in the domain of x i 3/25/2013Constraint Optimization5

6. Example: Combinatorial Auction Boolean variables b i for bids – Each bid has an associated cost r i – b i = 1 if bid is selected, b i =v0 if bid is rejected Hard constraint R ij between b i and b j – If they share an item: ( b i =1,b i =1)∉R ij Soft constraint F(b i ) – F(b i )=r i if b i =1, F(b i )=0 if b i =0 Goal is to maximize: 3/25/2013Constraint Optimization6 R ij bibi bjbj cost 000 10riri 01rjrj

7. Example: Combinatorial Auction Consider the following set of bids: 3/25/2013Constraint Optimization7 b1b1 b2b2 b3b3 b4b4 b5b5 Optimal solution: b 1 = 0, b 2 = 1, b 3 = 1, b 4 = 0, b 5 = 0 b 1 = {1,2,3,4} D b 1 = {0,1} r 1 = 8 b 2 = {2,3,6} D b 2 = {0,1} r 2 = 6 b 3 = {1,4,5} D b 3 = {0,1} r 3 = 5 b 4 = {2,8} D b 4 = {0,1} r 4 = 2 b 5 = {5,6} D b 5 = {0,1} r 5 = 2 R 12, R 13, R 14, R 24, R 25, R 35

8. 4-tuple: – (X,D,C h ) is a constraint network – C s = {F Q 1,…, F Q l } is a set of cost components – C h and C s also called hard and soft constraints – Graph representation – Variables are nodes – Arcs connect variables in the same hard or soft constraint 3/25/2013Constraint Optimization8 Cost Network

9. Solving COP as a Series of CSPs It is always possible to a solve COP as a series of CSPs Add a single hard constraint: Increase the cost-bound: Eventually the cost-bound so high no solution exists, the previous solution is optimal 3/25/2013Constraint Optimization9

10. Solving COP as a Series of CSPs With cost C j = 1, we have 4 solutions C 2 = 11, 1 solution C 1 = 12, 0 solutions Backtrack to previous solution, which is optimal The cost of solving so many CSPs is prohibitive Instead, we use search & inference 3/25/2013Constraint Optimization10

11. Outline 3/25/2013Constraint Optimization11 Motivation Constraint Optimization and Cost Networks Section 13.1 Branch-and-Bound Search Section 13.2 Bucket Elimination for Optimization Section 13.3 Mini-bucket Elimination Section 13.4 Search with Mini-bucket Heuristics Section 13.5 Summary

12. Branch-and-Bound Search Extends backtracking search – Traverses hard constraint search tree with DFS – Keeps track of two bounds using cost function Bounds – Lower bound : cost of best solution so far – Upper bound using bounding evaluation function – For partial assignment, : – If, backtrack 3/25/2013Constraint Optimization12

13. Example: Combinatorial Auction 3/25/2013Constraint Optimization13 1 0 0 1 0 0 1 0 1 0 0 0 23 10 17 12 10 8 15 13 11 9 0 10 23 17 12 10 8 15 13 11 9 10 r 1 = 8 r 2 = 6 r 3 = 5 r 4 = 2 r 5 = 2 R 12, R 13, R 14, R 24, R 25, R 35 b1b2b3b4b5b1b2b3b4b5

14. Russian Doll Search Run n successive BnB searches – Each search involves a single additional variable – First subproblem includes just nth variable – The (n-i+1)th subproblem involves x i,…,x n Bounds are augmented by previous searches Previous solutions can also heuristically guide search Extra pruning power is often worth extra cost of doing n searches 3/25/2013Constraint Optimization14

15. Example: Combinatorial Auction 3/25/2013Constraint Optimization15 1 0 0 0 1 0 1 0 0 0 0 8+11=19 8+7=15 8+4=12 8+2=10 11 7 10 Russian Doll: Previous searches: Over b 2,b 3,b 4,b 5 : b 2 =1,b 3 =1,b 4 =0,b 5 =0 11 Cost: Initial lower bound given by previous optimal solution Over b 3,b 4,b 5 : b 3 =1,b 4 =0,b 5 =1 7 Over b 4,b 5 : b 4 =1,b 5 =1 4 Over b 5 : b 5 =1 2 Optimal solutions from previous searches estimate upper bound r 1 = 8, r 2 = 6, r 3 = 5, r 4 = 2, r 5 = 2

16. Example: Combinatorial Auction 3/25/2013Constraint Optimization16 1 0 0 0 1 0 1 0 0 0 0 19 15 12 10 11 7 10 Russian Doll: 10 1 0 0 1 0 0 1 0 1 0 0 0 0 23 17 12 10 8 15 13 11 9 10 Original: b1b2b3b4b5b1b2b3b4b5

17. Improving Branch-and-Bound Substantial work on BnB has been done for integer programs by OR community Primary way to improve BnB is to improve accuracy of bounding function Constraint community extends same ideas to general constraints and cost functions Generate bounding functions using Bucket Elimination 3/25/2013Constraint Optimization17

18. Outline 3/25/2013Constraint Optimization18 Motivation Constraint Optimization and Cost Networks Section 13.1 Branch-and-Bound Search Section 13.2 Bucket Elimination for Optimization Section 13.3 Mini-bucket Elimination Section 13.4 Search with Mini-bucket Heuristics Section 13.5 Summary

19. Example: All soft constraints Consider the following cost network: 3/25/2013Constraint Optimization19 a b c f g d and ordering: a,c,b,f,d,g g d f b c a

20. Example: All soft constraints 3/25/2013Constraint Optimization20 a c b f d g

21. Derivation of Bucket Elimination 3/25/2013Constraint Optimization21 Cost function: Move each function into lowest (leftmost) bucket: Maximize functions in highest bucket for bucket variable and move to new lowest bucket: Repeat:

22. Bucket Elimination for Optimization Inference algorithm for COP Analogous to normal bucket-elimination – Each bucket holds a set of cost functions – Join operation replaced with summation – Projection replaced with maximization (or min) Forward direction generates an augmented cost network representation Backward direction generates optimal solution in linear time 3/25/2013Constraint Optimization22

23. Joining Soft Constraints Joining is replaced by summation: 3/25/2013Constraint Optimization23 x1x1 x2x2 Cost 004 015 103 111 x2x2 x3x3 003 012 101 110 x1x1 x2x2 x3x3 0007 0016 0106 011 … 5

24. Projecting Soft Constraints Projection is replaced by maximization: 3/25/2013Constraint Optimization24 x1x1 x2x2 x3x3 Cost 0007 0016 0106 0115 … x1x1 x3x3 007 01 … 6

25. Bucket Elimination for Opt: Properties Can incorporate hard constraints by adding back join and project operations It is always possible to treat hard constraints as cost functions, but not optimal – Hard constraints have stronger properties that can only be exploited if expressed explicitly – Hard constraints permit use of local consistency enforcing Complexity is based on induced width – For r hard and soft constraints and ordering d with induced width w*(d) – Time and space: O(r exp w*(d)+1 ) 3/25/2013Constraint Optimization25

26. Example: Combinatorial Auction 3/25/2013 Constraint Optimization 26 b1b1 b2b2 b3b3 b4b4 b5b5 Opt

27. R 41 b1b1 b4b4 00 01 10 r(b4)r(b4) b4b4 C 00 12 R 42 b2b2 b4b4 00 01 10 R 124 b1b1 b2b2 b4b4 000 001 010 100 110 h4(b1,b2)h4(b1,b2) b1b1 b2b2 C 002 010 100 110 h4(b1,b2,b4)h4(b1,b2,b4) b1b1 b2b2 b4b4 C 0000 0012 0100 1000 1100 Example: Combinatorial Auction 3/25/2013Constraint Optimization27 R 21, R 42, r(b 4 )h 4 (b 1,b 2 ) Join hard constraints Sum soft constraints (cost) Maximize over bucket variable Opt b1b1 b2b2 b3b3 b4b4 b5b5

28. h3(b1,b3,b5)h3(b1,b3,b5) b1b1 b3b3 b5b5 C 0000 0010 0105 1000 1010 h3(b1,b5)h3(b1,b5) b1b1 b5b5 C 005 010 100 110 R 135 b1b1 b3b3 b5b5 000 001 010 100 101 r(b3)r(b3) b3b3 C 00 15 R 31 b1b1 b3b3 00 01 10 R 35 b3b3 b5b5 00 01 10 Example: Combinatorial Auction 3/25/2013Constraint Optimization28 R 31, R 35, r(b 3 )h 3 (b 1,b 5 ) Join hard constraints Sum soft constraints (cost) Maximize over bucket variable Opt b1b1 b2b2 b3b3 b4b4 b5b5

29. R 125 b1b1 b2b2 b5b5 000 001 010 100 101 h 2 (b 1, b 2,b 5 ) b1b1 b2b2 b5b5 C 0002 0012 0106 1000 1010 h4(b1,b2)h4(b1,b2) b1b1 b2b2 C 002 010 100 110 h2(b1,b5)h2(b1,b5) b1b1 b5b5 C 006 012 100 110 r(b2)r(b2) b2b2 C 00 16 R 12 b1b1 b2b2 00 01 10 R 25 b2b2 b5b5 00 01 10 Example: Combinatorial Auction 3/25/2013Constraint Optimization29 R 12, R 25, r(b 2 ), h 4 (b 1,b 2 )h 2 (b 1,b 5 ) Join hard constraints Sum soft constraints (cost) Maximize over bucket variable Opt b1b1 b2b2 b3b3 b4b4 b5b5

30. h5(b1)h5(b1) b1b1 C 011 12 h5(b1,b5)h5(b1,b5) b1b1 b5b5 C 00 014 100 112 r(b5)r(b5) b5b5 C 00 12 h2(b1,b5)h2(b1,b5) b1b1 b5b5 C 006 012 100 110 h3(b1,b5)h3(b1,b5) b1b1 b5b5 C 005 010 100 110 Example: Combinatorial Auction 3/25/2013Constraint Optimization30 r(b 5 ), h 3 (b 1,b 5 ), h 2 (b 1,b 5 )h 5 (b 1 ) Sum soft constraints (cost) Maximize over bucket variable Opt b1b1 b2b2 b3b3 b4b4 b5b5

31. 11 h1(b1)h1(b1) b1b1 C 0 110 r(b1)r(b1) b5b5 C 00 18 h5(b1)h5(b1) b1b1 C 011 12 Example: Combinatorial Auction 3/25/2013Constraint Optimization31 r(b 1 ), h 5 (b 1 )Opt Sum soft constraints (cost) Maximize over bucket variable Opt b1b1 b2b2 b3b3 b4b4 b5b5

32. h1(b1)h1(b1) b1b1 C 011 110 h5(b1,b5)h5(b1,b5) b1b1 b5b5 C 0011 014 100 112 h 2 (b 1, b 2,b 5 ) b1b1 b2b2 b5b5 C 0002 0012 0106 1000 1010 h3(b1,b3,b5)h3(b1,b3,b5) b1b1 b3b3 b5b5 C 0000 0010 0105 1000 1010 h4(b1,b2,b4)h4(b1,b2,b4) b1b1 b2b2 b4b4 C 0000 0012 0100 1000 1100 Example: Combinatorial Auction 3/25/2013Constraint Optimization32 Go in the reverse direction to generate a solution: b 1 =0b 5 =0b 2 =1b 3 =1b 4 =0 Cost of optimal solution = 0 + 0 + 6 + 5 + 0 = 11

33. Outline 3/25/2013Constraint Optimization33 Motivation Constraint Optimization and Cost Networks Section 13.1 Branch-and-Bound Search Section 13.2 Bucket Elimination for Optimization Section 13.3 Mini-bucket Elimination Section 13.4 Search with Mini-bucket Heuristics Section 13.5 Summary

34. Example: All Soft Constraints 3/25/2013Constraint Optimization34 Bucket: Mini-bucket (i=3): Upper Bound (constraints are ignored, thus, the bound is optimistic)

35. Mini-bucket Elimination BE is exponential in separator size (space) Instead use mini-bucket elimination – Bucket are partitioned to restrict arity of projected constraints to at most i (variables) – Result generates an estimation of optimal solution – Size of mini-buckets limits complexity and accuracy of estimation – Selection of partition will also affect results, so a good heuristic is needed 3/25/2013Constraint Optimization35

36. Example: Combinatorial Auction 3/25/2013Constraint Optimization36 b1b1 b2b2 b3b3 b4b4 b5b5 Upper Bound

37. h4(b2,b4)h4(b2,b4) b2b2 b4b4 C 000 012 100 h4(b2)h4(b2) b2b2 C 02 10 r(b4)r(b4) b4b4 C 00 12 R 42 b2b2 b4b4 00 01 10 Example: Combinatorial Auction 3/25/2013Constraint Optimization37 R 42, r(b 4 )h4(b2)h4(b2) Sum soft constraints (cost) Maximize over bucket variable Bound b1b1 b2b2 b3b3 b4b4 b5b5

38. h3(b3,b5)h3(b3,b5) b3b3 b5b5 C 000 010 105 h3(b5)h3(b5) b5b5 C 05 10 r(b3)r(b3) b3b3 C 00 15 R 35 b3b3 b5b5 00 01 10 Example: Combinatorial Auction 3/25/2013Constraint Optimization38 R 35, r(b 3 )h3(b5)h3(b5) Sum soft constraints (cost) Maximize over bucket variable Bound b1b1 b2b2 b3b3 b4b4 b5b5

39. h2(b2,b5)h2(b2,b5) b2b2 b5b5 C 002 012 106 h2(b5)h2(b5) b5b5 C 06 12 r(b2)r(b2) b2b2 C 00 16 R 25 b2b2 b5b5 00 01 10 h4(b2)h4(b2) b2b2 C 02 10 Example: Combinatorial Auction 3/25/2013Constraint Optimization39 R 25, r(b 2 ), h 4 (b 2 )h2(b5)h2(b5) Sum soft constraints (cost) Maximize over bucket variable Bound b1b1 b2b2 b3b3 b4b4 b5b5

40. h5(b5)h5(b5) b5b5 C 011 12 h5h5 r(b5)r(b5) b5b5 C 00 12 h3(b5)h3(b5) b5b5 C 05 10 h2(b5)h2(b5) b5b5 C 06 10 Example: Combinatorial Auction 3/25/2013Constraint Optimization40 r(b 2 ), h 3 (b 5 ), h 2 (b 5 )h5h5 Sum soft constraints (cost) Maximize over bucket variable Bound b1b1 b2b2 b3b3 b4b4 b5b5

41. 19 h1(b1)h1(b1) b1b1 C 011 119 r(b1)r(b1) b1b1 C 00 18 h5h5 11 Example: Combinatorial Auction 3/25/2013Constraint Optimization41 r(b 1 ), h 5 Bound Sum soft constraints (cost) Maximize over bucket variable Bound b1b1 b2b2 b3b3 b4b4 b5b5

42. h5(b5)h5(b5) b5b5 C 011 12 h2(b2,b5)h2(b2,b5) b2b2 b5b5 C 002 012 106 h3(b3,b5)h3(b3,b5) b3b3 b5b5 C 000 010 105 h4(b2,b4)h4(b2,b4) b2b2 b4b4 C 000 012 100 h1(b1)h1(b1) b1b1 C 0 119 Example: Combinatorial Auction 3/25/2013Constraint Optimization42 Go in the forward direction to generate a solution (lower bound): b 1 =1b 5 =0b 2 =0b 3 =0b 4 =0 b1b1 b2b2 b3b3 b4b4 b5b5 Cost of solution/lower bound: 8 Interval bound on optimal solution: [8,19]

43. Mini-bucket Elimination: Properties Partitioning buckets amounts to moving maximization inside summation: Going up the elimination ordering (backward direction) – Maximization over mini-buckets gives an upper bound Going down the instantiation ordering (forward direction) – Finding a solution provides a lower bound Complexity depends on i, assuming i < w* – If : Time and space complexity = O(exp(i)) – Otherwise: Time = O(r exp(i)), Space = O(r exp(i-1)), where r is the number of cost functions 3/25/2013Constraint Optimization43

44. Outline 3/25/2013Constraint Optimization44 Motivation Constraint Optimization and Cost Networks Section 13.1 Branch-and-Bound Search Section 13.2 Bucket Elimination for Optimization Section 13.3 Mini-bucket Elimination Section 13.4 Search with Mini-bucket Heuristics Section 13.5 Summary

45. Search with Mini-Bucket Heuristics Mini-bucket allows us to bound error, however we may need the optimal solution We can improve search with estimations from mini-buckets – Search using BnB, use mini-bucket to improve accuracy of bounding function – At each step, estimate the cost of optimal solution given a partial assignment using mini-bucket – Assuming a static variable ordering, buckets can be computed before search 3/25/2013Constraint Optimization45

46. 8 f mb (b 1 =0,b 5 =0,b 2 =1,b 3 =0) = 6+0+h 4 (b 2 ) = 6+0+0 = 6 f mb (b 1 =0,b 5 =0,b 2 =0) = 0+h 4 (b 2 ) +h 3 (b 5 ) = 0+2+5 = 7 f mb (b 1 =1,b 5 =0) = 8+0+h 2 (b 5 ) +h 3 (b 5 ) = 8+0+6+5 = 19 f mb (b 1 =0) = 0+h 5 = 0+11 = 11 f mb (b 1 =0,b 5 =0) = 0+h 2 (b 5 ) +h 3 (b 5 ) = 0+6+5 = 11 f mb (b 1 =0,b 5 =1) = r 5 +h 2 (b 5 ) +h 3 (b 5 ) = 2+2+0 = 4 f mb (b 1 =0,b 5 =0,b 2 =1) = r 2 +h 4 (b 2 ) +h 3 (b 5 ) = 6+0+5 = 11 f mb (b 1 =0,b 5 =0,b 2 =1,b 3 =1) = 6+r 3 +h 4 (b 2 ) = 6+5+0 = 11 f mb (b 1 =1) = r 1 +h 5 = 8+11 = 19 f mb (b 1 =1,b 5 =1) = 8+r 5 +h 2 (b 5 ) +h 3 (b 5 ) = 8+2+0+0 = 10 Example: Combinatorial Auction 3/25/2013Constraint Optimization46 1 1 0 0 0 0 0 0 0 0 0 0 0 0 11 1 8 10 8 8 19 11 19 4 11 6 1 1 h4(b2)h4(b2) b2b2 C 02 10 h3(b5)h3(b5) b5b5 C 05 10 h2(b5)h2(b5) b5b5 C 06 10 7 b1b1 b2b2 b3b3 b4b4 b5b5 h5h5 Bound L= 10 11 Lower bound is initialized to lower bound found during mini-bucket

47. Mini-bucket Heuristics Properties 3/25/2013Constraint Optimization47 Definition 13.1: Given an ordered set of augmented buckets generated by the mini-bucket algorithm, the bounding function is the sum of all the new functions that are generated in buckets p+1 through n and reside in buckets 1 through p. That is: Theorem 13.5 (paraphrased): The mini-bucket heuristic never under-estimates the cost of the optimal extension of a given partial assignment The heuristic is monotonic, as search goes deeper the bound can only decrease, meaning it is more accurate.

48. Example: Combinatorial Auction 3/25/2013Constraint Optimization48 In this case the bounding functions perform similarly, but mini-bucket is more accurate! First-choice Mini-bucket 1 1 0 0 0 0 0 0 0 0 0 0 0 7 11 0 10 12 17 23 15 21 13 15 23 15 9 0 13 8 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 7 11 1 8 10 8 8 19 11 19 4 11 6 1 1 10

49. Outline 3/25/2013Constraint Optimization49 Motivation Constraint Optimization and Cost Networks Section 13.1 Branch-and-Bound Search Section 13.2 Bucket Elimination for Optimization Section 13.3 Mini-bucket Elimination Section 13.4 Search with Mini-bucket Heuristics Section 13.5 Summary

50. Constraint optimization problems consist of a constraint network and a global cost function. A COP can be solved as a CSP, but search and inference methods more efficient Branch-and-bound search is a simple method which extends backtrack search with a bounding function Bucket Elimination is an inference method which can generate COP solutions in a backtrack-free manner Mini-bucket Elimination partitions buckets into subsets in order to reduce complexity, while still generating a bound for a COP BnB can be augmented with a mini-bucket based heuristic allowing inference and search methods to be combined. 3/25/2013Constraint Optimization50

51. Thank You! Any Questions? 3/25/2013Constraint Optimization51