1. Performance of Distributed Constraint Optimization Algorithms A.Gershman, T. Grinshpon, A. Meisels and R. Zivan Dept. of Computer Science Ben-Gurion University

2. DCR workshop - May 20082 Optimization Problems Optimization Problems Problems are too tight No solution that satisfies all constraints exists Search for the solution with a minimal cost

3. DCR workshop - May 20083 Constraint Optimization Problems Constraint Optimization Problems Weighted Binary CSPs: Every pair of assignments [, ], is assigned a cost c The cost of a tupple is the sum of all costs of pairs included in it Specific case – Max-CSP: all costs c are either 0 or 1 [Larrosa & Meseguer 96], [Larrosa & Schiex 2004]

4. DCR workshop - May 20084 Distributed Constraint Optimization Problems (DisCOPs) Distributed Constraint Optimization Problems (DisCOPs) There are several approaches for solving DisCOPs: Branch and Bound SynchBB, AFB Using a Pseudo-Tree ADOPT, DPOP Merging partial solutions OptAPO Very different algorithms  different behavior Comparative evaluation of runtime performance is needed

5. DCR workshop - May 20085 AFB AFB Add text

6. DCR workshop - May 20086 ADOPT ADOPT Add text

7. DCR workshop - May 20087 DPOP DPOP Add text

8. DCR workshop - May 20088 OptAPO OptAPO Add text

9. DCR workshop - May 20089 A bit of history… DisCSPs A bit of history… DisCSPs The runtime performance of ABT [Yokoo et al. 1995] was first compared to AWC [Yokoo et al. 1998]: Performance was measured in cycles of a synchronous simulator

10. DCR workshop - May 200810 Runtime Performance of DisCSPs (II) In 2001 we have played with a synchronous algorithm that uses ordering heuristics and have found that it is faster than ABT – but how to measure ? In 2002 a non-concurrent runtime measure for DisCSP search – non-concurrent constraints checks (NCCCs) Synchronous CBJ (with variable ordering) was shown to be faster than ABT (NCCCs) [Brito & Meseguer 2004] All on randomly generated DisCSPs

11. DCR workshop - May 200811 Runtime Performance of Centralized COPs Certain search algorithms produce a phase transition for increasingly tighter (and harder) random problems Performance of standard Branch and Bound grows exponentially for ever harder problems. MaxCSPs are harder than Weighted CSPs

12. DCR workshop - May 200812 Evaluation of Runtime Performance – ADOPT* *[Modi et al. 2005]

13. DCR workshop - May 200813 Evaluation of Runtime Performance – OptAPO* Evaluation of Runtime Performance – OptAPO* *[Mailler & Lesser 2004]

14. DCR workshop - May 200814 What is a cycle? What is a cycle? cycles All quoted results are in cycles What is a cycle for each of the algorithms: Adopt, AFB – reading all messages and checking all local assignments against the current context or CPA DPOP – Calculating all costs of the sub tree for every combination of assignments of higher priority constrained agents OptAPO, Solving centrally a problem of the size of the mediation session

15. DCR workshop - May 200815 How to count NCCCs for DisCOPs ? How to count NCCCs for DisCOPs ? ADOPT, SBB and AFB perform CCs in each computation session and can be counted non- concurrently as for DisCSPs DPOP – for every row in the table sent by a DPOP agent, the number of CCs is the product of number of potential assignments times the number of constrained (up-tree) agents OptAPO - each mediation session is assigned the number of CCs needed to find the local solution

16. DCR workshop - May 200816 Choosing the right benchmark for DisCOPs Graph coloring problems do not cover important ranges of problem difficulty. Specific problems have special structures (MSP – equality binary constraints, Sensor Nets – very small density…) Evaluation – use random DisMaxCSPs and increase problem’s difficulty (tightness) One way to exhibit a “phase transition”

17. DCR workshop - May 200817 Experimental Set-up Experimental Set-up Randomly generated Max-CSPs Size: 10 variables 10 values Density: p 1 = 0.4, 0.7 Tightness: p 2 = 0.4 – 0.99 Rune time Measure: Non-Concurrent Constraint Checkes (NCCCs)

18. DCR workshop - May 200818 Logarithmic scale (p1 = 0.4)

19. DCR workshop - May 200819 Linear scale (p1 = 0.4)

20. DCR workshop - May 200820 Low to intermediate difficulty Low to intermediate difficulty ADOPT performs well AFB, OptAPO, and SynchBB are fairly close DPOP performs extremely poor Probably due to the lack of pruning of the search space

21. DCR workshop - May 200821 As difficulty grows… The runtime of ADOPT, OptAPO, and SynchBB grows at a high exponential rate ADOPT did not terminate its run on the tightest problems (p 2 ≥ 0.9) DPOP and AFB perform far better, by two orders of magnitude

22. DCR workshop - May 200822 Linear scale – A Closer Look

23. DCR workshop - May 200823 High Constraints Density (p 1 = 0.7) High Constraints Density (p 1 = 0.7)

24. DCR workshop - May 200824 Linear scale (p 1 = 0.7)

25. DCR workshop - May 200825 High Density and Low Tightness The performance of ADOPT, AFB, OptAPO, and SynchBB is fairly similar The performance of DPOP is much worse

26. DCR workshop - May 200826 High Density and Tightness The performance of ADOPT, OptAPO, and SynchBB deteriorates exponentially The algorithms did not terminate their run on the tightest problems (p 2 > 0.9) ADOPT is the worst, since it failed to terminate its run at a lower tightness value (0.7) DPOP’s runtime is high, but it always terminated and is independent of tightness AFB is clearly the best performing algorithm Outperforms DPOP by orders of magnitude

27. DCR workshop - May 200827 Analysis of ADOPT Analysis of ADOPT ADOPT is unable to solve hard problems High tightness generates an excess of (greedy) context switches along with an exponential increase in the number of messages sent Each agent in ADOPT sends out two messages following every single message it receives This causes the runtime of ADOPT to increase at a very high exponential rate

28. DCR workshop - May 200828 Analysis of DPOP Analysis of DPOP DPOP does not perform search or pruning Computes the same size of matrices regardless of the tightness A change in the problem’s tightness would only affect the content of the matrices

29. DCR workshop - May 200829 Analysis of AFB Analysis of AFB The performance of AFB exhibits a ”phase-transition” AFB’s runtime increases as the problem’s difficulty (tightness) increases, and then suddenly drops by an order of magnitude at very high values of p 2 This is very similar to COPs – deeper lookahead leads to much improved performance [Gershman et al. 2006] [Larrosa & Schiex 2004]

30. DCR workshop - May 200830 Conclusions Conclusions ADOPT appears to be affected the most by the increase of the problem’s tightness OptAPO performs up to three times better than SynchBB DPOP’s performance is independent of the problem’s tightness AFB performs well on the whole range of problem difficulty. It is unique in its phase transition. Probably due to its use of pruning techniques through asynchronous forward-bounding