1 Combinatorial Problems in Cooperative Control: Complexity and Scalability Carla Gomes and Bart Selman Cornell University Muri Meeting March 2002

2 We are investigating how to scale up solutions of the ROBOFLAG Drill focusing on: - Mixed Integer Program (MIP) formulations - Randomization - Approximation methods - Portfolios of Algorithms - Combining MIP and constraint search techniques.

3 Problem Representation ROBOFLAG Drill Formulation by Raff D’Andrea and Matt Earl. Problem is hybrid, combining discrete and continuous components, with multiple constraints. Represented as a mixed logical system (MLD) in which the objective is to compute optimal control policies that minimize the total score of the game. Mathematical Formulation of the Optimization Problem  Mixed Integer Linear Program

4 Scaling Up Mixed Integer Linear Program Formulations (MILP) Standard approach for solving MILP: Branch and Bound How can we improve upon Branch and Bound strategies? Ideas: Randomization Different search strategies for node selection Portfolios of algorithms

5 Branch & Bound: Depth First vs. Best bound Critical to performance of Branch & Bound is the way in which the next node to be expanded is selected. Standard approach: Best-bound --- select the node with the best LP bound Alternative: Depth-first --- often quickly reaches an integer solution (may take longer to produce an overall optimal value) Tradeoffs between these choices depend on underlying problem stucture (Gomes et al. 2001).

6 ROBOFLAG Testbed Depth First search works well. Problems that could not be solved before with best bound using were solved with depth first. Current largest problem solved with CPLEX using Depth First Search (8 attackers and 3 defenders): Integer variables = 4040 Continuous variables 400 Constraints constraints Time secs (Matt Earl 2002)

7 Much room for improvement… We are not yet incorporating any randomization or discrete constraint propagation techniques. Nor are we yet exploiting parallelism using a portfolio approach. Doing so should allow us to solve problems at least one or two orders of magnitude larger. (100,000 to 500,000 vars and 1,000,000+ constraints) Also, we should be able to include more complex constraints.

8 Other Formulations for Solving the Control Optimization Problem Encodings that provide “tighter” relaxations for the LP problem. Approximate representations using abstractions (“synthesize larger movements / trajecturies”). Less compact representations may allow for more propagation and scale up better. Constraint Satisfaction Problem (CSP) formulations. (*) Hybrid CSP/LP formulation. Approximations based on LP randomized rounding. (*)Sat – the satisfiability problem is a particular case of CSP; however, we believe that SAT encodings may not scale up well in this domain.

9 Overall the Roboflag control problem provides an excellent test bed for the development of scalable techniques for complex optimization.

10 Auxiliary Slides Background on improvements on branch and bound using randomization and parallel portfolios.

11 Branch & Bound (Randomized) Solve linear relaxation of MIP Branch on the integer variables for which the solution of the LP relaxation is non-integer: apply a good heuristic (e.g., max infeasibility) for variable selection ( + randomization ) and create two new nodes (floor and ceiling of the fractional value) Once we have found an integer solution, its objective value can be used to prune other nodes, whose relaxations have worse values

12 The performance of randomized Branch and Bound varies dramatically, on the same instance. In fact, the run time distributions often exhibit long tails (Heavy-tailed Distributions)

13 Heavy-tailed behavior of Depth-first

14 So, how can we take advantage of the high variability of randomized methods? - restart strategies - portfolio strategies

15 Algorithm Portfolio Design

16 Motivation The runtime and performance of randomized algorithms can vary dramatically on the same instance and on different instances. Goal: Improve the performance of different algorithms by combining them into a portfolio to exploit their relative strengths.

17 Portfolio of Algorithms A portfolio of algorithm is a collection of algorithms and / or copies of the same algorithm running interleaved or on different processors. Goal: to improve on the performance of the component algorithms in terms of: expected computational cost “risk” (variance) Efficient Set or Efficient Frontier: set of portfolios that are best in terms of expected value and risk.

18 Depth-first vs. Best-bound (logistics planning) Number of nodes Cumulative Frequencies Depth-First ~50% Best-Bound ~30%

19 Depth-First and Best and Bound do not dominate each other overall. What if we have more than one processors or if we interleave processes on a single processor?

20 Portfolio for heavy-tailed search procedures (2 processors) 0 DF / 2 BB 2 DF / 0 BB Standard deviation of run time of portfolios Expected run time of portfolios

21 Portfolio for heavy-tailed search procedures (20 processors) 0 DF / 20 BB 20 DF / 0 BB Standard deviation of run time of portfolios Expected run time of portfolios The optimal strategy is to run Depth First on the 20 processors!

22 Optimal collective behavior can emerge from suboptimal individual behavior.

23 A portfolio approach can lead to substantial improvements in the expected cost and risk of stochastic algorithms, especially in the presence of heavy-tailed phenomena.