2. Carla P. Gomes School on Optimization CPAIOR02 Exploiting Structure and Randomization in Combinatorial Search Carla P. Gomes gomes@cs.cornell.edu www.cs.cornell.edu/gomes Intelligent Information Systems Institute Department of Computer Science Cornell University www.cs.cornell.edu/gomes Exploiting Structure and Randomization in Combinatorial Search Carla P. Gomes gomes@cs.cornell.edu www.cs.cornell.edu/gomes Intelligent Information Systems Institute Department of Computer Science Cornell University www.cs.cornell.edu/gomes

3. Carla P. Gomes School on Optimization CPAIOR02 Outline A Structured Benchmark Domain Randomization Conclusions

4. Carla P. Gomes School on Optimization CPAIOR02 Outline A Structured Benchmark Domain Randomization Conclusions

5. Carla P. Gomes School on Optimization CPAIOR02 Given an N X N matrix, and given N colors, a quasigroup of order N is a a colored matrix, such that: -all cells are colored. - each color occurs exactly once in each row. - each color occurs exactly once in each column. Quasigroup or Latin Square (Order 4) Quasigroups or Latin Squares: An Abstraction for Real World Applications

6. Carla P. Gomes School on Optimization CPAIOR02 Quasigroup Completion Problem (QCP) Given a partial assignment of colors (10 colors in this case), can the partial quasigroup (latin square) be completed so we obtain a full quasigroup? Example: 32% preassignment (Gomes & Selman 97)

7. Carla P. Gomes School on Optimization CPAIOR02 Quasigroup Completion Problem A Framework for Studying Search NP-Complete. Has a structure not found in random instances, such as random K-SAT. Leads to interesting search problems when structure is perturbed (more about it later). Good abstraction for several real world problems: scheduling and timetabling, routing in fiber optics, coding, etc (Anderson 85, Colbourn 83, 84, Denes & Keedwell 94, Fujita et al. 93, Gent et al. 99, Gomes & Selman 97, Gomes et al. 98, Meseguer & Walsh 98, Stergiou and Walsh 99, Shaw et al. 98, Stickel 99, Walsh 99 )

8. Carla P. Gomes School on Optimization CPAIOR02 Fiber Optic Networks Nodes connect point to point fiber optic links

9. Carla P. Gomes School on Optimization CPAIOR02 Fiber Optic Networks Nodes connect point to point fiber optic links Each fiber optic link supports a large number of wavelengths Nodes are capable of photonic switching --dynamic wavelength routing -- which involves the setting of the wavelengths.

10. Carla P. Gomes School on Optimization CPAIOR02 Routing in Fiber Optic Networks Routing Node How can we achieve conflict-free routing in each node of the network? Dynamic wavelength routing is a NP-hard problem. Input PortsOutput Ports 1 2 3 4 1 2 3 4 preassigned channels

11. Carla P. Gomes School on Optimization CPAIOR02 QCP Example Use: Routers in Fiber Optic Networks Dynamic wavelength routing in Fiber Optic Networks can be directly mapped into the Quasigroup Completion Problem. (Barry and Humblet 93, Cheung et al. 90, Green 92, Kumar et al. 99) each channel cannot be repeated in the same input port (row constraints); each channel cannot be repeated in the same output port (column constraints); CONFLICT FREE LATIN ROUTER Input ports Output ports 3 1 2 4 Input PortOutput Port 1 2 4 3

12. Carla P. Gomes School on Optimization CPAIOR02 Traditional View of Hard Problems - Worst Case View “They’re NP-Complete—there’s no way to do anything but try heuristic approaches and hope for the best.”

13. Carla P. Gomes School on Optimization CPAIOR02 New Concepts in Computation Not all NP-Hard problems are the same! We now have means for discriminating easy from hard instances ---> Phase Transition concepts

14. Carla P. Gomes School on Optimization CPAIOR02 NP-completeness is a worst- case notion – what about average complexity? Structural differences between instances of the same NP- complete problem (QCP)

15. Carla P. Gomes School on Optimization CPAIOR02 Are all the Quasigroup Instances (of same size) Equally Difficult? 1820150 Time performance: 165 What is the fundamental difference between instances?

16. Carla P. Gomes School on Optimization CPAIOR02 Are all the Quasigroup Instances Equally Difficult? 1820165 40% 50% 150 Time performance: 35% Fraction of preassignment:

17. Carla P. Gomes School on Optimization CPAIOR02 Complexity of Quasigroup Completion Fraction of pre-assignment Median Runtime (log scale) Critically constrained area Overconstrained area Underconstrained area 42%50%20%

18. Carla P. Gomes School on Optimization CPAIOR02 Phase Transition Almost all unsolvable area Fraction of pre-assignment Fraction of unsolvable cases Almost all solvable area Complexity Graph Phase transition from almost all solvable to almost all unsolvable

19. Carla P. Gomes School on Optimization CPAIOR02 These results for the QCP - a structured domain, nicely complement previous results on phase transition and computational complexity for random instances such as SAT, Graph Coloring, etc. (Broder et al. 93; Clearwater and Hogg 96, Cheeseman et al. 91, Cook and Mitchell 98, Crawford and Auton 93, Crawford and Baker 94, Dubois 90, Frank et al. 98, Frost and Dechter 1994, Gent and Walsh 95, Hogg, et al. 96, Mitchell et al. 1992, Kirkpatrick and Selman 94, Monasson et 99, Motwani et al. 1994, Pemberton and Zhang 96, Prosser 96, Schrag and Crawford 96, Selman and Kirkpatrick 97, Smith and Grant 1994, Smith and Dyer 96, Zhang and Korf 96, and more)

20. Carla P. Gomes School on Optimization CPAIOR02 QCP Different Representations / Encodings

21. Carla P. Gomes School on Optimization CPAIOR02 Cubic representation of QCP Columns Rows Colors

22. Carla P. Gomes School on Optimization CPAIOR02 QCP as a MIP Variables - Constraints - Row/color line Column/color line Row/column line

23. Carla P. Gomes School on Optimization CPAIOR02 QCP as a CSP Variables - Constraints - row column [ vs. for MIP]

24. Carla P. Gomes School on Optimization CPAIOR02 Exploiting Structure for Domain Reduction A very successful strategy for domain reduction in CSP is to exploit the structure of groups of constraints and treat them as global constraints. Example using Network Flow Algorithms: All-different constraints (Caseau and Laburthe 94, Focacci, Lodi, & Milano 99, Nuijten & Aarts 95, Ottososon & Thorsteinsson 00, Refalo 99, Regin 94 )

25. Carla P. Gomes School on Optimization CPAIOR02 Exploiting Structure in QCP ALLDIFF as Global Constraint Two solutions: we can update the domains of the column variables Analogously, we can update the domains of the other variables Matching on a Bipartite graph All-different constraint (Berge 70, Regin 94, Shaw and Walsh 98 )

26. Carla P. Gomes School on Optimization CPAIOR02 Exploiting Structure Arc Consistency vs. All Diff Arc Consistency Solves up to order 20 Size search space AllDiff Solves up to order 33 Size search space

27. Carla P. Gomes School on Optimization CPAIOR02 Quasigroup as Satisfiability Two different encodings for SAT: 2D encoding (or minimal encoding); 3D encoding (or full encoding);

28. Carla P. Gomes School on Optimization CPAIOR02 2D Encoding or Minimal Encoding Variables: Each variables represents a color assigned to a cell. Clauses: Some color must be assigned to each cell (clause of length n); No color is repeated in the same row (sets of negative binary clauses); No color is repeated in the same column (sets of negative binary clauses);

29. Carla P. Gomes School on Optimization CPAIOR02 3D Encoding or Full Encoding This encoding is based on the cubic representation of the quasigroup: each line of the cube contains exactly one true variable; Variables: Same as 2D encoding. Clauses: Same as the 2 D encoding plus: Each color must appear at least once in each row; Each color must appear at least once in each column; No two colors are assigned to the same cell;

30. Carla P. Gomes School on Optimization CPAIOR02 Capturing Structure - Performance of SAT Solvers State of the art backtrack and local search and complete SAT solvers using 3D encoding are very competitive with specialized CSP algorithms. In contrast SAT solvers perform very poorly on 2D encodings (SATZ or SATO); In contrast local search solvers (Walksat) perform well on 2D encodings;

31. Carla P. Gomes School on Optimization CPAIOR02 SATZ on 2D encoding (Order 20 -28) SATZ and SATO can only solve up to order 28 when using 2D encoding; When using 3D encoding problems of the same size take only 0 or 1 backtrack and much higher orders can be solved; 1,000,000 Order 28 Order 20

32. Carla P. Gomes School on Optimization CPAIOR02 Walksat on 2D and 3D encoding (Order 30-33) 1,000,000 2D order 333D order 33 Walksat shows an unsual pattern - the 2D encodings are somewhat easier than the 3D encoding at the peak and harder in the undereconstrained region;

33. Carla P. Gomes School on Optimization CPAIOR02 Quasigroup - Satisfiability Encoding the quasigroup using only Boolean variables in clausal form using the 3D encoding is very competitive. Very fast solvers - SATZ, GRASP, SATO,WALKSAT;

34. Carla P. Gomes School on Optimization CPAIOR02 Structural features of instances provide insights into their hardness namely: Backbone Inherent Structure and Balance

35. Carla P. Gomes School on Optimization CPAIOR02 Backbone This instance has 4 solutions: Backbone Total number of backbone variables: 2 Backbone is the shared structure of all the solutions to a given instance.

36. Carla P. Gomes School on Optimization CPAIOR02 Phase Transition in the Backbone We have observed a transition in the backbone from a phase where the size of the backbone is around 0% to a phase with backbone of size close to 100%. The phase transition in the backbone is sudden and it coincides with the hardest problem instances. (Achlioptas, Gomes, Kautz, Selman 00, Monasson et al. 99)

37. Carla P. Gomes School on Optimization CPAIOR02 New Phase Transition in Backbone QCP (satisfiable instances only) % Backbone Sudden phase transition in Backbone Fraction of preassigned cells Computational cost % of Backbone

38. Carla P. Gomes School on Optimization CPAIOR02 Inherent Structure and Balance

39. Carla P. Gomes School on Optimization CPAIOR02 Quasigroup Patterns and Problems Hardness Rectangular PatternAligned PatternBalanced Pattern TractableVery hard (Kautz, Ruan, Achlioptas, Gomes, Selman 2001)

40. Carla P. Gomes School on Optimization CPAIOR02 SATZ Balanced QCP Rectangular QCP Aligned QCP QCP QWH

41. Carla P. Gomes School on Optimization CPAIOR02 Walksat aligned rectangular Balanced filtered QCP Balance QWH QCP QWH We observe the same ordering in hardness when using Walksat, SATZ, and SATO – Balacing makes instances harder

42. Carla P. Gomes School on Optimization CPAIOR02 Phase Transitions, Backbone, Balance Summary The understanding of the structural properties of problem instances based on notions such as phase transitions, backbone, and balance provides new insights into the practical complexity of many computational tasks. Active research area with fruitful interactions between computer science, physics (approaches from statistical mechanics), and mathematics (combinatorics / random structures).

43. Carla P. Gomes School on Optimization CPAIOR02 Outline A Structured Benchmark Domain Randomization Conclusions

44. Carla P. Gomes School on Optimization CPAIOR02 Randomized Backtrack Search Procedures

45. Carla P. Gomes School on Optimization CPAIOR02 Background Stochastic strategies have been very successful in the area of local search. Simulated annealing Genetic algorithms Tabu Search Gsat and variants. Limitation: inherent incomplete nature of local search methods.

46. Carla P. Gomes School on Optimization CPAIOR02 Background We want to explore the addition of a stochastic element to a systematic search procedure without losing completeness.

47. Carla P. Gomes School on Optimization CPAIOR02 We introduce stochasticity in a backtrack search method, e.g., by randomly breaking ties in variable and/or value selection. Compare with standard lexicographic tie-breaking. Randomization

48. Carla P. Gomes School on Optimization CPAIOR02 Randomization At each choice point break ties (variable selection and/or value selection) randomly or: “Heuristic equivalence” parameter (H) - at every choice point consider as “equally” good H% top choices; randomly select a choice from equally good choices.

49. Carla P. Gomes School on Optimization CPAIOR02 Randomized Strategies

50. Carla P. Gomes School on Optimization CPAIOR02 Quasigroup Demo

51. Carla P. Gomes School on Optimization CPAIOR02 Distributions of Randomized Backtrack Search Key Properties: I Erratic behavior of mean II Distributions have “heavy tails”.

52. Carla P. Gomes School on Optimization CPAIOR02 Median = 1! sample mean 3500! Erratic Behavior of Search Cost Quasigroup Completion Problem 500 2000 number of runs

53. Carla P. Gomes School on Optimization CPAIOR02 1

54. Carla P. Gomes School on Optimization CPAIOR02 75%<=30 Number backtracks Proportion of cases Solved 5%>100000

55. Carla P. Gomes School on Optimization CPAIOR02 Heavy-Tailed Distributions … infinite variance … infinite mean Introduced by Pareto in the 1920’s --- “probabilistic curiosity.” Mandelbrot established the use of heavy-tailed distributions to model real-world fractal phenomena. Examples: stock-market, earth- quakes, weather,...

56. Carla P. Gomes School on Optimization CPAIOR02 Decay of Distributions Standard --- Exponential Decay e.g. Normal: Heavy-Tailed --- Power Law Decay e.g. Pareto-Levy:

57. Carla P. Gomes School on Optimization CPAIOR02 Standard Distribution (finite mean & variance) Power Law Decay Exponential Decay

58. Carla P. Gomes School on Optimization CPAIOR02 Normal, Cauchy, and Levy Normal - Exponential Decay Cauchy -Power law Decay Levy -Power law Decay

59. Carla P. Gomes School on Optimization CPAIOR02 Tail Probabilities (Standard Normal, Cauchy, Levy)

60. Carla P. Gomes School on Optimization CPAIOR02 Example of Heavy Tailed Model (Random Walk) Random Walk: Start at position 0 Toss a fair coin: with each head take a step up (+1) with each tail take a step down (-1) X --- number of steps the random walk takes to return to position 0.

61. Carla P. Gomes School on Optimization CPAIOR02 The record of 10,000 tosses of an ideal coin (Feller) Zero crossing Long periods without zero crossing

62. Carla P. Gomes School on Optimization CPAIOR02 Random Walk Heavy-tails vs. Non-Heavy-Tails Normal (2,1000000) Normal (2,1) O,1%>200000 50% 2 Median=2 1-F(x) Unsolved fraction X - number of steps the walk takes to return to zero (log scale)

63. Carla P. Gomes School on Optimization CPAIOR02 How to Check for “Heavy Tails”? Log-Log plot of tail of distribution should be approximately linear. Slope gives value of infinite mean and infinite variance infinite mean and infinite variance infinite variance infinite variance

64. Carla P. Gomes School on Optimization CPAIOR02 Number backtracks (log) (1-F(x))(log) Unsolved fraction => Infinite mean Heavy-Tailed Behavior in QCP Domain 18% unsolved 0.002% unsolved

65. Carla P. Gomes School on Optimization CPAIOR02 Formal Models of Heavy-Tailed Behavior in Combinatorial Search Chen, Gomes, Selman 2001

66. Carla P. Gomes School on Optimization CPAIOR02 MotivationMotivation Research on heavy-tails has been largely based on empirical studies of run time distribution. Goal: to provide a formal characterization of tree search models and show under what conditions heavy-tailed distributions can arise. Intuition: Heavy-tailed behavior arises: from the fact that wrong branching decisions may lead the procedure to explore an exponentially large subtree of the search space that contains no solutions; the procedure is characterized by a large variability in the time to find a solution on different runs, which leads to highly different trees from run to run;

67. Carla P. Gomes School on Optimization CPAIOR02 Balanced vs. Imbalanced Tree Model Balanced Tree Model: chronological backtrack search model; fixed variable ordering; random child selection with no propagation mechanisms; (show demo)

68. Carla P. Gomes School on Optimization CPAIOR02 T(n) - the number of leaf nodes visited - choice at level i; (1 - bad choice; 0 -good choice) (note : there is exactly one choice of zero-one assignments to the variables for each possible value of T(n); any such assignment has probability. T(n) follows an Uniform distribution

69. Carla P. Gomes School on Optimization CPAIOR02 The run time distribution of chronological backtrack search on a complete balanced tree is uniform (therefore not heavy-tailed). Both the expected run time and variance scale exponentially

70. Carla P. Gomes School on Optimization CPAIOR02 Balanced Tree Model The expected run time and variance scale exponentially, in the height of the search tree (number of variables); The run time distribution is Uniform, (not heavy tailed ). Backtrack search on balanced tree model has no restart strategy with exponential polynomial time. Chen, Gomes & Selman 01

71. Carla P. Gomes School on Optimization CPAIOR02 How can we improve on the balanced serach tree model? Very clever search heuristic that leads quickly to the solution node - but that is hard in general; Combination of pruning, propagation, dynamic variable ordering that prune subtrees that do not contain the solution, allowing for runs that are short. ---> resulting trees may vary dramatically from run to run.

72. Carla P. Gomes School on Optimization CPAIOR02 T - the number of leaf nodes visited up to and including the successful node; b - branching factor Formal Model Yielding Heavy-Tailed Behavior b = 2 (show demo)

73. Carla P. Gomes School on Optimization CPAIOR02 Expected Run Time (infinite expected time) Variance (infinite variance) Tail (heavy-tailed)

74. Carla P. Gomes School on Optimization CPAIOR02 Bounded Heavy-Tailed Behavior (show demo)

75. Carla P. Gomes School on Optimization CPAIOR02 No Heavy-tailed behavior for Proving Optimality

76. Carla P. Gomes School on Optimization CPAIOR02 Proving Optimality

77. Carla P. Gomes School on Optimization CPAIOR02 Small-World Vs. Heavy-Tailed Behavior Does a Small-World topology (Watts & Strogatz) induce heavy-tail behavior? The constraint graph of a quasigroup exhibits a small-world topology (Walsh 99)

78. Carla P. Gomes School on Optimization CPAIOR02 Exploiting Heavy-Tailed Behavior Heavy Tailed behavior has been observed in several domains: QCP, Graph Coloring, Planning, Scheduling, Circuit synthesis, Decoding, etc. Consequence for algorithm design: Use restarts or parallel / interleaved runs to exploit the extreme variance performance. Restarts provably eliminate heavy-tailed behavior. (Gomes et al. 97, Hoos 99, Horvitz 99, Huberman, Lukose and Hogg 97, Karp et al 96, Luby et al. 93, Rish et al. 97, Wlash 99)

79. Carla P. Gomes School on Optimization CPAIOR02 XXXXX solved 10 Sequential: 50 +1 = 51 seconds Parallel: 10 machines --- 1 second 51 x speedup Super-linear Speedups Interleaved (1 machine): 10 x 1 = 10 seconds 5 x speedup

80. Carla P. Gomes School on Optimization CPAIOR02 Restarts 70% unsolved 1-F(x) Unsolved fraction Number backtracks (log) no restarts restart every 4 backtracks 250 (62 restarts) 0.001% unsolved

81. Carla P. Gomes School on Optimization CPAIOR02 Example of Rapid Restart Speedup (planning) 20 2000 ~100 restarts Cutoff (log) Number backtracks (log) ~10 restarts 100000

82. Carla P. Gomes School on Optimization CPAIOR02 Sketch of proof of elimination of heavy tails Let’s truncate the search procedure after m backtracks. Probability of solving problem with truncated version: Run the truncated procedure and restart it repeatedly.

83. Carla P. Gomes School on Optimization CPAIOR02 Y - does not have Heavy Tails

84. Carla P. Gomes School on Optimization CPAIOR02 Decoding in Communication Systems SourceEncoderDecoderDestination Channel Voice waveform, binary digits from a cd, output of a set of sensors in a space probe, etc. Telephone line, a storage medium, a space communication link, etc. usually subject to NOISE Processing prior to transmission, e.g., insertion of redundancy to combat the channel noise. Processing of the channel output with the objective of producing at the destination an acceptable replica of the source output. Decoding in communication systems is NP-hard. (Berlekamp, McEliece, and van Tilborg 1978, Barg 1998)

85. Carla P. Gomes School on Optimization CPAIOR02 Retransmissions in Sequential Decoding 1-F(x) Unsolved fraction Number backtracks (log) without retransmissions with retransmissions Gomes et al. 2000 / 20001

86. Carla P. Gomes School on Optimization CPAIOR02 Paramedic Crew Assignment Paramedic crew assignment is the problem of assigning paramedic crews from different stations to cover a given region, given several resource constraints.

87. Carla P. Gomes School on Optimization CPAIOR02 Deterministic Search

88. Carla P. Gomes School on Optimization CPAIOR02 Restarts

89. Carla P. Gomes School on Optimization CPAIOR02 Deterministic Logistics Planning108 mins.95 sec. Scheduling 14411 sec250 sec (*) not found after 2 days Scheduling 16---(*)1.4 hours Scheduling 18 ---(*)~18 hrs Circuit Synthesis 1---(*)165sec. Circuit Synthesis 2---(*)17min. Results on Effectiveness of Restarts R 3

90. Carla P. Gomes School on Optimization CPAIOR02 Algorithm Portfolio Design Gomes and Selman 1997 - Proc. UAI-97; Gomes et al 1997 - Proc. CP97.

91. Carla P. Gomes School on Optimization CPAIOR02 Motivation The runtime and performance of randomized algorithms can vary dramatically on the same instance and on different instances. Goal: Improve the performance of different algorithms by combining them into a portfolio to exploit their relative strengths.

92. Carla P. Gomes School on Optimization CPAIOR02 Branch & Bound: Best Bound vs. Depth First Search

93. Carla P. Gomes School on Optimization CPAIOR02 Branch & Bound (Randomized) Standard OR approach for solving Mixed Integer Programs (MIPs) Solve linear relaxation of MIP Branch on the integer variables for which the solution of the LP relaxation is non-integer: apply a good heuristic (e.g., max infeasibility) for variable selection ( + randomization ) and create two new nodes (floor and ceiling of the fractional value) Once we have found an integer solution, its objective value can be used to prune other nodes, whose relaxations have worse values

94. Carla P. Gomes School on Optimization CPAIOR02 Branch & Bound Depth First vs. Best bound Critical in performance of Branch & Bound: the way in which the next node to be expanded is selected. Best-bound - select the node with the best LP bound (standard OR approach) ---> this case is equivalent to A*, the LP relaxation provides an admissible search heuristic Depth-first - often quickly reaches an integer solution (may take longer to produce an overall optimal value)

95. Carla P. Gomes School on Optimization CPAIOR02 Portfolio of Algorithms A portfolio of algorithm is a collection of algorithms and / or copies of the same algorithm running interleaved or on different processors. Goal: to improve on the performance of the component algorithms in terms of: expected computational cost “risk” (variance) Efficient Set or Efficient Frontier: set of portfolios that are best in terms of expected value and risk.

96. Carla P. Gomes School on Optimization CPAIOR02 Depth-First: Average - 18000;St. Dev. 30000 Brandh & Bound for MIP Depth-first vs. Best-bound Cumulative Frequencies Number of nodes 30% Best bound Best-Bound: Average-1400 nodes; St. Dev.- 1300 Optimal strategy: Best Bound 45% Depth-first

97. Carla P. Gomes School on Optimization CPAIOR02 Depth-First and Best and Bound do not dominate each other overall.

98. Carla P. Gomes School on Optimization CPAIOR02 Heavy-tailed behavior of Depth-first

99. Carla P. Gomes School on Optimization CPAIOR02 Portfolio for heavy-tailed search procedures (2 processors) 0 DF / 2 BB 2 DF / 0 BB Standard deviation of run time of portfolios Expected run time of portfolios

100. Carla P. Gomes School on Optimization CPAIOR02 Portfolio for 6 processors 0 DF / 6 BB 6 DF / 0BB Expected run time of portfolios 5 DF / 1BB 3 DF / 3 BB 4 DF / 2 BB Efficient set Standard deviation of run time of portfolios

101. Carla P. Gomes School on Optimization CPAIOR02 Portfolio for 20 processors 0 DF / 20 BB 20 DF / 0 BB Expected run time of portfolios The optimal strategy is to run Depth First on the 20 processors! Optimal collective behavior emerges from suboptimal individual behavior. Standard deviation of run time of portfolios

102. Carla P. Gomes School on Optimization CPAIOR02 Compute Clusters and Distributed Agents With the increasing popularity of compute clusters and distributed problem solving / agent paradigms, portfolios of algorithms --- and flexible computation in general --- are rapidly expanding research areas. (Baptista and Marques da Silva 00, Boddy & Dean 95, Bayardo 99, Davenport 00, Hogg 00, Horvitz 96, Matsuo 00, Steinberg 00, Russell 95, Santos 99, Welman 99. Zilberstein 99)

103. Carla P. Gomes School on Optimization CPAIOR02 Portfolio for heavy-tailed search procedures (2-20 processors)

104. Carla P. Gomes School on Optimization CPAIOR02 A portfolio approach can lead to substantial improvements in the expected cost and risk of stochastic algorithms, especially in the presence of heavy-tailed phenomena.

105. Carla P. Gomes School on Optimization CPAIOR02 Summary of Randomization Considered randomized backtrack search. Showed Heavy-Tailed Distributions. Suggests: Rapid Restart Strategy. --- cuts very long runs --- exploits ultra-short runs Experimentally validated on previously unsolved planning and scheduling problems. Portfolio of Algorithms for cases where no single heuristic dominates

106. Carla P. Gomes School on Optimization CPAIOR02 Research Direction: Learning Restart Policies

107. Carla P. Gomes School on Optimization CPAIOR02 Bayesian Model Structure Learning (Horvitz, Ruan, Gomes, Kautz, Selman, Chickering 2001) Learning to infer predictive models from data and to identify key variables ==> restarts, cutoffs and other adaptive behavior of search algorithms.

108. Carla P. Gomes School on Optimization CPAIOR02 Green - long runs Gray - short runs Variance in number of uncolored cells across rows and columns Number uncolored cells per column Min depthAvg Depth Max number of uncolored cells across rows and columns Quasigroup Order 34 (CSP) Model accuracy 96.8% vs 48% for the marginal model

109. Carla P. Gomes School on Optimization CPAIOR02 Analysis of different solver features and problem features

110. Carla P. Gomes School on Optimization CPAIOR02 Outline A Structured Benchmark Domain Randomization Conclusions

111. Carla P. Gomes School on Optimization CPAIOR02 Summary The understanding of the structural properties of problem instances based on notions such as phase transitions, backbone, and balance provides new insights into the practical complexity of many computational tasks. Active research area with fruitful interactions between computer science, physics (approaches from statistical mechanics), and mathematics (combinatorics / random structures).

112. Carla P. Gomes School on Optimization CPAIOR02 Stochastic search methods (complete and incomplete) have been shown very effective. Restart strategies and portfolio approaches can lead to substantial improvements in the expected runtime and variance, especially in the presence of heavy-tailed phenomena. Randomization is therefore a tool to improve algorithmic performance and robustness. Machine Learning techniques can be used to learn predicitive models. Summary

113. Carla P. Gomes School on Optimization CPAIOR02 General Solution Methods Real World Problems Exploiting Structure: Tractable Components Transition Aware Systems (phase transition constrainedness backbone resources) Randomization Exploits variance to improve robustness and performance Bridging the Gap

114. Carla P. Gomes School on Optimization CPAIOR02 www.cs.cornell.edu/gomes www.cs.cornell.edu/gomes Check also: www.cis.cornell.edu/iisi www.cs.cornell.edu/gomes www.cs.cornell.edu/gomes Check also: www.cis.cornell.edu/iisi Demos, papers, etc.