DIMACS EAA:ICS 2006/04/041 Franc Brglez Raleigh, NC, USA Software Analysis via Data Analysis Xiao Yu Li Seattle, WA, USA Matthias F. Stallmann, 2006/04/04,

Slides:

Advertisements

Similar presentations

Distributional Property Estimation Past, Present, and Future Gregory Valiant (Joint work w. Paul Valiant)

Advertisements

4/17/20151 Franc Brglez Raleigh, NC, USA Matthias F. Stallmann Raleigh, NC, USA Effective Bounding Techniques For Solving Unate and Binate Covering Problems.

Recursion vs. Iteration The original Lisp language was truly a functional language: –Everything was expressed as functions –No local variables –No iteration.

ISE480 Sequencing and Scheduling Izmir University of Economics ISE Fall Semestre.

1 Backdoor Sets in SAT Instances Ryan Williams Carnegie Mellon University Joint work in IJCAI03 with: Carla Gomes and Bart Selman Cornell University.

Dynamic Restarts Optimal Randomized Restart Policies with Observation Henry Kautz, Eric Horvitz, Yongshao Ruan, Carla Gomes and Bart Selman.

On the Potential of Automated Algorithm Configuration Frank Hutter, University of British Columbia, Vancouver, Canada. Motivation for automated tuning.

S. J. Shyu Chap. 1 Introduction 1 The Design and Analysis of Algorithms Chapter 1 Introduction S. J. Shyu.

Automatic Algorithm Configuration based on Local Search EARG presentation December 13, 2006 Frank Hutter.

CP Formal Models of Heavy-Tailed Behavior in Combinatorial Search Hubie Chen, Carla P. Gomes, and Bart Selman

1 Optimisation Although Constraint Logic Programming is somehow focussed in constraint satisfaction (closer to a “logical” view), constraint optimisation.

Foundations of Constraint Processing Evaluation to BT Search 1 Foundations of Constraint Processing CSCE421/821, Spring

Implicit Hitting Set Problems Richard M. Karp Harvard University August 29, 2011.

Analysis of Algorithms CS 477/677

An Introduction to Black-Box Complexity

1 Backdoors To Typical Case Complexity Ryan Williams Carnegie Mellon University Joint work with: Carla Gomes and Bart Selman Cornell University.

Structure and Phase Transition Phenomena in the VTC Problem C. P. Gomes, H. Kautz, B. Selman R. Bejar, and I. Vetsikas IISI Cornell University University.

2-Layer Crossing Minimisation Johan van Rooij. Overview Problem definitions NP-Hardness proof Heuristics & Performance Practical Computation One layer:

Energy-Efficient Broadcasting in Ad-Hoc Networks: Combining MSTs with Shortest-Path Trees Carmine Ventre Joint work with Paolo Penna Università di Salerno.

Statistical Methods in Computer Science Hypothesis Testing I: Treatment experiment designs Ido Dagan.

Carla P. Gomes CS4700 CS 4700: Foundations of Artificial Intelligence Carla P. Gomes Module: Instance Hardness and Phase Transitions.

Simulated Annealing Van Laarhoven, Aarts Version 1, October 2000.

Ch 13 – Backtracking + Branch-and-Bound

1 CS 4700: Foundations of Artificial Intelligence Carla P. Gomes Module: Satisfiability (Reading R&N: Chapter 7)

Statistical Methods in Computer Science Hypothesis Testing I: Treatment experiment designs Ido Dagan.

Lukas Kroc, Ashish Sabharwal, Bart Selman Cornell University, USA SAT 2010 Conference Edinburgh, July 2010 An Empirical Study of Optimal Noise and Runtime.

1 Combinatorial Problems in Cooperative Control: Complexity and Scalability Carla Gomes and Bart Selman Cornell University Muri Meeting March 2002.

1 IE 607 Heuristic Optimization Introduction to Optimization.

Metaheuristics Meta- Greek word for upper level methods

1.1 Chapter 1: Introduction What is the course all about? Problems, instances and algorithms Running time v.s. computational complexity General description.

1 Efficient Stochastic Local Search for MPE Solving Frank Hutter The University of British Columbia (UBC), Vancouver, Canada Joint work with Holger Hoos.

Modeling and simulation of systems Simulation optimization and example of its usage in flexible production system control.

1 The Theory of NP-Completeness 2012/11/6 P: the class of problems which can be solved by a deterministic polynomial algorithm. NP : the class of decision.

Structure and Phase Transition Phenomena in the VTC Problem C. P. Gomes, H. Kautz, B. Selman R. Bejar, and I. Vetsikas IISI Cornell University University.

Stallmann/Brglez, High contrast… ExpCS, 2007/06/14, San Diego 1 High-Contrast Algorithm Behavior Observation, Hypothesis, and Experimental Design Matthias.

CHAPTERS 7, 8 Oliver Schulte Logical Inference: Through Proof to Truth.

Simulated Annealing.

Heavy-Tailed Phenomena in Satisfiability and Constraint Satisfaction Problems by Carla P. Gomes, Bart Selman, Nuno Crato and henry Kautz Presented by Yunho.

1 The Theory of NP-Completeness 2 Cook ’ s Theorem (1971) Prof. Cook Toronto U. Receiving Turing Award (1982) Discussing difficult problems: worst case.

§1.4 Algorithms and complexity For a given (optimization) problem, Questions: 1)how hard is the problem. 2)does there exist an efficient solution algorithm?

T. Messelis, S. Haspeslagh, P. De Causmaecker B. Bilgin, G. Vanden Berghe.

Version 1.1 Improving our knowledge of metaheuristic approaches for cell suppression problem Andrea Toniolo Staggemeier Alistair R. Clark James Smith Jonathan.

Optimization Problems

Minesweeper Solver Marina Maznikova Artificial Intelligence II Fall 2011.

Balance and Filtering in Structured Satisfiability Problems Henry Kautz University of Washington joint work with Yongshao Ruan (UW), Dimitris Achlioptas.

Review of Statistical Terms Population Sample Parameter Statistic.

Outline of Today’s Discussion 1.Displaying the Order in a Group of Numbers: 2.The Mean, Variance, Standard Deviation, & Z-Scores 3.SPSS: Data Entry, Definition,

Chapter 3 Statistical Models or Quality Control Improvement.

Tommy Messelis * Stefaan Haspeslagh Burak Bilgin Patrick De Causmaecker Greet Vanden Berghe *

Metaheuristics for the New Millennium Bruce L. Golden RH Smith School of Business University of Maryland by Presented at the University of Iowa, March.

Global Optimization of Complex Healthcare Systems

Integer Programming An integer linear program (ILP) is defined exactly as a linear program except that values of variables in a feasible solution have.

Heuristic Optimization Methods

Particle Swarm Optimization

Decision trees Polynomial-Time

Central Tendency and Variability

Statistical significance & the Normal Curve

NP-Completeness Yin Tat Lee

Optimal Solutions for Generalized Covering Problems

Worst-Case Execution Time

Probability & Statistics Probability Theory Mathematical Probability Models Event Relationships Distributions of Random Variables Continuous Random.

Constraint Programming and Backtracking Search Algorithms

Dispersion How values arrange themselves around the mean

Evaluation of (Deterministic) BT Search Algorithms

Topic 3: Prob. Analysis Randomized Alg.

Evaluation of (Deterministic) BT Search Algorithms

Click the mouse button or press the Space Bar to display the answers.

Sit in Groups Lab5_8.31 Effects on Adolescents’ Romantic Ideals

NP-Completeness Yin Tat Lee

Evaluation of (Deterministic) BT Search Algorithms

Presentation transcript:

DIMACS EAA:ICS 2006/04/041 Franc Brglez Raleigh, NC, USA Software Analysis via Data Analysis Xiao Yu Li Seattle, WA, USA Matthias F. Stallmann, 2006/04/04, DIMACS EAA:ISC planning meeting Based on joint work with:

DIMACS EAA:ICS 2006/04/042 Software versus Algorithms o Problems are NP-hard; require heuristics or (worst case) exponential algorithms. o Simple algorithms must be compared with cplex and other ILP solvers metaheuristics (SA, GA, “particle swarm”, “ants…”, etc.) with lots of adjustable parameters o Want “black box” comparison with (in many cases) no prior understanding of (some of) the algorithms o Problems are NP-hard; require heuristics or (worst case) exponential algorithms. o Simple algorithms must be compared with cplex and other ILP solvers metaheuristics (SA, GA, “particle swarm”, “ants…”, etc.) with lots of adjustable parameters o Want “black box” comparison with (in many cases) no prior understanding of (some of) the algorithms

DIMACS EAA:ICS 2006/04/043 Data Presentations based on CPLEX runs for (permutations of) a single instance o Descriptive Statistics mean / median / stdev / 25.3 / o Histogram o Percent solved o Descriptive Statistics mean / median / stdev / 25.3 / o Histogram o Percent solved

DIMACS EAA:ICS 2006/04/044 Stretching the Truth or making it clearer? more information here, but we need to look carefully

DIMACS EAA:ICS 2006/04/045 A more “normal” distribution CPLEX with different settings under the same conditions

DIMACS EAA:ICS 2006/04/046 uf : QT2/QT1 vs UW2/UW1 (1) runtime (seconds) solvability What about random 3-SAT instances? exp. d. 16.7/17.2

DIMACS EAA:ICS 2006/04/047 uf : QT2/QT1 vs UW2/UW1 (2) runtime (seconds) solvability exp. d. 12.3/12.5 exp. d. 16.7/17.2 UW1 performs the same as QT1 (t-test: t = 1.88 > 1.97)

DIMACS EAA:ICS 2006/04/048 uf : QT2/QT1 vs UW2/UW1 (3) runtime (seconds) solvability exp. d. 12.3/12.5 exp. d. 16.7/17.2 exp. d. 0.39/0.29 UW2 outperforms UW1 by a factor of UW1 performs the same as QT1 (t-test: t = 1.88 > 1.97)

DIMACS EAA:ICS 2006/04/049 uf : QT2/QT1 vs UW2/UW1 (4) exp. d. 0.31/0.28 exp. d. QT2 outperforms UW2 slightly (t-test: t = 2.24 > 1.97) UW2 outperforms UW1 by a factor of UW1 performs the same as QT1 (t-test: t = 1.88 > 1.97) runtime (seconds) solvability exp. d. 16.7/17.2 exp. d. 12.3/12.5 exp. d. 0.39/0.29 exp. d. 0.21/0.19

DIMACS EAA:ICS 2006/04/0410 Sources of Data Distributions first, 100 random instances (SAT) median4.8 mean21.2 stdev42.9 Heavy tail

DIMACS EAA:ICS 2006/04/0411 Not all instances are equal: here’s an easy one (128 permutations + original) median4.4 mean6.7 stdev6.5 Exponential

DIMACS EAA:ICS 2006/04/0412 …and here’s a harder one median84.8 mean126.7 stdev117.6 Exponential

DIMACS EAA:ICS 2006/04/0413 Another wrinkle: stochastic search first, same seed and 32 permuted instances + original median42484 mean62457 stdev86551 slightly worse than Exponential number of flips

DIMACS EAA:ICS 2006/04/0414 …versus 33 different seeds; same distribution? median36637 mean50185 stdev83226 slightly worse than Exponential number of flips

DIMACS EAA:ICS 2006/04/0415 Things get strange when the solver is not completely stochastic (e.g. B&B with stochastic search) Bi-modal: Stochastic search either finds optimum at root, or at first branch. Lower bound “finds” optimum at root. No randomness in LB method.

DIMACS EAA:ICS 2006/04/0416 Lower bound method is extremely sensitive to input ordering Heavy tail (and one instance times out): Lower bound method finds optimum at root or in an early branch only if input order is “friendly”.

DIMACS EAA:ICS 2006/04/0417 Another Data Analysis Application: Instance Profiling o sao2.b is very easy to solve (variables easy to distinguish) o e64.b is very difficult (lots of variables occur equally often)