T. Messelis, S. Haspeslagh, P. De Causmaecker B. Bilgin, G. Vanden Berghe.

Slides:

Advertisements

Similar presentations

R2Y-InterceptTstat YintSlopeTstat Slope Education014% Education211% Education40% Education62% Education827%

Advertisements

Preference Elicitation Partial-revelation VCG mechanism for Combinatorial Auctions and Eliciting Non-price Preferences in Combinatorial Auctions.

IBM Labs in Haifa © 2005 IBM Corporation Adaptive Application of SAT Solving Techniques Ohad Shacham and Karen Yorav Presented by Sharon Barner.

6-1 Introduction To Empirical Models 6-1 Introduction To Empirical Models.

Progress in Linear Programming Based Branch-and-Bound Algorithms

Linear Regression Using Excel 2010 Linear Regression Using Excel ® 2010 Managerial Accounting Prepared by Diane Tanner University of North Florida Chapter.

LECTURE 3 Introduction to Linear Regression and Correlation Analysis

THE SINGLE MACHINE EARLY/TARDY PROBLEM* PENG SI OW & THOMAS E. MORTON IE Paper Presentation A. İrfan Mahmutoğulları *Ow, P. S., & Morton, T. E. (1989).

Classification and Prediction: Regression Via Gradient Descent Optimization Bamshad Mobasher DePaul University.

SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT Lin Xu, Frank Hutter, Holger H. Hoos and Kevin Leyton-Brown University of British.

Hierarchical Hardness Models for SAT Lin Xu, Holger H. Hoos and Kevin Leyton-Brown University of British Columbia {xulin730, hoos,

St. Louis City Crime Analysis 2015 Homicide Prediction Presented by: Kranthi Kancharla Scott Manns Eric Rodis Kenneth Stecher Sisi Yang.

Foundations of Constraint Processing, Spring 2008 Evaluation to BT SearchApril 16, Foundations of Constraint Processing CSCE421/821, Spring 2008:

Chapter 12 Simple Regression

A new crossover technique in Genetic Programming Janet Clegg Intelligent Systems Group Electronics Department.

Chapter 13 Introduction to Linear Regression and Correlation Analysis

1 Chapter 9 Variable Selection and Model building Ray-Bing Chen Institute of Statistics National University of Kaohsiung.

McGraw-Hill/Irwin Copyright © 2013 by The McGraw-Hill Companies, Inc. All rights reserved. Business Statistics: Communicating with Numbers By Sanjiv Jaggia.

Hierarchical Hardness Models for SAT Hierarchical Hardness Models for SAT Building the hierarchical hardness models 1.The classifier is trained on a set.

1 Combinatorial Problems in Cooperative Control: Complexity and Scalability Carla Gomes and Bart Selman Cornell University Muri Meeting March 2002.

Chapter 14 Introduction to Linear Regression and Correlation Analysis

Hardness-Aware Restart Policies Yongshao Ruan, Eric Horvitz, & Henry Kautz IJCAI 2003 Workshop on Stochastic Search.

Chapter 6 (cont.) Regression Estimation. Simple Linear Regression: review of least squares procedure 2.

Classification and Prediction: Regression Analysis

1 Simple Linear Regression 1. review of least squares procedure 2. inference for least squares lines.

Quantitative Demand Analysis

Statistics for Business and Economics 8 th Edition Chapter 11 Simple Regression Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall Ch.

Time Series Analysis Introduction Averaging Trend Seasonality.

Efficient Model Selection for Support Vector Machines

Group Recommendations with Rank Aggregation and Collaborative Filtering Linas Baltrunas, Tadas Makcinskas, Francesco Ricci Free University of Bozen-Bolzano.

Statistics for Business and Economics 7 th Edition Chapter 11 Simple Regression Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall Ch.

An Online Auction Framework for Dynamic Resource Provisioning in Cloud Computing Weijie Shi*, Linquan Zhang +, Chuan Wu*, Zongpeng Li +, Francis C.M. Lau*

You want to examine the linear dependency of the annual sales of produce stores on their size in square footage. Sample data for seven stores were obtained.

Chap 12-1 A Course In Business Statistics, 4th © 2006 Prentice-Hall, Inc. A Course In Business Statistics 4 th Edition Chapter 12 Introduction to Linear.

Applied Quantitative Analysis and Practices LECTURE#23 By Dr. Osman Sadiq Paracha.

Three personnel structure examinations for improving nurse roster quality Komarudin, G. Vanden Berghe, M.-A. Guerry, and T. De Feyter.

1 Multiple Regression A single numerical response variable, Y. Multiple numerical explanatory variables, X 1, X 2,…, X k.

Transportation Engineering (CIVTREN) notes of AM Fillone, DLSU-Manila

ANOVA for Regression ANOVA tests whether the regression model has any explanatory power. In the case of simple regression analysis the ANOVA test and the.

Statistics for Business and Economics 8 th Edition Chapter 11 Simple Regression Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall Ch.

Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms Frank Hutter 1, Youssef Hamadi 2, Holger Hoos 1, and Kevin Leyton-Brown.

Business Statistics: A Decision-Making Approach, 6e © 2005 Prentice-Hall, Inc. Chap 13-1 Introduction to Regression Analysis Regression analysis is used.

Statistics for Business and Economics 8 th Edition Chapter 11 Simple Regression Copyright © 2013 Pearson Education, Inc. Publishing as Prentice Hall Ch.

Simple Linear Regression In the previous lectures, we only focus on one random variable. In many applications, we often work with a pair of variables.

Schreiber, Yevgeny. Value-Ordering Heuristics: Search Performance vs. Solution Diversity. In: D. Cohen (Ed.) CP 2010, LNCS 6308, pp Springer-

Ohad Hageby IDC Support Vector Machines & Kernel Machines IP Seminar 2008 IDC Herzliya.

Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms: An Initial Investigation Frank Hutter 1, Youssef Hamadi 2, Holger.

PLC Group: Mr. Keefe Mr. Brewer Mr. Skramstad Student Reading Habits and its Impact on CST.

Performance prediction for real world optimisation problems Tommy Messelis Stefaan Haspeslagh Burak Bilgin Patrick De Causmaecker Greet Vanden Berghe.

 Input parameters 1, 2, …, n  Values of each denoted X 1, X 2, X n  For each setting of X 1, X 2, X n observe a Y  Each set (X 1, X 2, X n,Y) is one.

Quality of LP-based Approximations for Highly Combinatorial Problems Lucian Leahu and Carla Gomes Computer Science Department Cornell University.

Patrick De Causmaecker Stefaan Haspeslagh Tommy Messelis.

Systems design for scheduling: Open Tools Patrick De Causmaecker, Peter Demeester, Greet Vanden Berghe and Bart Verbeke KaHo Sint-Lieven, Gent, Belgium.

Tommy Messelis * Stefaan Haspeslagh Patrick De Causmaecker *

Houston, Texas FAT CITY, USA Gloria Lobo-Stratton Sharon Lovdahl Dennis Glendenning.

A Kernel Approach for Learning From Almost Orthogonal Pattern * CIS 525 Class Presentation Professor: Slobodan Vucetic Presenter: Yilian Qin * B. Scholkopf.

Regression Modeling Applications in Land use and Transport.

Tommy Messelis * Stefaan Haspeslagh Burak Bilgin Patrick De Causmaecker Greet Vanden Berghe *

Metaheuristics for the New Millennium Bruce L. Golden RH Smith School of Business University of Maryland by Presented at the University of Iowa, March.

Construction Engineering 221 Probability and Statistics.

Statistics 350 Lecture 2. Today Last Day: Section Today: Section 1.6 Homework #1: Chapter 1 Problems (page 33-38): 2, 5, 6, 7, 22, 26, 33, 34,

Chapter 12 Simple Regression Statistika.  Analisis regresi adalah analisis hubungan linear antar 2 variabel random yang mempunyai hub linear,  Variabel.

Data Transformation: Normalization

Data Driven Resource Allocation for Distributed Learning

Investigation of fairness measures for nurse rostering

Evaluation of (Deterministic) BT Search Algorithms

Evaluation of (Deterministic) BT Search Algorithms

Evaluation of (Deterministic) BT Search Algorithms

Area Coverage Problem Optimization by (local) Search

Presentation transcript:

T. Messelis, S. Haspeslagh, P. De Causmaecker B. Bilgin, G. Vanden Berghe

 Introduction  Method  Our work  Conclusions  Future work T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 2

 predict performance ◦ of one or more algorithms ◦ on a specific problem instance  to be able to ◦ know in advance how good an algorithm will do ◦ choose the ‘best’ algorithm out of a portfolio ◦ choose the ‘best’ parameter setting T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 3

 Build empirical hardness models ◦ empirical: performance of some algorithm ◦ hardness: measured by some performance criteria  time spent by an algorithm searching for a solution  quality of an (optimal) solution  gap between found and optimal solution  Model hardness as a function of features ◦ computationally inexpensive ‘properties’  e.g. clauses-to-variables ratio (SAT)  e.g. maximum consecutive working days (NRP) T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 4

 Introduced by K. Leyton-Brown et al. 1.Select problem instance distribution 2.Select one or more algorithms 3.Create a set of features 4.Generate an instance set, calculate features and determine the algorithm performances 5.Eliminate redundant or uninformative features 6.Use machine learning techniques to select functions of the features that approximate the algorithm’s performances K. Leyton-Brown, E. Nudelman, Y. Shoham. Learning the empirical hardness of optimisation problems: The case of combinatorial auctions. In LNCS, 2002 T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 5

 This strategy has been successful in different areas: ◦ combinatorial auction: winner determination problem ◦ uniform random 3-SAT  accurate algorithm performance prediction  algorithm portfolio approach (SATzilla)  won several gold medals in SAT competitions  Apply it to Nurse Rostering! T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 6

 problem of assigning nurses to shifts, given a set of hard and soft constraints  Performance: ◦ time spent by a complete search algorithm to find the optimal roster ◦ quality of this optimal roster ◦ quality of a roster obtained by a heuristic algorithm, ran for some fixed period of time ◦ quality gap between both solutions T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 7

Translate NRP instances to SAT instances and use existing SAT features to build models  translation based on numberings  solve instances to optimum using CPLEX  run a metaheuristic for 10 seconds T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 8

 Regression results on predicting CPLEX objective T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 9 Regression Statistics Multiple R0, R Square0, Adjusted R Square0, Standard Error3, Observations500 ANOVAdfSSMSFSignificance F Regression , , , Residual , , Total CoefficientsStandard Errort StatP-valueLower 95%Upper 95% Intercept VCG CN variation549, , , ,6305E-69497, , VCG CN max16, , , ,1547E-12015, , VCG VN min-27, , , ,7809E , , CG mean-126, , , ,6608E , , BAL PL/C variation14819, , , ,9828E , ,28757 BAL , , , ,8022E , ,44717 BAL , , , ,3813E , ,843571

Come up with a feature set specifically for NRP and build models from these features  on the same set of NRP instances  for the same performance indicators T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 10

 very simple set of features: ◦ some problem parameters  max & min number of assignments  max & min number of consecutive working days  max & min number of consecutive free days ◦ and ratio’s of those parameters  max cons working days / min cons working days  max num assignments / min cons working days  availability / coverage requirements (tightness) ... T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 11

 Regression results on predicting CPLEX objective T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 12 Regression Statistics Multiple R0, R Square0, Adjusted R Square0, Standard Error2, Observations500 ANOVAdfSSMSFSignificance F Regression752571, , , ,2709E-297 Residual , , Total ,55 CoefficientsStandard Errort StatP-valueLower 95%Upper 95% Intercept66, , , ,6721E-16763, , max num assignments (6 - 10) - 6, , , ,9072E , , min cons working days (2 - 6)2, , , ,86075E-262, , max cons working days (3 - 8) - 2, , , ,2938E , , min cons free days (1 - 3)7, , , ,09346E-296, , max cons free days (2 - 3) - 2, , , , , , max num assignments / min cons work days1, , , ,24188E-060, , max cons FD / min cons FD2, , , ,0444E-051, ,

 We can build accurate models to predict algorithm performance, based on very basic properties of NRP instances. ◦ objective values  models for the objective values of both CPLEX and the metaheuristic are fairly accurate ◦ gap  less accurate predictions, however with a standard error of 0.95 ◦ CPLEX running time  not very accurate, due to very high variability in the running time T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 13

 building models on a larger scale ◦ now only a very limited dataset  more (sophisticated) features for NRP instances ◦ now only a very basic set with some aggregate functions of it  combining both SAT features and NRP features T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 14

T. Messelis, S. Haspeslagh, P. De Causmaecker, B. Bilgin, G. Vanden Berghe 15