Problem Solving with Constraints Ordering heuristics1 Foundations of Constraint Processing CSCE496/896, Fall 2011 www.cse.unl.edu/~choueiry/F11-496-896/

Slides:



Advertisements
Similar presentations
Slides by Peter van Beek Edited by Patrick Prosser.
Advertisements

Tree Clustering for Constraint Networks 1 Chris Reeson Advanced Constraint Processing Fall 2009 By Rina Dechter & Judea Pearl Artificial Intelligence,
Constraint Satisfaction Problems
1 Constraint Satisfaction Problems A Quick Overview (based on AIMA book slides)
This lecture topic (two lectures) Chapter 6.1 – 6.4, except 6.3.3
Traveling Salesperson Problem
1 Finite Constraint Domains. 2 u Constraint satisfaction problems (CSP) u A backtracking solver u Node and arc consistency u Bounds consistency u Generalized.
Artificial Intelligence Constraint satisfaction problems Fall 2008 professor: Luigi Ceccaroni.
an incremental version of A*
Constraint Satisfaction problems (CSP)
Optimal Rectangle Packing: A Meta-CSP Approach Chris Reeson Advanced Constraint Processing Fall 2009 By Michael D. Moffitt and Martha E. Pollack, AAAI.
Review: Constraint Satisfaction Problems How is a CSP defined? How do we solve CSPs?
Foundations of Constraint Processing, Spring 2008 April 16, 2008 Tree-Structured CSPs1 Foundations of Constraint Processing CSCE421/821, Spring 2008:
Junction Trees: Motivation Standard algorithms (e.g., variable elimination) are inefficient if the undirected graph underlying the Bayes Net contains cycles.
1 Discrete Structures & Algorithms Graphs and Trees: II EECE 320.
Constraint Satisfaction Problems
Anagh Lal Monday, April 14, Chapter 9 – Tree Decomposition Methods Anagh Lal CSCE Advanced Constraint Processing.
Foundations of Constraint Processing, Fall 2004 November 8, 2004Ordering heuristics1 Foundations of Constraint Processing CSCE421/821, Fall 2004:
Artificial Intelligence Constraint satisfaction Chapter 5, AIMA.
Foundations of Constraint Processing, Fall 2005 October 20, 2005Ordering heuristics1 Foundations of Constraint Processing CSCE421/821, Fall 2005:
Constraint Satisfaction Problems
Foundations of Constraint Processing, Fall 2005 Sep 20, 2005BT: A Theoretical Evaluation1 Foundations of Constraint Processing CSCE421/821, Fall 2005:
Ch 13 – Backtracking + Branch-and-Bound
Chapter 5 Outline Formal definition of CSP CSP Examples
MAC and Combined Heuristics: Two Reasons to Forsake FC (and CBJ?) on Hard Problems Christian Bessière and Jean-Charles Régin Presented by Suddhindra Shukla.
Foundations of Constraint Processing, Fall 2004 November 18, 2004More on BT search1 Foundations of Constraint Processing CSCE421/821, Fall 2004:
1 Constraint Satisfaction Problems Slides by Prof WELLING.
The Theory of NP-Completeness 1. Nondeterministic algorithms A nondeterminstic algorithm consists of phase 1: guessing phase 2: checking If the checking.
Artificial Intelligence CS482, CS682, MW 1 – 2:15, SEM 201, MS 227 Prerequisites: 302, 365 Instructor: Sushil Louis,
Constraint Satisfaction Read Chapter 5. Model Finite set of variables: X1,…Xn Variable Xi has values in domain Di. Constraints C1…Cm. A constraint specifies.
CP Summer School Modelling for Constraint Programming Barbara Smith 2. Implied Constraints, Optimization, Dominance Rules.
Constraint Satisfaction CPSC 386 Artificial Intelligence Ellen Walker Hiram College.
Searching by Constraint CMSC Artificial Intelligence January 24, 2008.
Hande ÇAKIN IES 503 TERM PROJECT CONSTRAINT SATISFACTION PROBLEMS.
Problem Solving with Constraints Lookahead Schemas 1 Foundations of Constraint Processing CSCE496/896, Fall
CSCI 5582 Fall 2006 CSCI 5582 Artificial Intelligence Fall 2006 Jim Martin.
CSE 589 Part VI. Reading Skiena, Sections 5.5 and 6.8 CLR, chapter 37.
1 Network Models Transportation Problem (TP) Distributing any commodity from any group of supply centers, called sources, to any group of receiving.
Wednesday, January 29, 2003CSCE Spring 2003 B.Y. Choueiry Directional Consistency Chapter 4.
Chapter 5 Constraint Satisfaction Problems
Review Test1. Robotics & Future Technology Future of Intelligent Systems / Ray Kurzweil futurist Ray Kurzweil / A Long Bet A Long Bet / Robot Soccer.
Foundations of Constraint Processing Ordering heuristics1 Foundations of Constraint Processing CSCE421/821, Spring
An Introduction to Artificial Intelligence Lecture 5: Constraint Satisfaction Problems Ramin Halavati In which we see how treating.
Foundations of Constraint Processing, Spring 2009 Structure-Based Methods: An Introduction 1 Foundations of Constraint Processing CSCE421/821, Spring 2009.
Foundations of Constraint Processing, Spring 2008 Feb 11, 2008 Ordering heuristics1 Foundations of Constraint Processing CSCE421/821, Spring 2008:
EXAMPLE: MAP COLORING. Example: Map coloring Variables — WA, NT, Q, NSW, V, SA, T Domains — D i ={red,green,blue} Constraints — adjacent regions must.
Structure-Based Methods Foundations of Constraint Processing
CSPs: Search and Arc Consistency Computer Science cpsc322, Lecture 12
CSPs: Search and Arc Consistency Computer Science cpsc322, Lecture 12
A Unified View of Graph Searching
Foundations of Constraint Processing
Structure-Based Methods Foundations of Constraint Processing
Extensions to BT Search Foundations of Constraint Processing
Extensions to BT Search Foundations of Constraint Processing
Foundations of Constraint Processing
Structure-Based Methods Foundations of Constraint Processing
Structure-Based Methods Foundations of Constraint Processing
Foundations of Constraint Processing All questions to Piazza
Extensions to BT Search Foundations of Constraint Processing
Lookahead Schemas Foundations of Constraint Processing
Extensions to BT Search Foundations of Constraint Processing
Constraints and Search
Intelligent Backtracking Algorithms: A Theoretical Evaluation
Foundations of Constraint Processing All questions to Piazza
Extensions to BT Search Foundations of Constraint Processing
Directional consistency Chapter 4
Structure-Based Methods Foundations of Constraint Processing
An Introduction Structure-Based Methods
Structure-Based Methods Foundations of Constraint Processing
Presentation transcript:

Problem Solving with Constraints Ordering heuristics1 Foundations of Constraint Processing CSCE496/896, Fall Berthe Y. Choueiry (Shu-we-ri) Avery Hall, Room 360 Tel: +1(402) Search Orders in CSPs

Problem Solving with Constraints Ordering heuristics2 Assumptions – Finite domains – Binary constraints Required reading –Chapter 6 of Tsang’s book (web accessible) Recommended reading –Dual viewpoint heuristics for binary Constraint Satisfaction Problems [Geelen, ECAI 92] In my experience the most powerful, but also the most costly (require lots of constraint checks)

Problem Solving with Constraints Ordering heuristics3 Context In BT, there are fewer backtracks –under some orderings than others In look-ahead, –failure can be detected earlier under some orderings than others When searching for one solution, –value ordering may speed up search as branches that have a better chance to reach a solution are explored first When using some kind of lookahead (e.g., FC) –dynamic ordering often reduce (remove?) the need for intelligent backtracking (personal experience)

Problem Solving with Constraints Goal of variable/value ordering Avoid constraint violation –Select values that do not cause constraint violation –Most promising value first (general principal) Discover constraint violation quickly –Select variables that do not delay the discovery of constraint violation –Most constrained variable first (general principal) Ordering heuristics4

Problem Solving with Constraints Ordering heuristics5 Value ordering: quickly get to a solution Min-conflict heuristic: orders values according to the conflicts in which they are involved with the future variables (most popular) [Minton] Cruciality [Keng & Yu ’89] Promise (most powerful) [Geelen ’92] Etc.

Problem Solving with Constraints Notation Geelen ECAI 92 For a given future variable V i, the assignment V i  x partitions the domain of a future variable V j –A set of values consistent with V i  x of size Left(V j | V i  x) –A set of values inconsistent with V i  x of size Lost(V j | V i  x) Ordering heuristics6

Problem Solving with Constraints Value selection in terms of Lost, Left V i, V j : future variables Minimize min-conflict Cost( V i  x )=  Vj  i Lost (V j | V i  x) Minimize [Keng & Yun 89] Cruciality ( V i  x ) =  Vj  i Lost (V j | V i  x) / |D Vj | Maximize [Geelen ECAI 92] Promise ( V i  x ) =  Vj  i Left (V j | V i  x) number of assignments that can be done such that no constraint on V i is broken Ordering heuristics7

Problem Solving with Constraints Advantages of Promise for value selection Advantages that are unique to Promise: Product recognizes domain wipe-out, sum does not –Compare 6+0 and 6x0 Product discriminates better than sum: –6+0, 5+1, 4+2, 3+3 are all equivalent However (6x0) < (5x1) < (4x2) < (3x3) Semantics of Promise (i.e., physical interpretation) –Upper bound on number of solutions Ordering heuristics8

Problem Solving with Constraints Value selection: example Minimize cost Cost( V i  x )=  Vj  i Lost (V j | V i  x) Ordering heuristics9 Maximize Promise Promise ( V i  x ) =  Vj  I Left (V j | V i  x) Q1Q Q2Q Q3Q Q4Q Q1Q Q2Q Q3Q Q4Q4 6666

Problem Solving with Constraints Ordering heuristics10 Variable Ordering: Fail-first principle Recognize dead-ends ASAP to save search effort Terminology (FFP) is historic, currently contested If you are on –a correct path, stay on it –an incorrect path, ‘fail’ on it Problem: –How to guess whether a path is correct? Advice: –choose the ordering that reduces the branching factor, the enemy of search..

Problem Solving with Constraints Ordering heuristics11 Variable ordering heuristics Least domain (LD), a.k.a. smallest domain Maximal (future/forward) degree (deg, fdeg, ddeg) Minimal ratio domain size over degree (ddr, dom/deg, dom/ddeg) Promise for variables Brélaz heuristic (originally for graph coloring) Weighted degree (wdeg) [Boussemart et al, ECAI 04] Minimal width ordering (MWO) Maximal cardinality ordering (MCO) Minimal bandwidth ordering (BBO) Alert: Always exploit domino effect (domain has 1 value), especially under dynamic variable ordering Always state how you are breaking ties (e.g., lexicographically) In general: –Cheap and effective –In most cases, suitable for both static and dynamic ordering

Problem Solving with Constraints Promise for variable selection Minimize Promise ( V i )=  x  Dvi Promise ( V i  x ) Interpretation: –minimize the number of assignments that can be done such that no constraint on V i is broken –Upper bound on the number of solutions Ordering heuristics12 Q Q Q Q Q1, Q4 promise 28 solutions Q2, Q3 promise 20 solutions Start with Q2 or Q3 (more constraining) choosing Col1 or 4 (more promising)

Problem Solving with Constraints Ordering heuristics13 Brélaz CACM, 79 Originally designed for coloring. –Assigns the most constrained nodes first (i.e., those with the most distinctly colored neighbors) –breaking ties by choosing nodes with the most uncolored neighbors. 1.Arrange the variables in decreasing order of degrees 2.Color a vertex with maximal degree with color 1. 3.Choose a vertex with a maximal saturation degree (number of different colors to which is it adjacent). –If there is equality (tie), choose any vertex of maximal degree in the uncolored graph 4.Color the chosen vertex (with the lowest numbered color) 5.If all vertices are colored, stop. Otherwise, return to 3.

Problem Solving with Constraints Ordering heuristics14 wdeg [Boussemart et al, ECAI 04] All constraints are assigned a weight, first set to 1 Every time a constraint is broken during propagation with look- ahead (constraint causing domain wipe-out), its weight is increased The weight of an un-assigned variable is defined as the sum of the weights of the constraints that apply to the variable The variable with largest weight is chosen for instantiation Weights are not reinitialized upon backtracking Refinement: dom/wdeg Historically: inspired by breakout heuristic of [Morris, AAAI 93], commonly used in local search

Problem Solving with Constraints Ordering heuristics15 Variable ordering heuristics Least domain (LD), a.k.a. smallest domain Maximal (future/forward) degree (deg, fdeg, ddeg) Minimal ratio domain size over degree (ddr, dom/deg, dom/ddeg) Promise for variables Brélaz heuristic (originally for graph coloring) Weighted degree (wdeg) [Boussemart et al, ECAI 04] Minimal width ordering (MWO) Maximal cardinality ordering (MCO) Minimal bandwidth ordering (BBO) Alert: Always exploit domino effect (domain has 1 value), especially under dynamic variable ordering Always state how you are breaking ties (e.g., lexicographically) In general: –Cheap and effective –In most cases, suitable for both static and dynamic ordering

Problem Solving with Constraints Ordering heuristics16 Graph-based heuristics Minimal width ordering (MWO) Maximal cardinality ordering (MCO) Minimal bandwidth ordering (BBO)

Problem Solving with Constraints Ordering heuristics17 Width of a graph A graph-theoretic criterion Constraint graph Ordering of nodes how many possible orderings? Width of an ordering Width of the graph (independent of the ordering)

Problem Solving with Constraints Ordering heuristics18 Minimal width ordering (MWO) Reduces the chance of backtracking: Variables at the front of the ordering are in general more constrained Variables that have more variables depending on them are labeled first Finding minimum width ordering: O(n 2 ) A B C D G F E Compute width of A, B, C, D, E, F, G Compute width of G, F, E, D, C, B, A

Problem Solving with Constraints Ordering heuristics19 Procedure: Width or a graph G Remove from the graph all nodes not connected to any others Set k  0 Do while there are nodes left in the graph –Set k  (k+1) –Do While there are nodes not connected to more than k others Remove such nodes from the graph, along with any edges connected to them Return k The minimal width ordering of the nodes is obtained by taking the nodes in the reverse order than they were removed

Problem Solving with Constraints Ordering heuristics20 Graph-based heuristics Minimal width ordering (MWO) Maximal cardinality ordering (MCO) –Goal: minimize induced width w* Minimal bandwidth ordering (BBO)

Problem Solving with Constraints Induced width w* Induced width of an ordering –Given an ordering –Moralize the graph (i.e., marry the parents) –Induced width is the width of the moralized graph Induced width of a graph w* –Is the minimum induced width of all possible orderings Given a graph –Finding w* (and the corresponding ordering) –Is NP-hard –Solution: heuristically approximated Ordering heuristics21

Problem Solving with Constraints Moralized graph –Is a triangulated or chordal graph –Is a graph where ∀ cycle of length ≥ 4 has a chord (chord = an edge between 2 non-consecutive vertices) A triangulated has a perfect elimination ordering σ Triangulating a graph –May require adding edges, fill-ins e ⊆ e’ –Minimizing e’ is NP-hard –MinFill = good heuristic for reducing |e’| MaxCardinality –Recognizes triangulated graphs –If graph is triangulated, gives PE ordering σ An approximation of w* Ordering heuristics22 MinFill MaxCardinality G=(v,e) G=(v,e ⊆ e’) σ

Problem Solving with Constraints Ordering heuristics23 Min Fill Dechter Figure When removing a node –add a fill-in edge between every two nodes connected to the node but not connected between themselves 2.Remove the node that, after removal, yields the smallest number of fill-in edges

Problem Solving with Constraints Ordering heuristics24 Maximal cardinality ordering (1) An approximation of min. width ordering Choose a node arbitrarily Among the remaining nodes –choose the one connected to the maximum number of already chosen nodes –break ties arbitrarily Repeat… Obtained ordering: instantiation ordering Reverse ordering yields σ, a PEO Tsang Dechter Fig 4.5

Problem Solving with Constraints Ordering heuristics25 Maximal cardinality ordering (2) 1.Choose a node arbitrarily 2.Among the remaining nodes, choose the one connected to the maximum number of already chosen nodes, break ties arbitrarily 3.Repeat Step 2 until all nodes have been selected 4.Reverse the order of nodes Tsang Dechter Fig 4.5 After Step 3: Instantiation order After Step 4: if graph is triangulated, σ (PEO) Use of Max Cardinality Ordering –A decent ordering heuristic (a good approximation of induced width) –Recognition of a triangulated graph: when every vertex after Step 4 simplicial –Simplicial: a vertex whose neighbors are all connected

Problem Solving with Constraints Ordering heuristics26 Graph-based heuristics Minimal width ordering (MWO) Maximal cardinality ordering (MCO) Minimal bandwidth ordering (BBO)

Problem Solving with Constraints Ordering heuristics27 Minimal Bandwidth Ordering Tsang Definitions: The bandwidth of… –A node is the largest distance between the node and its neighbors –An ordering is the largest bandwidth of the nodes in the ordering –A graph is the smallest bandwidth across all its possible orderings Localizes/confines backtracking The smaller the bandwidth, the sooner one could backtrack to relevant decisions Finding minimum bandwidth ordering is NP-hard  Is there an ordering of a given bandwidth k? –O(n k+1 ), i.e. polynomial

Problem Solving with Constraints Ordering heuristics28 Ordering heuristics: how, when? How –Static variable, value ordering –Dynamic variable (static value) –Dynamic variable, dynamic value (dynamic vvp) When –Finding one solution –Finding all solutions

Problem Solving with Constraints Ordering heuristics29 Computing the orders Static –Sort all variables, at pre-processing –Based on: Initial domain (for LD, dom/deg, etc.) All or remaining neighbors of a variable (for deg, dom/deg, etc.) Dynamic –Select one variable, during search –Based on: Current domain (for LD, ddr, etc.) All un-instantiated neighbors of a variable (for deg, ddr, etc.) –Exploit the domino effect When the domain of any future variable has a single value, instantiate this variable first.

Problem Solving with Constraints Ordering heuristics30 Search & ordering heuristics At a given level h of the search tree, we encounter:

Problem Solving with Constraints Ordering heuristics31 Static variable, static value vvps pertaining to the same variable across a given level

Problem Solving with Constraints Ordering heuristics32 Dynamic variable, static value vvps pertaining to the same variable for nodes with a common parent, but possibly to different variables for nodes with different parents

Problem Solving with Constraints Ordering heuristics33 Dynamic vvp vvps pertaining to different variables

Problem Solving with Constraints Ordering heuristics34 Side comment Common wisdom –When looking for all solutions, value ordering does not matter Above ‘wisdom’ holds for k-way branching –[Smith, IJCAI 05] showed that this is not true for 2-way branching, which, apparently, is used in commercial products such as ECLiPSe and Ilog The benefits (if any) and implications of 2-way branching versus k-way branching are not fully studied yet