1. Foundations of Constraint Processing
Constraint Satisfaction 101
Foundations of Constraint Processing
CSCE421/821, Fall 2004:
Berthe Y. Choueiry (Shu-we-ri)
Avery Hall, Room 123B
Tel: +1(402)
5/1/2019
Lecture slides 1

2. Outline Motivating example, application areas
CSP: Definition, representation
Some simple modeling examples
More on definition and formal characterization
Basic solving techniques
(Implementing backtrack search)
Advanced solving techniques
Issues & research directions
5/1/2019
Lecture slides 1

3. Motivating example I Context: You are a senior in college
Problem: You need to register in 4 courses for the Spring semester
Possibilities: Many courses offered in Math, CSE, EE, CBA, etc.
Constraints: restrict the choices you can make
Unary: Courses have prerequisites you have/don't have Courses/instructors you like/dislike
Binary: Courses are scheduled at the same time
n-ary: In CE: 4 courses from 5 tracks such as at least 3 tracks are covered
You have choices, but are restricted by constraints
Make the right decisions!!
5/1/2019
Lecture slides 1

4. Motivating example II Given Questions
A set of variables: 4 courses at UNL
For each variable, a set of choices (values)
A set of constraints that restrict the combinations of values the variables can take at the same time
Questions
Does a solution exist? (classical decision problem)
How two or more solutions differ? How to change specific choices without perturbing the solution?
If there is no solution, what are the sources of conflicts? Which constraints should be retracted?
etc.
5/1/2019
Lecture slides 1

5. Practical applications
Adapted from E.C. Freuder
Radio resource management (RRM)
Databases (computing joins, view updates)
Temporal and spatial reasoning
Planning, scheduling, resource allocation
Design and configuration
Graphics, visualization, interfaces
Hardware verification and software engineering
HC Interaction and decision support
Molecular biology
Robotics, machine vision and computational linguistics
Transportation
Qualitative and diagnostic reasoning
5/1/2019
Lecture slides 1

6. Constraint Processing
is about ...
Solving a decision problem…
… While allowing the user to state arbitrary constraints in an expressive way and
Providing concise and high-level feedback about alternatives and conflicts
Related areas:
AI, OR, Algorithmic, DB, TCS, Prog. Languages, etc.
5/1/2019
Lecture slides 1

7. Power of Constraints Processing
Flexibility & expressiveness of representations
Interactivity
users can constraints
relax
reinforce
5/1/2019
Lecture slides 1

8. Outline Motivating example, application areas
CSP: Definition, representation
Some simple modeling examples
More on definition and formal characterization
Basic solving techniques
5/1/2019
Lecture slides 1

9. Definition of a Problem
General template of any computational problem
Given:
Example: a set of objects, their relations, etc.
Query/Question:
Example: Find x such that the condition y is satisfied
How about the Constraint Satisfaction Problem?
5/1/2019
Lecture slides 1

10. Definition of a CSP Given P = (V, D, C ) V is a set of variables,
D is a set of variable domains (domain values)
C is a set of constraints,
Query: can we find a value for each variable such that all constraints are satisfied?
5/1/2019
Lecture slides 1

11. Other queries… Find a solution Find all solutions
Find the number of solutions
Find a set of constraints that can be removed so that a solution exists
Etc.
5/1/2019
Lecture slides 1

12. Representation I Given P = (V, D, C ), where
Find a consistent assignment for variables
Constraint Graph
Variable   node (vertex)
Domain  node label
Constraint  arc (edge) between nodes
5/1/2019
Lecture slides 1

13. Representation II Given P = (V, D, C ), where Example I: Define C ?
{1, 2, 3, 4}
{ 3, 5, 7 }
{ 3, 4, 9 }
{ 3, 6, 7 }
v2 > v4 V4 V2
v1+v3 < 9 V3 V1
v2 < v3
v1 < v2
5/1/2019
Lecture slides 1

14. Outline Motivating example, application areas
CSP: Definition, representation
Some simple modeling examples
More on definition and formal characterization
Basic solving techniques
5/1/2019
Lecture slides 1

15. Example II: Temporal reasoning
Give one solution: …….
Satisfaction, yes/no: decision problem
5/1/2019
Lecture slides 1

16. Example III: Map coloring
Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color
Variables?
Domains?
Constraints?
5/1/2019
Lecture slides 1

17. Example III: Map coloring
Using 3 colors (R, G, & B), color the US map such that no two adjacent states have the same color
5/1/2019
Lecture slides 1

18. Example IV: Resource Allocation
What is the CSP formulation?
{ a, b, c }
{ a, b } { a, c, d }
{ b, c, d }
5/1/2019
Lecture slides 1

19. Example IV: Resource Allocation
{ a, b, c }
{ a, b } { a, c, d }
{ b, c, d } T1
Constraint Graph T4
{ R1, R3 } T3
{ R2, R4 } T6 T7 T5
{ R1, R2, R3 } 
Interval Order
{ R1, R3 } T1
{ R1, R3 } T2
{ R1, R3 }
{ R1, R3, R4 } T3 T4 T2
{ R1, R2, R3 } T5
{ R2, R4 } T6
{ R2, R4 } T7
5/1/2019
Lecture slides 1

20. Example V: Puzzle Given:
Four musicians: Fred, Ike, Mike, and Sal, play bass, drums, guitar and keyboard, not necessarily in that order.
They have 4 successful songs, ‘Blue Sky,’ ‘Happy Song,’ ‘Gentle Rhythm,’ and ‘Nice Melody.’
Ike and Mike are, in one order or the other, the composer of ‘Nice Melody’ and the keyboardist.
etc ...
Query: Who plays which instrument and who composed which song?
5/1/2019
Lecture slides 1

21. Example V: Puzzle Formulation 1: Formulation 2:
Variables: Bass, Drums, Guitar, Keyboard, Blue Sky, Happy Song
Gentle Rhythm and Nice Melody.
Domains: Fred, Ike, Mike, Sal
Constraints: …
Formulation 2:
Variables: Fred's-instrument, Ike's-instrument, …,
Fred's-song, Ikes's-song, Mike’s-song, …, etc.
Domains:
{ bass, drums, guitar, keyboard }
{ Blue Sky, Happy Song, Gentle Rhythm, Nice Melody}
5/1/2019
Lecture slides 1

22. Example VI: Product Configuration
Train, elevator, car, etc.
Given:
Components and their attributes (variables)
Domain covered by each characteristic (values)
Relations among the components (constraints)
A set of required functionalities (more constraints)
Find: a product configuration
i.e., an acceptable combination of components
that realizes the required functionalities
5/1/2019
Lecture slides 1

23. Examples of Constraint Types
Example I: algebraic constraints
Example II:
(algebraic) constraints
of bounded difference
Example III & IV: coloring, mutual exclusion, difference constraints
Example V & VI: elements of C must be made explicit
{1, 2, 3, 4}
{ 3, 5, 7 }
{ 3, 4, 9 }
{ 3, 6, 7 }
v2 > v4 V4 V2
v1+v3 < 9 V3 V1
v2 < v3
v1 < v2 T1
Constraint Graph T4
{ R1, R3 } T3
{ R2, R4 } T6 T7 T5
{ R1, R2, R3 } 
5/1/2019
Lecture slides 1

24. More examples Example VII: Databases Example VIII: Interactive systems
Join operation in relational DB is a CSP
View materialization is a CSP
Example VIII: Interactive systems
Data-flow constraints
Spreadsheets
Graphical layout systems and animation
Graphical user interfaces
Example IX: Molecular biology (bioinformatics)
Threading, etc
5/1/2019
Lecture slides 1

25. Outline Motivating example, application areas
CSP: Definition, representation
Some simple modeling examples
More on definition and formal characterization
Basic solving techniques
5/1/2019
Lecture slides 1

26. Representation (again)
Macrostructure G(P):
- constraint graph for
binary constraints
- constraint network for
non-binary constraints
Micro-structure  (P):
Co-microstructure co-(P):
a, b
a, c
b, c  V1 V2 V3
(V1, a ) (V1, b)
(V2, a ) (V2, c)
(V3, b ) (V3, c)
no goods
(V1, a ) (V1, b)
(V2, a ) (V2, c)
(V3, b ) (V3, c)
5/1/2019
Lecture slides 1

27. Constraint arity I Given P = (V, D, C ) , where
{1, 2, 3, 4}
{ 3, 5, 7 }
{ 3, 4, 9 }
{ 3, 6, 7 }
v2 > v4 V4 V2
v1+v3 < 9 V3 V1
v2 < v3
v1 < v2
How to represent the constraint V1 + V2 + V4 < 10 ?
5/1/2019
Lecture slides 1

28. Constraint arity II Given P = (V, D, C ) , where
Constraints: universal, unary, binary, ternary, … , global.
A Constraint Network:
5/1/2019
Lecture slides 1

29. Domain types Domains: P = (V, D, C ) where
Finite (discrete), enumeration works
Continuous, sophisticated algebraic techniques are needed
5/1/2019
Lecture slides 1

30. Constraint terminology
Arity:
universal, unary, binary, ternary, …, global
Scope:
The set of variables to which the constraint applies
Definition:
Intention, extension
Implementation:
predicate, set of tuples, binary matrix, etc.
5/1/2019
Lecture slides 1

31. Complexity of CSP Characterization: Decision problem
In general, NP-complete
by reduction from 3SAT
5/1/2019
Lecture slides 1

32. Proving NP-completeness
Show that 1 is in NP
Given a problem 1 in NP, show that an known NP-complete problem 2 can be efficiently reduced to 1
Select a known NP-complete problem 2 (e.g., SAT)
Construct a transformation f from 2 to 1
Prove that f is a polynomial transformation
(Check Chapter 3 of Garey & Johnson)
5/1/2019
Lecture slides 1

33. What is SAT? Given a sentence:
Sentence: conjunction of clauses
Clause: disjunction of literals
Literal: a term or its negation
Term: Boolean variable
Question: Find an assignment of truth values to the Boolean variables such the sentence is satisfied.
5/1/2019
Lecture slides 1

34. CSP is NP-Complete
Verifying that an assignment for all variables is a solution
Provided constraints can be checked in polynomial time
Reduction from 3SAT to CSP
Many such reductions exist in the literature (perhaps 7 of them)
5/1/2019
Lecture slides 1

35. Problem reduction
Example: CSP into SAT (proves nothing, just an exercise)
Notation: variable-value pair = vvp
vvp  term
V1 = {a, b, c, d} yields x1 = (V1, a), x2 = (V1, b), x3 = (V1, c), x4 = (V1, d),
V2 = {a, b, c} yields x5 = (V2, a), x6 = (V2, b), x7 = (V2,c).
The vvp’s of a variable  disjunction of terms
V1 = {a, b, c, d} yields
(Optional) At most one VVP per variable
5/1/2019
Lecture slides 1

36. CSP into SAT (cont.) Constraint:
Way 1: Each inconsistent tuple  one disjunctive clause
For example: how many?
Way 2:
Consistent tuple  conjunction of terms
Each constraint  disjunction of these conjunctions
 transform into conjunctive normal form (CNF)
Question: find a truth assignment of the Boolean variables such that the sentence is satisfied
5/1/2019
Lecture slides 1

37. Outline Motivating example, application areas
CSP: Definition, representation
Some simple modeling examples
More on definition and formal characterization
Basic solving techniques
Modeling and consistency checking
Constructive, systematic search
Iterative improvement, local search
5/1/2019
Lecture slides 1

38. How to solve a CSP? Search 1. Constructive, systematic
2. Iterative repair, local search
5/1/2019
Lecture slides 1

39. Before starting search!
Consider:
Importance of modeling/formulation:
To control the size of the search space
Preprocessing
A.k.a. constraint filtering/propagation, consistency checking
reduces size of search space
5/1/2019
Lecture slides 1

40. Importance of modeling
N-queen: formulation 1
Variables?
Domains?
Size of CSP?
N-queens: formulation 2
4 Column positions
4 Rows V1 V2 V3 V4
16 Cells
V11 V12 V13 V14
V21 V22 V23 V24
V31 V32 V33 V34
V41 V42 V43 V44
{0,1}
5/1/2019
Lecture slides 1

41. Constraint checking Arc-consistency B C: [ 6 .. 15 ] 1- B: [ 5 .. 14 ]13
C: [ ]
1- B: [ ]
A < B 14 A
[ ] 2
2- A: [ ]
C: [ ]
B < C
[ ]
2 < C - A < 5
3- B: [ ] 6 14
[ ] C
5/1/2019
Lecture slides 1

42. Constraint checking
Arc-consistency: every combination of two adjacent variables
3-consistency, k-consistency (k  n)
Constraint filtering, constraint checking, etc..
Eliminate non-acceptable tuples prior to search
Warning: arc-consistency does not solve the problem
still is not a solution!
5/1/2019
Lecture slides 1

43. Systematic search Starting from a root node
Consider all values for a variable V1
For every value for V1, consider all values for V2
etc..
For n variables, each of domain size d
Maximum depth? fixed!
Maximum number of paths? size of search space, size of CSP
5/1/2019
Lecture slides 1

44. Systematic search (I) Back-checking
Systematic search generates dn possibilities
Are all possibilities acceptable?
Expand a partial solution only when it is consistent
This yields early pruning of inconsistent paths
5/1/2019
Lecture slides 1

45. Systematic search (II) Chronological backtracking
What if only one solution is needed?
Depth-first search & Chronological backtracking
DFS: Soundness? Completeness?
5/1/2019
Lecture slides 1

46. Systematic search (III) Intelligent backtracking
What if the reason for failure was higher up in the tree?
Backtrack to source of conflict !!
Backjumping, conflict-directed backjumping, etc.
5/1/2019
Lecture slides 1

47. Systematic search (IV) Ordering heuristics
Which variable to expand first?
Heuristics:
most constrained variable first (reduce branching factor)
most promising value first (find quickly first solution)
5/1/2019
Lecture slides 1

48. Systematic search (V) Back-checking
Search tree with only backtrack search?
Root node 1 Q 2 Q 4 Q 1 2 3 3 Q 1 2 4 Q 1 Q 2 Q 1 2 3 4 1 3 4 3 Q 1 2 3 4 1 2
Solution!
26 nodes visited.
5/1/2019
Lecture slides 1

49. Systematic search (VI) Forward checking
Search Tree with domains filter by Forward Check
Root node 2 Q 1 Q
Domain Wipe Out V4 4 Q 3 Q Q 4
Domain Wipe Out V3 1 Q 2 Q 3 Q
Solution!
8 nodes visited.
5/1/2019
Lecture slides 1

50. Summary of backtrack search
Constructive, systematic, exhaustive
In general sound and complete
Back-checking: expands nodes consistent with past
Backtracking: Chronological vs. intelligent
Ordering heuristics:
Static
Dynamic variable
Dynamic variable-value pairs
Looking ahead:
Partial look-ahead
Forward checking (FC)
Directional arc-consistency (DAC)
Full (a.k.a. Maintaining Arc-consistency or MAC)
5/1/2019
Lecture slides 1

51. CSP: a decision problem (NP-complete)
Modeling: abstraction and reformulation
Preprocessing techniques:
eliminate non-acceptable tuples prior to search
Systematic search:
potentially dn paths of fixed lengths
chronological backtracking
intelligent backtracking
variable/value ordering heuristics
Search ‘hybrids:’
Mixing consistency checking with search: look-ahead strategies
5/1/2019
Lecture slides 1

52. Non-systematic search
Iterative repair, local search: modifies a global but inconsistent solution to decrease the number of violated constraints
Examples: Hill climbing, taboo search, simulated annealing, GSAT, WalkSAT, Genetic Algorithms, Swarm intelligence, etc.
Features: Incomplete & not sound
Advantage: anytime algorithm
Shortcoming: cannot tell that no solution exists
5/1/2019
Lecture slides 1

53. Outline of CSP 101 We have seen We will move to
Motivating example, application areas
CSP: Definition, representation
Some simple modeling examples
More on definition and formal characterization
Basic solving techniques
We will move to
(Implementing backtrack search)
Advanced solving techniques
Issues & research directions
5/1/2019
Lecture slides 1