1. Principles of Intelligent Systems Constraint Satisfaction + Local Search Written by Dr John Thornton School of IT, Griffith University Gold Coast

2. Outline q The Constraint Satisfaction Paradigm q Variables, Domains and Constraints q Example Problem Domains q Constraint Satisfaction Algorithms ê Local Search Algorithms q Summary

3. The Constraint Satisfaction Paradigm q Knowledge represented by constraints ê What is allowed or disallowed. q Transforms broad range of problems into a computable form q Standard representation means general purpose algorithms can be used ê solvers ‘know’ how to solve problems ê issue becomes how to model a problem

4. Variables, Domains and Constraints q A Constraint Satisfaction Problem (CSP) is: ê a set of variables each with ê a set of domain of values, and ê a set of constraints that define which combinations of variable values are allowed ê the task is to find a domain value for each variable such that all constraints are simultaneously satisfied

5. Problem Domains q N-Queens q Satisfiability Problems q Binary Constraint Satisfaction q Nurse Rostering q University Timetabling

6. Local Search Algorithms q Getting ‘stuck’ q Variable and Constraint-based Search q Random Walk q Tabu Search ê NOVELTY and RNOVELTY q Constraint Weighting

7. Satisfiability Problems q Conjunctive Normal Form: Example {  x  y  z}  {x  y   z}  {  x  y}  {  y  z} q To Solve: ê All clauses evaluate true ê e.g.: x = false, y = true, z = true q As a CSP: ê Each clause is a constraint, each literal is a variable with a {true, false} domain

8. Predicate to Conjunctive Form q Predicate:  (X) (dog (X)  animal (X))  Clause: (  dog(X)  animal(X)) q Clause is false only if  dog(X) is false and animal(X) is false, i.e. if X is a dog and X is not an animal - in other words if X is a dog then X is an animal

9. Binary Constraint Satisfaction q Two variable constraint representation q Non-binary to binary transformation q Phase transition: p 2crit = 1 - m -2/p1(n - 1) where p 1 = constraint density p 2 = constraint tightness m = uniform domain size n = number of variables

10. Nurse Scheduling

11. Binary Constraint Representation Variable 1 Variable 2 0 1 2 3 4 0 1 2 3

12. Random Walk Local Search q if randomly generated probability < p: ê Take best cost local move q else: ê Take randomly selected local move

13. Tabu Search q if cost(best local move) < best cost yet: ê Take best local move q else ê Take best cost local move from the set of domain values that have not been used in the last n moves

14. The Eight Queens Problem Variable Constraint Domain

15. Constraint Satisfaction Algorithms q Backtracking ê Algorithm ê 8-Queens Example ê 8-Queens Performance q Local Search ê Algorithm ê 8-Queens Example ê 8-Queens Performance

16. Summary q Constraint Satisfaction ê practical knowledge representation ê allows general purpose approaches q Algorithms ê Backtracking ê Local Search

17. Backtracking Algorithm V = list of all variables v i (i = 1 to n) U = empty TryVariable() select next v i from V and move to U for each domain value d i of v i if d i satisfies all constraints with U Value(v i ) = d i if(V is empty) SaveSolution() else TryVariable() move v i from U back to V

18. Basic Local Search Algorithm assign a domain value d i to each variable v i while no solution and not stuck and not timed out bestCost   ;bestList   ; for each variable v i | Cost(Value(v i )) > 0 for each domain value d i of v i if Cost(d i ) < bestCost bestCost  Cost(d i ); bestList  d i ; else if Cost(d i ) = bestCost bestList  bestList  d i Take a randomly selected move from bestList

19. Backtracking Performance

20. Local Search Performance

21. Local Search Diagram

22. Eight Queens using Backtracking Try Queen 1 Try Queen 2Try Queen 3Try Queen 4Try Queen 5 Stuck! Undo move for Queen 5 Try next value for Queen 5 Still Stuck Undo move for Queen 5 no move left Backtrack and undo last move for Queen 4 Try next value for Queen 4 Try Queen 5Try Queen 6 Try Queen 7 Stuck Again Undo move for Queen 7 and so on...

23. Place 8 Queens randomly on the board Eight Queens using Local Search Pick a Queen: Calculate cost of each move 31054111 Take least cost move then try another Queen 04441111 434111131 333211112 344111132 223422212 323211231 204212231 232213231 233212221 232322131 223213211 322333301 Answer Found

24. Variable/Constraint-based Search q Variable-based search: ê Randomly select variable in constraint violation ê Try all domain values for that variable q Constraint-based search: ê Randomly select a violated constraint ê Try all domain values for all variables in the constraint that improve the constraint

25. NOVELTY q select best and second best cost moves that improve a violated constraint q if best cost move not most recent or p >= random value ê select best cost move q else ê select second best cost move

26. RNOVELTY q Same as NOVELTY except when best cost move is also most recent, then : q if second best cost - best cost <= c or p < random value ê select second best cost move q else ê select best cost move

27. Constraint Weighting q Add weight to all constraints violated at a local minimum: graph colouring example q Use these weights to calculate cost of subsequent moves q Adding weights changes shape of cost surface: “fills in” the minimum

28. Constraint Weighting Example q Graph Colouring a green b green c red d green c ab = 2w c ac = 0 c cd = 0 c bd = w c bc = 0 a green b red c red d green c ab = 0 c ac = 0 c cd = 0 c bd = 0 c bc = w a green b green c red d red c ab = w c ac = 0 c cd = w c bd = 0 c bc = 0 (a) Local minimum, cost 2w (b) After Weighting, cost 3w (c) Final solution, cost w