Presentation is loading. Please wait.

Presentation is loading. Please wait.

CS B551: E LEMENTS OF A RTIFICIAL I NTELLIGENCE Instructor: Kris Hauser 1.

Similar presentations


Presentation on theme: "CS B551: E LEMENTS OF A RTIFICIAL I NTELLIGENCE Instructor: Kris Hauser 1."— Presentation transcript:

1 CS B551: E LEMENTS OF A RTIFICIAL I NTELLIGENCE Instructor: Kris Hauser http://cs.indiana.edu/~hauserk 1

2 C ONSTRAINT P ROPAGATION … … is the process of determining how the constraints and the possible values of one variable affect the possible values of other variables It is an important form of “least-commitment” reasoning 2 2

3 F ORWARD C HECKING 3 Whenever a pair (X  v) is added to assignment A do: For each variable Y not in A do: For every constraint C relating Y to the variables in A do: Remove all values from Y’s domain that do not satisfy C  n = number of variables  d = size of initial domains  s = maximum number of constraints involving a given variable (s  n-1)  Forward checking takes O(nsd) time

4 F ORWARD C HECKING IN M AP C OLORING 4 WANTQNSWVSAT RGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGB RRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGB RGBGBGRGBRGBRGBRGBGBGBRGBRGB RBGRBRBBBRGBRGB Empty set: the current assignment {(WA  R), (Q  G), (V  B)} does not lead to a solution 4

5 F ORWARD C HECKING IN M AP C OLORING 5 T WA NT SA Q NSW V Contradiction that forward checking did not detect WANTQNSWVSAT RGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGB RRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGB RGBGBGRGBRGBRGBRGBGBGBRGBRGB RBGRBRBBBRGBRGB 5

6 F ORWARD C HECKING IN M AP C OLORING 6 T WA NT SA Q NSW V Contradiction that forward checking did not detect WANTQNSWVSAT RGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGB RRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGBRGB RGBGBGRGBRGBRGBRGBGBGBRGBRGB RBGRBRBBBRGBRGB Detecting this contradiction requires a more powerful constraint propagation technique 6

7 C ONSTRAINT P ROPAGATION FOR B INARY C ONSTRAINTS REMOVE-VALUES(X,Y) 1. removed  false 2. For every value v in the domain of Y do – If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then a. Remove v from Y‘s domain b. removed  true 3. Return removed 7

8 C ONSTRAINT P ROPAGATION FOR B INARY C ONSTRAINTS 8 AC3 1. Initialize queue Q with all variables (not yet instantiated) 2. While Q   do a. X  Remove(Q) b. For every (not yet instantiated) variable Y related to X by a (binary) constraint do – If REMOVE-VALUES(X,Y) then i. If Y’s domain =  then exit ii. Insert(Y,Q)

9 E DGE L ABELING 9 We consider an image of a scene composed of polyhedral objects such that each vertex is the endpoint of exactly three edges 9

10 E DGE L ABELING 10 An “edge extractor” has accurately extracted all the visible edges in the image. The problem is to label each edge as convex (+), concave (-), or occluding (  ) such that the complete labeling is physically possible 10

11 11 Convex edges Concave edges Occluding edges 11

12 12 + - - + + + + + The arrow is oriented such that the object is on the right of the occluding edge 12

13 O NE P OSSIBLE E DGE L ABELING 13 + + + + + + + + + + - -

14 J UNCTION T YPES 14 Fork L T Y 14

15 J UNCTION L ABEL S ETS 15 ++ - - - -- ++ ++ + + + - - - - - + (Waltz, 1975; Mackworth, 1977) 15

16 E DGE L ABELING AS A CSP A variable is associated with each junction The domain of a variable is the label set associated with the junction type Constraints: The values assigned to two adjacent junctions must give the same label to the joining edge 16

17 AC3 A PPLIED TO E DGE L ABELING 17 Q = (X 1, X 2, X 3,...) X1X1 X5X5 X3X3 X8X8 X 12 X2X2 X4X4 17

18 18 + - + - + - - + + X1X1 X5X5 Q = (X 1,...) 18 AC3 A PPLIED TO E DGE L ABELING

19 19 + - + - + - - + + X1X1 X5X5 Q = (X 1,...) 19

20 20 + - + - + - - + + X5X5 Q = (X 5,...) 20

21 21 + + + + - - - - - - + Q = (X 5,...) X5X5 X3X3 21

22 22 + + + + - - - - - - + Q = (X 5,...) X5X5 X3X3 22

23 23 + + + + - - - - - - + Q = (X 3,...) X3X3 23

24 24 + + + + + - -- ++ ++ Q = (X 3,...) X3X3 X8X8 + 24

25 25 + + + + + - -- ++ ++ Q = (X 3,...) X3X3 X8X8 + 25

26 26 + + + + + - -- ++ ++ Q = (X 8,...) X8X8 + 26

27 27 + + -- ++ ++ - - + X 12 X8X8 Q = (X 8,...) 27

28 C OMPLEXITY A NALYSIS OF AC3 n = number of variables d = size of initial domains s = maximum number of constraints involving a given variable (s  n-1) Each variables is inserted in Q up to d times REMOVE-VALUES takes O(d 2 ) time AC3 takes O(n  d  s  d 2 ) = O(n  s  d 3 ) time Usually more expensive than forward checking 28 AC3 1.Initialize queue Q with all variables (not yet instantiated) 2.While Q   do a.X  Remove(Q) b.For every (not yet instantiated) variable Y related to X by a (binary) constraint do – If REMOVE-VALUES(X,Y) then i.If Y’s domain =  then exit ii.Insert(Y,Q) REMOVE-VALUES(X,Y) 1. removed  false 2. For every value v in the domain of Y do – If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then a. Remove v from Y‘s domain b. removed  true 3. Return removed

29 I S AC3 ALL THAT WE NEED ? No !! AC3 can’t detect all contradictions among binary constraints 29 X Z Y XYXY XZXZ YZYZ {1, 2} 29

30 I S AC3 ALL THAT WE NEED ? No !! AC3 can’t detect all contradictions among binary constraints 30 X Z Y XYXY XZXZ YZYZ {1, 2} 30 REMOVE-VALUES(X,Y) 1. removed  false 2. For every value v in the domain of Y do – If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then a. Remove v from Y‘s domain b. removed  true 3. Return removed

31 I S AC3 ALL THAT WE NEED ? No !! AC3 can’t detect all contradictions among binary constraints 31 X Z Y XYXY XZXZ YZYZ {1, 2} 31 REMOVE-VALUES(X,Y) 1. removed  false 2. For every value v in the domain of Y do – If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then a. Remove v from Y‘s domain b. removed  true 3. Return removed REMOVE-VALUES(X,Y,Z) 1. removed  false 2. For every value w in the domain of Z do – If there is no pair (u,v) of values in the domains of X and Y verifying the constraint on (X,Y) such that the constraints on (X,Z) and (Y,Z) are satisfied then a. Remove w from Z‘s domain b. removed  true 3. Return removed

32 I S AC3 ALL THAT WE NEED ? No !! AC3 can’t detect all contradictions among binary constraints Not all constraints are binary 32 X Z Y XYXY XZXZ YZYZ {1, 2} 32

33 T RADEOFF Generalizing the constraint propagation algorithm increases its time complexity  Tradeoff between time spent in backtracking search and time spent in constraint propagation A good tradeoff when all or most constraints are binary is often to combine backtracking with forward checking and/or AC3 (with REMOVE- VALUES for two variables) 33

34 M ODIFIED B ACKTRACKING A LGORITHM WITH AC3 34 CSP-BACKTRACKING(A, var-domains) 1.If assignment A is complete then return A 2.Run AC3 and update var-domains accordingly 3.If a variable has an empty domain then return failure 4.X  select a variable not in A 5.D  select an ordering on the domain of X 6.For each value v in D do a.Add (X  v) to A b.var-domains  forward checking(var-domains, X, v, A) c.If no variable has an empty domain then (i) result  CSP-BACKTRACKING(A, var-domains) (ii) If result  failure then return result d.Remove (X  v) from A 7.Return failure

35 A C OMPLETE E XAMPLE : 4-Q UEENS P ROBLEM 35 1 3 2 4 3241 X 1 {1,2,3,4} X 3 {1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 1) The modified backtracking algorithm starts by calling AC3, which removes no value 35

36 4-Q UEENS P ROBLEM 36 1 3 2 4 3241 X 1 {1,2,3,4} X 3 {1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 2)The backtracking algorithm then selects a variable and a value for this variable. No heuristic helps in this selection. X 1 and the value 1 are arbitrarily selected 36

37 4-Q UEENS P ROBLEM 37 1 3 2 4 3241 X 1 {1,2,3,4} X 3 {1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 3)The algorithm performs forward checking, which eliminates 2 values in each other variable’s domain

38 4-Q UEENS P ROBLEM 38 1 3 2 4 3241 X 1 {1,2,3,4} X 3 {1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 4)The algorithm calls AC3 38

39 4-Q UEENS P ROBLEM 39 1 3 2 4 3241 X 1 {1,2,3,4} X 3 {1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 4)The algorithm calls AC3, which eliminates 3 from the domain of X 2 X 2 = 3 is incompatible with any of the remaining values of X 3 39 REMOVE-VALUES(X,Y) 1. removed  false 2. For every value v in the domain of Y do – If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then a. Remove v from Y‘s domain b. removed  true 3. Return removed

40 4-Q UEENS P ROBLEM 40 1 3 2 4 3241 X 1 {1,2,3,4} X 3 {1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 4)The algorithm calls AC3, which eliminates 3 from the domain of X 2, and 2 from the domain of X 3 40

41 4-Q UEENS P ROBLEM 41 1 3 2 4 3241 X 1 {1,2,3,4} X3{1,2,3,4}X3{1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 4)The algorithm calls AC3, which eliminates 3 from the domain of X 2, and 2 from the domain of X 3, and 4 from the domain of X 3 41

42 4-Q UEENS P ROBLEM 42 1 3 2 4 3241 X 1 {1,2,3,4} X3{1,2,3,4}X3{1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 5)The domain of X 3 is empty  backtracking 42

43 4-Q UEENS P ROBLEM 43 1 3 2 4 3241 X 1 {1,2,3,4} X 3 {1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 6)The algorithm removes 1 from X 1 ’s domain and assign 2 to X 1 43

44 4-Q UEENS P ROBLEM 44 1 3 2 4 3241 X 1 {1,2,3,4} X 3 {1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 7)The algorithm performs forward checking 44

45 4-Q UEENS P ROBLEM 45 1 3 2 4 3241 X 1 {1,2,3,4} X 3 {1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 8)The algorithm calls AC3 45

46 4-Q UEENS P ROBLEM 46 1 3 2 4 3241 X 1 {1,2,3,4} X 3 {1,2,3,4} X 4 {1,2,3,4} X 2 {1,2,3,4} 8)The algorithm calls AC3, which reduces the domains of X 3 and X 4 to a single value 46

47 E XPLOITING THE S TRUCTURE OF CSP If the constraint graph contains several components, then solve one independent CSP per component 47 T WA NT SA Q NSW V 47

48 E XPLOITING THE S TRUCTURE OF CSP If the constraint graph is a tree, then : 48 1.Order the variables from the root to the leaves  (X 1, X 2, …, X n ) 2.For j = n, n-1, …, 2 call REMOVE-VALUES(X j, X i ) where X i is the parent of X j 3.Assign any valid value to X 1 4.For j = 2, …, n do Assign any value to X j consistent with the value assigned to its parent X i X YZ U V W  (X, Y, Z, U, V, W) 48

49 E XPLOITING THE S TRUCTURE OF CSP Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph 49 WA NT SA Q NSW V 49

50 E XPLOITING THE S TRUCTURE OF CSP Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph 50 WA NT Q NSW V If the graph becomes a tree, then proceed as shown in previous slide 50

51 51 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation Variables X ij for i, j in {1,..,9} Domains {1,…,9} Constraints: X ij  X ik, for j  k X ij  X kj, for i  k X ij  X mn, for (i,j), (m,n) in same cell

52 52 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation Variables X ij for i, j in {1,..,9} Domains {1,…,9} Constraints: X ij  X ik, for j  k X ij  X kj, for i  k X ij  X mn, for (i,j), (m,n) in same cell

53 53 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation Variables X ij for i, j in {1,..,9} Domains {1,…,9} Constraints: X ij  X ik, for j  k X ij  X kj, for i  k X ij  X mn, for (i,j), (m,n) in same cell Can we detect this using constraint propagation?

54 54 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation Variables X ij for i, j in {1,..,9} Domains {1,…,9} Constraints: X ij  X ik, for j  k X ij  X kj, for i  k X ij  X mn, for (i,j), (m,n) in same cell Must detect 9- way interactions

55 55 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation 2 Variables R i j in {1,…,9} C i j in {1,…,9} X i j in {(1,1),…,(3,3)} Constraints: ??? R 1 8 =3 R 2 2 =8 R 3 2 =6 R 3 7 =9 … C 1 5 =4 C 1 2 =9 C 2 8 =4C 3 8 =1 C 3 3 =8 C 3 9 =5 …

56 56 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation 2 Variables R i j in {1,…,9} C i j in {1,…,9} X i j in {(1,1),…,(3,3)} Constraints: ??? R 1 8 =3 R 2 2 =8 R 3 2 =6 R 3 7 =9 … C 1 5 =4 C 1 2 =9 C 2 8 =4C 3 8 =1 C 3 3 =8 C 3 9 =5 … X 1 8 = (1,3)

57 57 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation 2 Variables R i j in {1,…,9} C i j in {1,…,9} X i j in {(1,1),…,(3,3)} Constraints: ??? R 1 8 =3 R 2 2 =8 R 3 2 =6 R 3 7 =9 … C 1 5 =4 C 1 2 =9 C 2 8 =4C 3 8 =1 C 3 3 =8 C 3 9 =5 … X 1 8 = (1,3)X 2 2 = (3,3)

58 58 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation 2 Variables R i j in {1,…,9} C i j in {1,…,9} X i j in {(1,1),…,(3,3)} Constraints: ??? R 1 8 =3 R 2 2 =8 R 3 2 =6 R 3 7 =9 … C 1 5 =4 C 1 2 =9 C 2 8 =4C 3 8 =1 C 3 3 =8 C 3 9 =5 … X 1 8 = (1,3)X 2 2 = (3,3) X 3 2 = (2,3) X 3 7 = (3,3)

59 59 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation 2 Variables R i j in {1,…,9} C i j in {1,…,9} X i j in {(1,1),…,(3,3)} Constraints: R i j =k  C k j =I Similar constraints between X’s and R’s, X’s and C’s R i j  R i k for j  k C i j  C i k for j  k X i j  X i k for j  k R 1 8 =3 R 2 2 =8 R 3 2 =6 R 3 7 =9 … C 1 5 =4 C 1 2 =9 C 2 8 =4C 3 8 =1 C 3 3 =8 C 3 9 =5 … X 1 8 = (1,3)X 2 2 = (3,3) X 3 2 = (2,3) X 3 7 = (3,3)

60 60 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation 2 Variables R i j in {1,…,9} C i j in {1,…,9} X i j in {(1,1),…,(3,3)} Constraints: R i j =k  C k j =I Similar constraints between X’s and R’s, X’s and C’s R i j  R i k for j  k C i j  C i k for j  k X i j  X i k for j  k C 1 2 = 9 R 1 2 = {1-9}

61 61 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation 2 Variables R i j in {1,…,9} C i j in {1,…,9} X i j in {(1,1),…,(3,3)} Constraints: R i j =k  C k j =i Similar constraints between X’s and R’s, X’s and C’s R i j  R i k for j  k C i j  C i k for j  k X i j  X i k for j  k C 2 2 = {1-9}C 1 2 = 9 R 1 2 = {2-9}

62 62 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation 2 Variables R i j in {1,…,9} C i j in {1,…,9} X i j in {(1,1),…,(3,3)} Constraints: R i j =k  C k j =I Similar constraints between X’s and R’s, X’s and C’s R i j  R i k for j  k C i j  C i k for j  k X i j  X i k for j  k C 1 2 = 9 R 1 2 = {2-9} X 2 2 = (3,3) X 3 2 = (2,2)

63 63 8 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation 2 Variables R i j in {1,…,9} C i j in {1,…,9} X i j in {(1,1),…,(3,3)} Constraints: R i j =k  C k j =I Similar constraints between X’s and R’s, X’s and C’s R i j  R i k for j  k C i j  C i k for j  k X i j  X i k for j  k C 1 2 = 9 R 1 2 =2 X 2 2 = (3,3) X 3 2 = (2,2)

64 64 28 58 9 3 2 2 9 5 6 9 2 7 7 4 Representation 2 Variables R i j in {1,…,9} C i j in {1,…,9} X i j in {(1,1),…,(3,3)} Constraints: R i j =k  C k j =I Similar constraints between X’s and R’s, X’s and C’s R i j  R i k for j  k C i j  C i k for j  k X i j  X i k for j  k C 1 2 = 9 R 1 2 =2 X 2 2 = (3,3) X 3 2 = (2,2)

65 L OCAL S EARCH FOR CSP S Init: Make an arbitrary assignment Repeat: Modify some variable to reduce # of violated constraints 65

66 B OOLEAN S ATISFIABILITY P ROBLEMS Highly successful local search algorithms WalkSAT See R&N 7.3 66 p constraints of form u i *  u j *  u k *= 1 where u i * is either u i or  u i n variables u i, …, u n

67 O BSERVATIONS … If a CSP has few constraints, local search solves it quickly Random starting assignment not too far from a solution Million-queens puzzles solved in < 1min, c. 1990 If a CSP has many constraints, local search solves it quickly Constraints “guide” solver to a solution (if one exists) 67

68 H ARD S UDOKU ’ S 68 1.. |... |.. 2. 9. | 4.. |. 5... 6 |... | 7.. ------+-------+------. 5. | 9. 3 |...... |. 7. |...... | 8 5. |. 4. ------+-------+------ 7.. |... | 6... 3. |.. 9 |. 8... 2 |... |.. 1 Human solvers: Lot of logic (deep constraint propagation) Computer solvers: Lot of backtracking

69 H ARD & E ASY 3-SAT P ROBLEMS Let R = # of constraints / # of variables As n , the fraction of hard problems reduces to 0 69

70 R ECAP Constraint propagation, AC3 Taking advantage of CSP structure Local search for CSPs 70

71 N EXT C LASS Intro to uncertainty R&N 4.3-4, 13.1-2 71


Download ppt "CS B551: E LEMENTS OF A RTIFICIAL I NTELLIGENCE Instructor: Kris Hauser 1."

Similar presentations


Ads by Google