Towards theoretical frameworks for comparing constraint satisfaction models and algorithms Peter van Beek, University of Waterloo CP 2001 · Paphos, Cyprus November 2001

2 Acknowledgements Joint work with: Fahiem Bacchus Xinguang Chen Grzegorz Kondrak Toby Walsh

3 Constraint programming methodology Model problem ·specify in terms of constraints on acceptable solutions ·define/choose constraint model: variables, domains, constraints Solve model ·define/choose algorithm ·define/choose heuristics Verify and analyze solution

4 Previous work Many models, algorithms, and heuristics proposed Choice of model, algorithm, and heuristic can greatly influence efficiency ( e.g., Nadel, 1990; Ginsberg et al., 1990; Frost and Dechter, 1994; Tsang et al., 1995 Smith et al., 2000; Beacham et al., 2001)

5 How to compare? Empirical studies ·benchmark sets ·random problems Theoretical studies ·worst case ·average case (e.g. Brown & Purdom, 1981; Nadel, 1983; Wilf, 1984) ·pair-wise comparisons -partial orders -bounded worse Performance measures ·nodes visited ·constraint checks ·CPU time

6 Part I Comparing backtracking algorithms

7 Some backtracking algorithms Chronological Backtracking (BT) Backjumping (BJ) (Gaschnig, 1978) Conflict-Directed Backjumping (CBJ) (Prosser, 1993) Forward Checking (FC) (Haralick & Elliott, 1980; McGregor, 1979) Maintaining arc consistency (MAC) (Haralick & Elliott, 1980; McGregor, 1979; Sabin & Freuder, 1994)

8 Example: 4-queens x1x1 x2x2 x3x3 x4x4 x i x j and | x i - x j | | i - j |

9 Search tree for 4-queens x1x1 x2x2 x3x3 x4x (1,1,1,1)(4,4,4,4)(2,4,1,3)(3,1,4,2)

{x 1 1} Backjumping Q Q {x 1 1, x 2 3} x1x1 x2x2 x1x1 x2x2 {x 1 1, x 2 3, x 4 2} Q x3x3 x1x1 x3x3 x4x4 x2x2

11 New algorithms: BJ 1, …, BJ n BJ k : ·allowed to backjump at most k times from a leaf ·after that it must chronologically backtrack Special cases: ·BJ 1 equivalent to BJ ·BJ n equivalent to CBJ

12 k-consistency Given ·any k-1 distinct variables, ·any consistent assignment to those variables, ·any additional variable x there exists a consistent assignment to x x1x1 x2x2 x3x3 x4x4 2-consistent ? Q Q 1-consistent ? 3-consistent ? strongly 2-consistent yes no

13 Maintaining consistency { x 1 1, x 2 4 } x1x1 x2x2 x3x3 x4x4 Q Q x 3 x 4 |x 3 - x 4 | 1 Binary constraints Unary constraints x 3 1, 3, 4 x 4 1, 2, 4 Induced CSP

14 Maintaining consistency x1x1 x2x2 x3x3 x4x4 Q Q { x 1 1, x 2 4 } x 3 x 4 |x 3 - x 4 | 1 Binary constraints Unary constraints x 3 1, 3, 4 x 4 1, 2, 4 FC: strong 1-consistency

15 Maintaining consistency Q Q { x 1 1, x 2 4 } x 3 x 4 |x 3 - x 4 | 1 Binary constraints Unary constraints x 3 1, 3, 4 x 4 1, 2, x1x1 x2x2 x3x3 x4x4 MAC: strong 2-consistency Q Q

16 New algorithms: MC 1, …, MC n MC k : ·maintains strong k-consistency on induced CSP at each node in backtrack tree Special cases: ·MC 1 equivalent to FC ·MC 2 equivalent to MAC for binary CSPs

17 Comparing backtracking algorithms Proposal: for all CSP models for all variable orderings algorithm a 1 algorithm a 2 Desirable features: ·still holds if use best possible CSP model for a 2 ·still holds if use best possible variable ordering for a 2

18 Theoretical methodology Using local consistency concepts: ·formulate necessary and sufficient conditions for a node to be visited by a backtracking algorithm Using conditions: ·construct partial orders of the algorithms

x1x1 x2x2 x3x3 x4x4 {x 1 1} {x 1 1, x 2 4} ? Example sufficient condition: FC If a node is consistent and its parent is strongly 1-consistent, then FC visits the node Q Q

20 Example necessary condition: MAC-CBJ If MAC-CBJ visits a node, then it is consistent and its parent is strongly 2-consistent x1x1 x2x2 x3x3 x4x4 {x 1 1} {x 1 1, x 2 4} ? Q

21 Example conclusion: MAC-CBJ vs FC FC visits p MAC-CBJ visits p p is consistent and parent(p) strongly 1-consistent p is consistent and parent(p) strongly 2-consistent

22 Partial order for nodes visited MC k+1 -CBJMC k -CBJMC 1 -CBJMC n -CBJMC k+1 MC k MC 1 MC n BJ k+1 BJ k BJ 1 BJ n BT less than or equal

23 Partial order for nodes visited MC k+1 -CBJMC k -CBJMC 1 -CBJMC n -CBJMC k+1 MC k MC 1 MC n BJ k+1 BJ k BJ 1 BJ n BT incomparable

24 Partial order for nodes visited BT MC k+1 -CBJ MC k -CBJ MC 1 -CBJ MC n -CBJ MC k+1 MC k MC 1 MC n BJ k+1 BJ k BJ 1 BJ n incomparable

25 Partial order for nodes visited MC k+1 -CBJ MC k -CBJ MC 1 -CBJ MC n -CBJ MC k+1 MC k MC 1 MC n BJ k+1 BJ k BJ 1 BJ n BT incomparable

26 Partial order for nodes visited MC k+1 -CBJMC k -CBJMC 1 -CBJMC n -CBJMC k+1 MC k MC 1 MC n CBJBT

27 Closer look: nodes visited, binary CSPs MAC-CBJFC-CBJMACFCCBJBJ 2 BJ BT less than or equal

28 Closer look: constraint checks, binary CSPs MAC-CBJFC-CBJMACFCCBJ BJ 2 BJ BT less than or equal at most O(f) more O(nd) O(n2d2)O(n2d2)

29 Experiments: lookahead FC MAC O(nd) Path BT O(nd)

30 Example: FC vs MAC Adapted from J. Ullman (1988); suggested by D. De Haan Odd-even relation variables: x 1, …, x n, n odd domains: {1, 2, 3, 4} x1x1 x2x2 xnxn... FC: 2 n MAC: 4

31 Increasing lookahead Time (sec.) to solve 6 x 6 puzzles MC 4 MC 3 MC 2 MC 1

32 Experiments: lookback FC-CBJFC

33 Experiments: lookahead vs lookback Increasing the level of consistency decreases effectiveness of CBJ ·Prosser (1993) ·Bacchus & van Run (1995) ·Bessiere & Regin (1996) MAC much better than FC-CBJ MAC better than MAC-CBJ MAC-CBJMAC FC-CBJFC O(nd) CBJBT O(nd) CBJ becomes useless ·Jussien, Debruyne, Boizumault (2000) MAC-DBT much better than MAC (binary) ·Chen & van Beek (2001) MGAC-CBJ much better than MAC (non-binary)

34 Theory: lookahead vs lookback MC k+1 -CBJ MC k -CBJ MC k+1 MC k BJ k+1 BJ k BJ k+1 better only if there exists a (k+1)-level backjump that is not a chronological backtrack Number of (k+1)-level backjumps number of k-level backjumps Therefore, as k increases, effectiveness of backjumping decreases

35 Part II Comparing CSP models

36 Conversion to binary Any non-binary CSP can be converted into one with only binary constraints ·dual graph method (Dechter & Pearl, 1989; Rossi et al. 1989) ·hidden variable method (Peirce, 1933; Rossi et al. 1989; Dechter 1990)

37 Crossword puzzles a aardvark aback abacus abaft abalone abandon... Mona Lisa monarch monarchy monarda... zymurgy zyrian zythum

38 Non-binary model (original) variables: one for each unknown letter (cell) domains: a, …, z constraints: contiguous letters must form words in dictionary

39 Dual model (binary) variables: one for each unknown word across and down domains: words from dictionary constraints intersecting words must agree on common letter

40 Hidden model (binary) variables: one for each unknown letter (cell) and one for each unknown word across and down domains: letters, words constraints: letter and word variable must agree

41 Comparing models One proposal: for all backtracking algorithms for all variable orderings model m 1 model m 2 Objections: ·effectiveness of a model is algorithm dependent ·models may contain different variables Desirable features: ·meets objections ·still holds if use best possible variable ordering for m 2 Another proposal: given backtrack algorithm for all v 2 for m 2 there exists a v 1 for m 1 s. t. model m 1 model m 2

42 Theoretical methodology Bound difference in number of nodes visited ·compare pruning power of local consistencies of alternative models ( e.g., Debruyne & Bessiere, 1997; Stergiou & Walsh, 1999; Prosser et al., 2000; Walsh, 2000) ·establish correspondence between nodes Bound difference in cost at each node: ·local consistency ·variable ordering

43 Some relationships MGAC-dual MGAC-hidden MGAC-orig FC-dual FC-hidden FC-orig BT-dual BT-hidden BT-orig polynomialnot polynomial

44 Part III Comparing variable ordering heuristics Future work

45 Conclusion Technical: ·comparing backtracking algorithms partial order ·comparing CSP models bounded worse relation on algorithm-model combinations Big picture: ·theory can guide, inform experimentation