1. From AND/OR Search to AND/OR BDDs
Rina Dechter
Information and Computer Science, UC-Irvine, and
Radcliffe Institue of Advanced Study, Cambridge
Joint work with Robert Mateescu and John Cobb
Verification/constraints workshop, 2006

2. Combining Decision Diagrams
f = A*E + B*F
g = C*G +D*H B A F 1 E D C H 1 G B A C D 1 H G F E
OBDD f * g =

3. Combining AND/OR BDDs f = A*E + B*F g = C*G +D*H AND AOBDD f * g = B A1 E D C H 1 G B A F 1 E D C H G
AND
AOBDD f * g =

4. OBDDs vs AOBDD f * g = AND AOBDD OBDD B A F 1 E D C H G B A C D 1 H G1 E D C H G
AND
AOBDD B A C D 1 H G F E
OBDD

5. Outline Background in Graphical models AND/OR search trees and Graphs
Minimal AND/OR graphs
From AND/OR graphs to AOMDDs
Compilation of AOMDDs
AOMDDs and earlier BDDs

6. Constraint Networks A Example: map coloring
Variables - countries (A,B,C,etc.)
Values - colors (red, green, blue)
Constraints:
A B
red green
red yellow
green red
green yellow
yellow green
yellow red
Constraint graph C A B D E F G A E D F B G C
Semantics: set of all solutions
Primary task: find a solution

7. Propositional Satisfiability
 = {(¬C), (A v B v C), (¬A v B v E), (¬B v C v D)}.

8. Graphical models A D B C E F A graphical model (X,D,C):
X = {X1,…Xn} variables
D = {D1, … Dn} domains
C = {F1,…,Ft} functions (constraints, CPTS, cnfs)
Examples:
Constraint networks
Belief networks
Cost networks
Markov random fields
Influence diagrams
Path width
All these tasks are NP-hard
 identify special cases
 approximate

9. Tree-solving is easy Belief updating (sum-prod) MPE (max-prod)
CSP – consistency (projection-join)
Belief updating (sum-prod)
P(X)
P(Y|X)
P(Z|X)
P(T|Y)
P(R|Y)
P(L|Z)
P(M|Z)
MPE (max-prod)
#CSP (sum-prod)
Trees are processed in linear time and memory

10. Transforming into a Tree
By Inference (thinking)
Transform into a single, equivalent tree of sub-problems
By Conditioning (guessing)
Transform into many tree-like sub-problems.

11. Inference and Treewidth
ABC
DGF A D G B
BDEF F C
EFH E K M H L
FHK J
Inference algorithm:
Time: exp(tree-width)
Space: exp(tree-width) HJ
KLM
treewidth = = 3
treewidth = (maximum cluster size) - 1

12. Conditioning and Cycle cutsetP J A L B E D F M O H K G N C P J A L B E D F M O H K G N C P J A L B E D F M O H K G N C P J L B E D F M O H K G N C P J L B E D F M O H K G N A
Cycle cutset = {A,B,C} B P J L E D F M O H K G N C P J L E D F M O H K G N C P J L E D F M O H K G N C

13. Search over the Cutset Inference may require too much memory GraphB K G L D F H M J E
Graph
Coloring
problem
Inference may require too much memory
Condition (guessing) on some of the variables
A=yellow
A=green C B K G L D F H M J E C B K G L D F H M J E

14. Search over the Cutset (cont)B K G L D F H M J E
Graph
Coloring
problem
Inference may require too much memory
Condition on some of the variables
A=yellow
A=green
B=red
B=blue
B=green
B=red
B=yellow
B=blue C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E

15. Inference vs. Conditioning
By Inference (thinking)
Exponential in treewidth
Time and memory E K F L H C B A M G J D
ABC
BDEF
DGF
EFH
FHK HJ
KLM
Variable-elimination
Directional resolution
Join-tree clustering
By Conditioning (guessing)
Exponential in cycle-cutset
Time-wise, linear memory
A=yellow
A=green
B=blue
B=red
B=green C K G L D F H M J E A B
Backtracking
Branch and bound
Depth-first search

16. Outline Background in Graphical models AND/OR search trees and Graphs
Minimal AND/OR graphs
From AND/OR graphs to AOMDDs
Compilation of AOMDDs
AOMDDs and earlier BDDs

17. Classic OR Search SpaceD B C E F
Ordering: A B E C D F E C F D B A 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

18. AND/OR Search Space Primal graph DFS tree A D B C E F A A D B C E F A
AND 1 B OR
AND 1 E OR C
AND 1 OR D F
AND 1

19. AND/OR vs. OR AND/OR AND/OR size: exp(4), OR size exp(6) OR A D B C EF A D B C E F OR A
AND 1
AND/OR OR B B
AND 1 1 OR E C E C E C E C
AND 1 1 1 1 1 1 1 1 OR D F D F D F D F D F D F D F D F
AND 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
AND/OR size: exp(4),
OR size exp(6) E 1 C F D B A E 1 C F D B A OR 1 1 1 1

20. AND/OR vs. OR with Constraints
No-goods
(A=1,B=1)
(B=0,C=0) A D B C E F A D B C E F A OR
AND 1
AND/OR OR B B
AND 1 1 OR E C E C E C E C
AND 1 1 1 1 1 1 1 1 OR D F D F D F D F D F D F D F D F
AND 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 E 1 C F D B A E 1 C F D B A E 1 C F D B A OR

21. AND/OR vs. OR with Constraints
No-goods
(A=1,B=1)
(B=0,C=0 A D B C E F A D B C E F OR A
AND 1
AND/OR OR B B
AND 1 OR E C E C E C
AND 1 1 1 1 1 1 OR D F D F D F D F
AND 1 1 1 1 1 1 1 1 A A A 1 1 1 OR B B B 1 1 1 E E E 1 1 1 1 1 1 1 1 1 C C C 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 D D D 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 F F F 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

22. DFS algorithm (#CSP example)B C E F A D B C E F
solution OR
AND B 1 E C D F 2 4 5 6 11 A B 4
OR node: Marginalization operator (summation) E 2 C 2
AND node: Combination operator (product) 1 2 1 1 1 D 2 F 1 D 2 F 1 1 1 1 1 1 1 1 1
Value of node = number of solutions below it

23. Pseudo-Trees (Freuder 85, Bayardo 95, Bodlaender and Gilbert, 91)4 1 6 2 3 2 7 5 3
m <= w* log n
(a) Graph 4 7 1 1 2 7 3 5 5 3 5 4 2 7 6 6 4 6
(b) DFS tree
depth=3
(c) pseudo- tree
depth=2
(d) Chain
depth=6

24. Tasks and value of nodes
V( n) is the value of the tree T(n) for the task:
Consistency: v(n) is 0 if T(n) inconsistent, 1 othewise.
Counting: v(n) is number of solutions in T(n)
Optimization: v(n) is the optimal solution in T(n)
Belief updating: v(n), probability of evidence in T(n).
Partition function: v(n) is the total probability in T(n).
Goal: compute the value of the root node recursively using dfs search of the AND/OR tree.
AND/OR search tree and algorithms are
([Freuder & Quinn85], [Collin, Dechter & Katz91], [Bayardo & Miranker95],[Darwiche 2001], [Bacchus et. Al, 2003])
Space: O(n)
Time: O(exp(m)), where m is the depth of the pseudo-tree
Time: O(exp(w* log n))
BFS is time and space O(exp(w* log n)

25. From AND/OR Tree A J A F B B C K E C E D D F G G H J H K OR AND OR AND1 OR B B
AND 1 1 OR E C E C E C E C
AND 1 1 1 1 1 1 1 1 OR D F D F D F D F D F D F D F D F G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K
AND OR
AND OR
AND

26. To an AND/OR Graph A J A F B B C K E C E D D F G G H J H K1 OR B B
AND 1 1 OR E C E C E C E C
AND 1 1 1 1 1 1 1 1 OR D F D F D F D F D F D F D F D F
===============================================================
AND 1 1 OR G G J J
AND 1 1 1 1 OR H H H H K K K K
AND 1 1 1 1 1 1 1 1

27. AND/OR Search Graphs
Any two nodes that root identical subtrees/subgraphs (are unifiable) can be merged
Minimal AND/OR search graph: of R relative to tree T is the closure under merge of the AND/OR search tree of R relative to T, where inconsistent subtrees are pruned.
Canonicity: The minimal AND/OR graph relative to T is unique for all constraints equivalent to R.

28. Context based caching Caching is possible when context is the same
context = parent-separator set in induced pseudo-graph = current variable + ancestors connected to subtree below A D B C E F G H J K
context(B) = {A, B}
context(C) = {A,B,C}
context(D) = {D}
context(F) = {F}

29. Caching context(B) = {A, B} context(C) = {A,B,C} context(D) = {D}J A F B B C K E C E D D F G J
context(B) = {A, B}
context(C) = {A,B,C}
context(D) = {D}
context(F) = {F} G H OR A H K
AND 1 OR B B
AND 1 1 OR E C E C E C E C
AND 1 1 1 1 1 1 1 1 OR D F D F D F D F D F D F D F D F G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K G H 1 1 G H J K 1 1 J K
AND OR
AND OR
AND

30. Caching context(D)={D} context(F)={F} A J A F B B C K E C E D D F G GOR A H K
AND 1 OR B B
AND 1 1 OR E C E C E C E C
AND 1 1 1 1 1 1 1 1 OR D F D F D F D F D F D F D F D F
AND 1 1 OR G G J J
AND 1 1 1 1 OR H H H H K K K K
AND 1 1 1 1 1 1 1 1

31. All Four Search Spaces A A B B C C D D E E F F A E C B F D1 C D F E B A 1 C D F E B A
Full OR search tree
126 nodes
Context minimal OR search graph
28 nodes A OR
AND B E F 1 C D A OR
AND B E F 1 C D
Context minimal AND/OR search graph
18 AND nodes
Full AND/OR search tree
54 AND nodes

32. Properties of minimal AND/OR graphs
Complexity:
Minimal AND/OR R relative to pseudo-tree T is O(exp(w*)) where w* is the tree-width of R along T.
Minimal OR search graph is O(exp(pw*)) where pw* is path-width
w* ≤ pw*, pw* ≤ w*log n
Canonicity:
The minimal AND/OR search graph is unique (canonical) for all equivalent formulas (Boolean or Constraints), consistent with its pseudo tree.

33. Searching AND/OR Graphs
AO(j): searches depth-first, cache i-context
j = the max size of a cache table (i.e. number of variables in a context)
i=0 j
i=w*
Space: O(n)
Time: O(exp(w* log n))
Space: O(exp w*)
Time: O(exp w*)
Space: O(exp(j) )
Time: O(exp(m_j+j )

34. Context-Based Caching
context(A) = {A}
context(B) = {B,A}
context(C) = {C,B}
context(D) = {D}
context(E) = {E,A}
context(F) = {F} A E C B F D A B
Primal graph C E D F A B E C D 1 5 2 3 6 4 7
Cache Table (C) B C
Value 5 1 2
Space: O(exp(2))

35. Outline Background in Graphical models AND/OR search trees and Graphs
Minimal AND/OR graphs
From AND/OR search graphs to AOMDDs
Compilation of AOMDDs
AOMDDs and earlier BDDs

36. OR Search Graphs vs OBDDs1 B C
Full AND/OR search tree A B C
f(ABC) 1 A B C A 1 B C
Backtrack free AND/OR search tree A 1 B C A 1 B C
An OBDD
redundant
Minimal AND/OR search graph

37. AND/OR Search Graphs; AOBDDs
F(A,B,C,D)= (0,1,1,1), (1,0,1,0), (1,1,1,0)
AOBDD(F) A 1 A 1 B B
redundancy B 1 1 1 C D D C D D 1 1 1 1

38. AND/OR Search Graphs; AOBDDs
F(A,B,C,D)= (0,1,1,1), (1,0,1,0), (1,1,1,0)
AOBDD(F) A 1 A 1 B B
redundancy B 1 1 1 C D D C D D 1 1 1 1

39. AOBDD Conventions Point dead-ends to terminal 01 B C D A 1 B C D
Point dead-ends to terminal 0
Point goods to terminal 1

40. AOBDD Conventions A B D C
Introduce
Meta-nodes A 1 B C D A 1 B C D

41. Combining AOBDD (apply)
f(ABC) 1 A A B D
g(ABC) 1 A A B B B C D C D A 1 B C A 1 B D A 1 B C D = *

42. Example:
(f+h) * (a+!h) * (a#b#g) * (f+g) * (b+f) * (a+e) * (c+e) * (c#d) * (b+c) A B C F D E G H
pseudo tree H G E D C F B A m7 m7 D C B F A E G H
primal graph A 1 B C D E m6 A 1 B F G H m3 A B C F D E G H m3 m6 A 1 B G F m2 A B F G H A 1 E C m4 A 1 F H m1 A B C D E B 1 C C9 C 1 D m5 B 1 F C5 m5 m4 m2 m1 C 1 D C8 A 1 E C6 C 1 E C7 A 1 B G C3 C E A A B F G F 1 H C1 A 1 H C2 A F H F 1 G C4 C D

43. Example (continued) m7 A m7 A B C D F G H E B m6 m3 A B C D E A B F G1 B C D F G H E m7 B A 1 B C D E m6 A 1 B F G H m3 A B C F D E G H

44. AOBDD vs. OBDD AOBDD 18 nonterminals 47 arcs OBDD 27 nonterminalsF A E G H
primal graph B B C C C C A 1 B C D F G H E D D D D D D E E E F F F F G G G G G A H H B C 1 D A B C F D E G H
AOBDD
18 nonterminals
47 arcs
OBDD
27 nonterminals
54 arcs E F G H

45. Complexity
Complexity of apply: Complexity of apply is bounded quadratically by the product of input AOBDDs restricted to each branch in the output pseudo-tree.
Complexity of VE-AOBDD: is exponential in the tree-width along the pseudo-tree.

46. AOMDDs and tree-BDDs
Tree-BDDs (McMillan1994) are :
AND/OR BDDS are

47. Related work Related work in Search Related to compilation schemes:
Backjumping + learning (Freuder 1988,Dechter 1990,Bayardo and Mirankar 1996)
Recursive-conditioning (Darwiche 2001)
Value elimination (Bacchus et. Al. 2003)
Search over tree-decompositions (Trrioux et. Al. 2002)
Related to compilation schemes:
Minimal AND/OR – related to tree-OBDDs (McMillan 94),
d-DNNF (Darwiche et. Al. 2002)
Case-factor diagrams (Mcallester, Collins, Pereira, 2005)
Tutorial references:
Dechter, Constraint processing, Morgan Kauffman, 2003
Dechter, Tractable Structures of Constraint Satisfaction problems,
In Handbook of constraint solving, forthcoming.

48. Conclusion AND/OR search should always be used.
AND/OR BDDs are superior to OBDDs.
A search algorithm with good and no-good learning generates an OBDD or an AND/OR Bdds.
Dynamic variable ordering can be incorporated
With caching, AND/OR search is similar to inference (variable-elimination)
The real tradeoff should be rephrased:
Time vs space rather than search vs inference, or search vs model-checking.
Search methods are more sensitive to this tradeoff.