Verification/constraints workshop, 2006 From AND/OR Search to AND/OR BDDs Rina Dechter Information and Computer Science, UC-Irvine, and Radcliffe Institue of Advanced Study, Cambridge Joint work with Robert Mateescu and John Cobb

Combining Decision Diagrams f = A*E + B*F g = C*G +D*H B A F 01 E B E D C H 01 G D G B A C DD C B DD C DD 01 H GG FFF EEEEEE OBDD f * g =

Combining AND/OR BDDs f = A*E + B*F g = C*G +D*H B A F 01 E B E D C H 01 G D G AOBDD f * g = B A F 01 E B E D C H 01 G D G AND

OBDDs vs AOBDD f * g = B A F 01 E B E D C H 01 G D G AND B A C DD C B DD C DD 01 H GG FFF EEEEEE AOBDD OBDD

Outline Background in Graphical models AND/OR search trees and Graphs Minimal AND/OR graphs From AND/OR graphs to AOMDDs Compilation of AOMDDs AOMDDs and earlier BDDs

AB redgreen redyellow greenred greenyellow yellowgreen yellow red Example: map coloring Variables - countries (A,B,C,etc.) Values - colors (red, green, blue) Constraints: C A B D E F G A Constraint Networks A B E G D F C Constraint graph Semantics: set of all solutions Primary task: find a solution

= {(¬C), (A v B v C), (¬A v B v E), (¬B v C v D)}. Propositional Satisfiability

A graphical model (X,D,C): X = {X 1,…X n } variables D = {D 1, … D n } domains C = {F 1,…,F t }functions (constraints, CPTS, cnfs) Graphical models A D BC E F All these tasks are NP-hard identify special cases approximate Examples: Constraint networks Belief networks Cost networks Markov random fields Influence diagrams

Tree-solving is easy Belief updating (sum-prod) MPE (max-prod) CSP – consistency (projection-join) #CSP (sum-prod) P(X) P(Y|X)P(Z|X) P(T|Y)P(R|Y)P(L|Z)P(M|Z) Trees are processed in linear time and memory

Transforming into a Tree By Inference (thinking) Transform into a single, equivalent tree of sub-problems By Conditioning (guessing) Transform into many tree-like sub- problems.

Inference and Treewidth E K F L H C B A M G J D ABC BDEF DGF EFH FHK HJKLM treewidth = = 3 treewidth = (maximum cluster size) - 1 Inference algorithm: Time: exp(tree-width) Space: exp(tree-width)

Conditioning and Cycle cutset C P J A L B E D F M O H K G N C P J L B E D F M O H K G N A C P J L E D F M O H K G N B P J L E D F M O H K G N C Cycle cutset = {A,B,C} C P J A L B E D F M O H K G N C P J L B E D F M O H K G N C P J L E D F M O H K G N C P J A L B E D F M O H K G N

Search over the Cutset A=yellow A=green Inference may require too much memory Condition (guessing) on some of the variables C B K G L D F H M J E C B K G L D F H M J E A C B K G L D F H M J E Graph Coloring problem

Search over the Cutset (cont) A=yellow A=green B=redB=blueB=redB=blueB=greenB=yellow C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E Inference may require too much memory Condition on some of the variables A C B K G L D F H M J E Graph Coloring problem

Inference vs. Conditioning By Inference (thinking) E K F L H C B A M G J D ABC BDEF DGF EFH FHK HJ KLM By Conditioning (guessing) Exponential in treewidth Time and memory Exponential in cycle-cutset Time-wise, linear memory A=yellowA=green B=blueB=redB=blueB=green C K G L D F H M J E A C B K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E Variable-elimination Directional resolution Join-tree clustering Backtracking Branch and bound Depth-first search

Outline Background in Graphical models AND/OR search trees and Graphs Minimal AND/OR graphs From AND/OR graphs to AOMDDs Compilation of AOMDDs AOMDDs and earlier BDDs

Classic OR Search Space E C F D B A 01 Ordering: A B E C D F A D BC E F

AND/OR Search Space A OR 0 AND 1 B OR B 0 AND E OR C EC EC EC DFDF DFDF DFDF DFDF AND A D BC E F A D B CE F Primal graphDFS tree A D BC E F A D BC E F

AND/OR vs. OR A OR 0 AND 1 B OR B 0 AND E OR C EC EC EC DFDF DFDF DFDF DFDF AND E C F D B 101 A 01 AND/OR OR A D BC E F A D B CE F AND/OR size: exp(4), OR size exp(6)

AND/OR vs. OR with Constraints A OR 0 AND 1 B OR B 0 AND E OR C EC EC EC DFDF DFDF DFDF DFDF AND E C F D B 101 A 01 AND/OR OR A D BC E F A D B CE F No-goods (A=1,B=1) (B=0,C=0)

AND/OR vs. OR with Constraints E C F D B 10 A 01 A OR 0 AND 1 B OR B 0 AND 1 0 E OR C EC EC DF DFDF DF AND E C F D B 10 A 01 E C F D B 10 A 01 AND/OR OR A D BC E F A D B CE F No-goods (A=1,B=1) (B=0,C=0

DFS algorithm (#CSP example) A 0 B 0 EC DFDF OR AND OR AND OR AND 0101 A D BC E F AND node: Combination operator (product) OR node: Marginalization operator (summation) A D B CE F Value of node = number of solutions below it solution

Pseudo-Trees ( Freuder 85, Bayardo 95, Bodlaender and Gilbert, 91 ) (a) Graph (b) DFS tree depth=3 (c) pseudo- tree depth=2 (d) Chain depth= m <= w* log n

Tasks and value of nodes V( n) is the value of the tree T(n) for the task: Consistency: v(n) is 0 if T(n) inconsistent, 1 othewise. Counting: v(n) is number of solutions in T(n) Optimization: v(n) is the optimal solution in T(n) Belief updating: v(n), probability of evidence in T(n). Partition function: v(n) is the total probability in T(n). Goal : compute the value of the root node recursively using dfs search of the AND/OR tree. AND/OR search tree and algorithms are ( [Freuder & Quinn85], [Collin, Dechter & Katz91], [Bayardo & Miranker95],[Darwiche 2001], [Bacchus et. Al, 2003] ) Space: O(n) Time: O(exp(m)), where m is the depth of the pseudo-tree Time: O(exp(w* log n)) BFS is time and space O(exp(w* log n)

From AND/OR Tree A D BC E F A D B CE F GH J K G H J K A OR 0 AND 1 B OR B 0 AND E OR C EC EC EC DFDF DFDF DFDF DFDF AND OR AND 0 G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK

To an AND/OR Graph A D BC E F A D B CE F GH J K G H J K A OR 0 AND 1 B OR B 0 AND E OR C EC EC EC DFDF DFDF DFDF DFDF AND OR AND 0 G HH G HH J KK J KK

AND/OR Search Graphs Any two nodes that root identical subtrees/subgraphs (are unifiable) can be merged Minimal AND/OR search graph: of R relative to tree T is the closure under merge of the AND/OR search tree of R relative to T, where inconsistent subtrees are pruned. Canonicity: The minimal AND/OR graph relative to T is unique for all constraints equivalent to R.

Context based caching Caching is possible when context is the same context=parent-separator set in induced pseudo-graph =current variable + ancestors connected to subtree below A D BC E F A D B CE F GH J K G H J K context(B) = {A, B} context(C) = {A,B,C} context(D) = {D} context(F) = {F}

Caching A D BC E F A D B CE F GH J K G H J K A OR 0 AND 1 B OR B 0 AND E OR C EC EC EC DFDF DFDF DFDF DFDF AND OR AND 0 G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK G HH G HH J KK J KK context(B) = {A, B} context(C) = {A,B,C} context(D) = {D} context(F) = {F}

Caching A D BC E F A D B CE F GH J K G H J K A OR 0 AND 1 B OR B 0 AND E OR C EC EC EC DFDF DFDF DFDF DFDF AND OR AND 0 G HH G HH J KK J KK context(D)={D} context(F)={F}

All Four Search Spaces Full OR search tree 126 nodes Full AND/OR search tree 54 AND nodes Context minimal OR search graph 28 nodes Context minimal AND/OR search graph 18 AND nodes C D F E B A 01 A E C B F D

Properties of minimal AND/OR graphs Complexity: Minimal AND/OR R relative to pseudo-tree T is O(exp(w*)) where w* is the tree-width of R along T. Minimal OR search graph is O(exp(pw*)) where pw* is path- width w* pw*, pw* w*log n Canonicity: The minimal AND/OR search graph is unique (canonical) for all equivalent formulas (Boolean or Constraints), consistent with its pseudo tree.

Searching AND/OR Graphs AO(j): searches depth-first, cache i-context j = the max size of a cache table (i.e. number of variables in a context) i=0i=w* Space:O(n) Time:O(exp(w* log n)) Space:O(exp w*) Time:O(exp w*) Space:O(exp(j) ) Time:O(exp(m_j+j ) j

Context-Based Caching A D B EC F context(A) = {A} context(B) = {B,A} context(C) = {C,B} context(D) = {D} context(E) = {E,A} context(F) = {F} Cache Table (C) BCValue A 0 B 0 E C DD 01 1 E C DD 01 1 B 0 E C 1 E C A E C B F D Primal graph Space: O(exp(2))

Outline Background in Graphical models AND/OR search trees and Graphs Minimal AND/OR graphs From AND/OR search graphs to AOMDDs Compilation of AOMDDs AOMDDs and earlier BDDs

OR Search Graphs vs OBDDs A B C ABCf(ABC) A 01 B 01 C 01 C 01 B 01 C 01 C 01 Full AND/OR search tree A 01 B 1 C 1 B 01 C 1 C 1 Backtrack free AND/OR search tree A 01 B 1 B 01 C 1 Minimal AND/OR search graph redundant A 01 B 1 C 1 An OBDD

AND/OR Search Graphs; AOBDDs A B D C A 0 1 B 1 C 1 D 1 D 0 redundancy F(A,B,C,D)= (0,1,1,1), (1,0,1,0), (1,1,1,0) A 0 1 B 1 C 1 D 1 D 0 B 0 1 AOBDD(F)

AND/OR Search Graphs; AOBDDs A B D C A 0 1 B 1 C 1 D 1 D 0 redundancy F(A,B,C,D)= (0,1,1,1), (1,0,1,0), (1,1,1,0) A 0 1 B 1 C 1 D 1 D 0 B 0 1 AOBDD(F)

AOBDD Conventions A B D C A 01 B 1 C 1 D 1 D 0 Point dead-ends to terminal 0 Point goods to terminal 1 A 01 B 01 0 C 01 1 D 01 D 01

AOBDD Conventions A B D C A 01 B 1 C 1 D 1 D 0 A 01 B 01 0 C 01 1 D 01 D 01 Introduce Meta-nodes

Combining AOBDD (apply) A B C ABCf(ABC) A 01 B 01 1 C 01 * A B D ABDg(ABC) A B D C 0 A 01 B 01 1 D 01 D 01 = A 01 B 01 0 C 01 1 D 01 D 01

H G E D C F B A F 01 H C1C1 A 01 H C2C2 A 01 B 01 G 01 G 01 B C3C3 F 01 G C4C4 A 01 E C6C6 C 01 E C7C7 C 01 D 01 D C8C8 B 01 C C9C9 B 01 F C5C5 m1m1 m3m3 m7m7 m6m6 m4m4 m2m2 m5m5 Example: (f+h) * (a+!h) * (a#b#g) * (f+g) * (b+f) * (a+e) * (c+e) * (c#d) * (b+c) A 01 F 01 H 01 F 01 H m1m1 A 01 B 01 G 01 F 01 B 01 G m2m2 A 01 E 01 C m4m4 C 01 D 01 D m5m5 A 01 B 01 F 01 G 01 H 01 F 01 G 01 B 01 F 01 F 01 H m3m3 A 01 B 01 C 01 C 01 D 01 E 01 D 01 B 01 C 01 C m6m6 m7m7 A F H D CBF AE G H primal graph A B CF DEGH pseudo tree A B F GH C D A B C DE A B CF DEGH A B F G C E A

B A Example (continued) m7m7 A 01 B 01 F 01 G 01 H 01 F 01 G 01 B 01 F 01 F 01 H m3m3 A 01 B 01 C 01 C 01 D 01 E 01 D 01 B 01 C 01 C m6m6 A B CF DEGH m7m7 A 01 B 01 C 01 0 D 01 1 F 01 G 01 H 01 C 01 D 01 E 01 F 01 G 01 B 01 C 01 F 01 C 01 F 01 H 01

AOBDD vs. OBDD D C B F A E G H 10 B CCC DDDDD EE FFF GGGG H AOBDD 18 nonterminals 47 arcs OBDD 27 nonterminals 54 arcs A 01 B 01 C 01 0 D 01 1 F 01 G 01 H 01 C 01 D 01 E 01 F 01 G 01 B 01 C 01 F 01 C 01 F 01 H 01 A B CF DEGH A B C F D E G H D CBF AE G H primal graph

Complexity Complexity of apply: Complexity of apply is bounded quadratically by the product of input AOBDDs restricted to each branch in the output pseudo-tree. Complexity of VE-AOBDD: is exponential in the tree-width along the pseudo-tree.

AOMDDs and tree-BDDs Tree-BDDs (McMillan1994) are : AND/OR BDDS are

Related work Related work in Search Backjumping + learning (Freuder 1988,Dechter 1990,Bayardo and Mirankar 1996) Recursive-conditioning (Darwiche 2001) Value elimination (Bacchus et. Al. 2003) Search over tree-decompositions (Trrioux et. Al. 2002) Related to compilation schemes: Minimal AND/OR – related to tree-OBDDs (McMillan 94), d-DNNF (Darwiche et. Al. 2002) Case-factor diagrams (Mcallester, Collins, Pereira, 2005) Tutorial references: Dechter, Constraint processing, Morgan Kauffman, 2003 Dechter, Tractable Structures of Constraint Satisfaction problems, In Handbook of constraint solving, forthcoming.

Conclusion AND/OR search should always be used. AND/OR BDDs are superior to OBDDs. A search algorithm with good and no-good learning generates an OBDD or an AND/OR Bdds. Dynamic variable ordering can be incorporated With caching, AND/OR search is similar to inference (variable- elimination) The real tradeoff should be rephrased: Time vs space rather than search vs inference, or search vs model-checking. Search methods are more sensitive to this tradeoff.