Tree Clustering for Constraint Networks 1 Chris Reeson Advanced Constraint Processing Fall 2009 By Rina Dechter & Judea Pearl Artificial Intelligence, Oct 1988

Overview Contributions of the Paper Context of the Paper Algorithms – Tree Clustering – Adaptive Consistency Relative Merits Conclusion Related algorithms 2

Contributions of the Paper Introduces Tree Clustering (T-C) for backtrack- free search Introduces the Adaptive Consistency (A-C) algorithm Compares the two algorithms 3

Context Two ways to restrict CSPs to make finding the minimal CSP efficient – Topology of constraint graph – Types of constraints T-C & A-C – Target the topology: tree – Are applicable to binary & non-binary CSPs Two ways to transform a constraint graph into a tree – Remove redundant arcs in dual graphs (join tree) – Form larger clusters of c-variables (simulates a join tree) 4

Definitions Hyper, dual, and primal graphs 5 h8 h7 h1 h5 h3 h4 h6 h HypergraphDual graphPrimal graph h3 h2 h4 h7 h6 h8 h5 The discussion focuses on primal graphs for simplicity

From Join Graph to Join Tree Join Graph – Start w/ the dual graph, remove redundant edges while maintaining the connectedness property – Connectedness property: For each two nodes sharing a variable, there is at least one path of labeled arcs containing the shared variable Join Tree – When the join graph is a tree, the dual CSP can be solved BT free w/ directional arc consistency – What if there isn’t a join tree?  The idea for Tree Clustering ACECDE AEFABC A CE EAC C AE ACECDE AEFABC CE AE AC

Motivating Problem 7 Constraint Graph D BA C E F G Domains: {1, 2, 3, 4, 5} A solution G > F C ≠ D C < A C < BD < A D < B F > BE > B G > E A ≠ B

Overview Contributions of the Paper Context of the Paper Algorithms – Tree Clustering (T-C) – Adaptive Consistency (A-C) Relative Merits Conclusion Related Algorithms 8

Tree Clustering (T-C): Idea A CSP organized as a join tree can be solved efficiently Tree Clustering Algorithm – Solves a CSP by breaking it into subproblems – Triangulates the primal graph – Solves subproblems & combines the solutions 9 D BA C E F G DB AC EF G ABCD BED DFE EFG BD DE EF

ABCD BED DFE EFG BD DE EF 4312, 5312, 5432, … 342, 351, 221, … 234, 415, 153, … 435, 123, 112, … Tree Clustering (T-C): Algorithm 1.Triangulate the primal graph 2.Identify all the maximal cliques in the primal chordal graph 3.Form a join tree 4.Solve the subproblems – Each cluster becomes single variable 5.Solve the tree problem – Perform DAC from leaves to root – Instantiate BT-free from root to leaves 10 DB AC EF G

Tree Clustering (T-C): Costs 1.Given a CSP and its primal graph generate a chordal primal graph: O(n 2 ) 2.Identify all the maximal cliques in the primal chordal graph: O(|E’|) 3.Form the dual graph: O(n) 4.Solve the sub problems: O(k r ) where k=domain size 5.Solve the tree problem: O(n ∙ t log t)… 11 DB AC EF G ABCD BED DFE EFG BD DE EF 4312, 5312, 5432, … 342, 351, 221, … 234, 415, 153, … 435, 123, 112, …

Tree Clustering (T-C): Total Cost Dominated by O(n ∙ t log t) – t is the largest number of solutions in a cluster, t ≤ k r – Time: O(n ∙ k r ∙ r log k) = O(nr ∙ k r ) – Space: O(n ∙ k r ) 12

Overview Contributions of the Paper Context of the Paper Algorithms – Tree Clustering (T-C) – Adaptive Consistency (A-C) Relative Merits Conclusion Related Algorithms 13

Adaptive Consistency (A-C) An ordered constraint graph is backtrack-free if the level of directional strong consistency along this order is greater than the width of the ordered graph Beware – Enforcing i-consistency for i > 2 often requires the addition of constraints which increase the width 14

Adaptive Consistency (A-C): Idea Given an ordering d, – d-i-consistency is defined recursively – letting i change dynamically from node to node (A-C later redefined as bucket elimination) 15

Adaptive Consistency: Algorithm 1.For i=n downto 1 do Steps Compute PARENTS(X i ) 3.Connect all PARENTS(X i ) 4.Perform Consistency(X i, PARENTS(X i ))  joining the constraints between X i & its parents 5.Build a solution BT-free in the ordering (X 1, …, X n ) 16 DB AC EF G C B D E F G A C B D E F G A tighten A by 2 consist ACB join CB join AC to tighten AC by 3c BE join CB join AC to tighten ACB by 4c tighten D by 2 consist EF join DE to tighten DE by 3 consist GF join GE to tighten EF by 3 consist

Adaptive Consistency: Cost Time: O(n ∙ exp(W*(d) + 1)), see Dechter page 109 Space: O(n ∙ k W*(d) ) 17

Overview Contributions of the Paper Context of the Paper Algorithms – Tree Clustering (T-C) – Adaptive Consistency (A-C) Relative Merits Conclusion Related Algorithms 18

Relative Merits Arcs resulting from triangulation match arcs added by adaptive consistency, for the same ordering Every cluster in T-C is represented in A- C by a series of smaller constraints Similar bounds – W*(d) + 1 = the size of the largest clique A-C eliminates the redundancy of generated solutions T-C enumerates all solutions that A-C represents via constraints. 19 DB AC EF G C B D E F G A DB AC EF G C B D E F G A

Conclusions Tree clustering groups c-nodes into a tree capable of supporting query answering backtrack-free Useful in systems that need to answer many questions about a dataset and where the environmental conditions undergo local changes Recently, researchers have started looking at T-C for solving the CSP, see BTD by Jégou & Terrioux (and others in soft CSPs) 20

Note On Triangulation Find the triangulated graph w/ smallest maximum clique: NP-hard Heuristics – Operation: when eliminating a node, connect all its neighbors, to form a clique (fill edges) – H1: choose the node w/ smallest degree – H2: choose the node that, after elimination, yields the smallest number of fill edges – H3: Given any ordering (e.g., maximal cardinality ordering), moralize the graph Elimination order is the reverse of instantiation order Elimination order of a triangulated graph is called a perfect elimination scheme – In this ordering, every node is simplicial: forms a clique w/ its neighbors – If you follow elimination order, no fill edges need to be added

Maximal Cardinality Ordering An approximation of min. width ordering Choose a node arbitrarily a simplicial node Among the remaining nodes, choose the one that is connected to the maximum number of already chosen nodes, break ties arbitrarily Repeat… Reverse the final order Tsang Dechter Fig 4.5

Two Additional Algorithms Maximal Cliques of the triangulated graph Join Tree of the triangulated graph