Download presentation
Presentation is loading. Please wait.
1
1 Hybrid of search and inference: time- space tradeoffs chapter 10 ICS-275 Spring 2007
2
ICS-275 2 Reasoning Methods Our focus - search and elimination Search (“guessing” assignments, reasoning by assumptions) Branch-and-bound (optimization) Backtracking search (CSPs) Cycle-cutset (CSPs, belief nets) Variable elimination (inference, “propagation” of constraints, probabilities, cost functions) Dynamic programming (optimization) Adaptive consistency (CSPs) Joint-tree propagation (CSPs, belief nets)
3
Spring 2007 ICS-275 3 Chapter 10: Hybrids of Search and Inference Interleaving conditioning and elimination Cutset decomposition AND/OR cutset decomposition Super-bucket and super-clusters Approximating conditioning in a hybrid
4
Spring 2007 ICS-275 4 Satisfiability: Inference vs search Search = O(exp(n)) Search = exp(dfs-height)
5
Spring 2007 ICS-275 5 DR versus DPLL: Complementary Properties Uniform random 3-CNFs (large induced width) (k,m)-tree 3-CNFs (bounded induced width )
6
Spring 2007 ICS-275 6 Exact CSP techniques: complexity
7
Spring 2007 ICS-275 7 Outline; Road Map Methods Tasks
8
Spring 2007 ICS-275 8 The effect of Conditioning X1X1 X3X3 X5X5 X4X4 X2X2
9
Spring 2007 ICS-275 9 The effect of Conditioning X1X1 X3X3 X5X5 X4X4 X2X2 Select a variable
10
Spring 2007 ICS-275 10 The effect of Conditioning X1X1 X3X3 X5X5 X4X4 X2X2 X3X3 X5X5 X4X4 X2X2 X3X3 X5X5 X4X4 X2X2 X3X3 X5X5 X4X4 X2X2 …... X 1 a X 1 b X 1 c
11
Spring 2007 ICS-275 11 The effect of Conditioning X1X1 X3X3 X5X5 X4X4 X2X2 X3X3 X5X5 X4X4 X2X2 X3X3 X5X5 X4X4 X2X2 X3X3 X5X5 X4X4 X2X2 …... X 1 a X 1 b X 1 c Condition on X 2, and BE or BE(w=2) now.
12
Spring 2007 ICS-275 12 The effect of Conditioning X1X1 X3X3 X5X5 X4X4 X2X2 X3X3 X5X5 X4X4 X2X2 X3X3 X5X5 X4X4 X2X2 X3X3 X5X5 X4X4 X2X2 …... X 1 a X 1 b X 1 c General principle: Condition until tractable
13
Spring 2007 ICS-275 13 Interleaving Cond and Elim (VE+C)
14
Spring 2007 ICS-275 14 Interleaving Cond and Elim
15
Spring 2007 ICS-275 15 Interleaving Cond and Elim
16
Spring 2007 ICS-275 16 Interleaving Cond and Elim
17
Spring 2007 ICS-275 17 Interleaving Cond and Elim
18
Spring 2007 ICS-275 18 Interleaving Cond and Elim
19
Spring 2007 ICS-275 19 Interleaving Cond and Elim...
20
Spring 2007 ICS-275 20 Interleaving Cond and Elim...
21
Spring 2007 ICS-275 21 Interleaving Cond and Elim...
22
Spring 2007 ICS-275 22 Interleaving Cond and Elim...
23
Spring 2007 ICS-275 23 Interleaving Cond and Elim...
24
Spring 2007 ICS-275 24 Interleaving Cond and Elim...
25
Spring 2007 ICS-275 25 Interleaving Cond and Elim...
26
Spring 2007 ICS-275 26 Interleaving Cond and Elim...
27
Spring 2007 ICS-275 27 Interleaving Cond and Elim...
28
Spring 2007 ICS-275 28
29
Spring 2007 ICS-275 29 DCDR(b): empirical results
30
Spring 2007 ICS-275 30 Road Map Interleaving conditioning and elimination Cutset decomposition W-cutset, cycle-cutset AND/OR w-cutset Super-bucket and super-clusters
31
Spring 2007 ICS-275 31 Cycle cutset C P J A L B E D F M O H K G N C P J L B E D F M O H K G N A C P J L E D F M O H K G N B P J L E D F M O H K G N C Cycle cutset = {A,B,C} C P J A L B E D F M O H K G N C P J L B E D F M O H K G N C P J L E D F M O H K G N C P J A L B E D F M O H K G N
32
Spring 2007 ICS-275 32 Conditioning and cycle-cutset A=yellow A=green Inference may require too much memory Condition (guessing) on some of the variables C B K G L D F H M J E C B K G L D F H M J E A C B K G L D F H M J E Graph Coloring problem
33
Spring 2007 ICS-275 33 Conditioning A=yellow A=green B=redB=blueB=redB=blueB=greenB=yellow C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E Inference may require too much memory Condition on some of the variables A C B K G L D F H M J E Graph Coloring problem
34
Spring 2007 ICS-275 34 The cycle-cutset scheme: condition until a tree
35
Spring 2007 ICS-275 35 Theorem: The w-cutset scheme yields space complexity exp(w) and time complexity exp(w+c_w), where c_w is the size of the w-cutset. As w decreases, c_w increases. The cycle-cutset decomposition is linear space and Has time complexity of Time-space tradeoff
36
Spring 2007 ICS-275 36 Elim-cond(w) or w-cutset scheme Idea: runs backtracking search on the w-cutset variables and bucket-elimination on the remaining variables. Input: A constraint network R = (X,D,C), Y a w-cutset, d an ordering that starts with Y whose adjusted induced-width, along d, is bounded by w, Z = X-Y. Output: A consistent assignment, if there is one. 1. while {y} next partial solution of Y found by backtracking, do a) z solution found by adaptive-consistency(R_y). B) if z is not false, return solution (y,z). 2. endwhile. return: the problem has no solutions.
37
Spring 2007 ICS-275 37 Finding a w-cutset Verifying a w-cutset can be done in polynomial time A simple greedy: use a good induced-width ordering and starting at the top add to the w- cutset any variable with more than w parents. Alternative: generate a tree-decomposition and select a w-cutset that reduce each cluster below w (Bidyuk and Dechter UAI2005).
38
Spring 2007 ICS-275 38 Treewidth equals cycle cutset E K F H C B A L G J D M N K C A L G J D M N treewidth = cycle cutset = 4
39
Spring 2007 ICS-275 39 Treewidth smaller than cycle cutset E K F H C B A L G J D F C B G D E treewidth = 2 cycle cutset = 5
40
Spring 2007 ICS-275 40 Time-Space complexity of of w-cutset Space: O(exp(w)) W -cutset: a set that when removed the induced- width is w. c(w): size of w-cutset. Time: O(exp(w+c(w))) on OR space
41
Spring 2007 ICS-275 41 Time vs space Random Graphs (50 nodes, 200 edges, average degree 8, w* 23) Branch and bound Bucket elimination 0 10 20 30 40 50 60 258 1114172023262932 w W+c(w)
42
Spring 2007 ICS-275 42 BDR-DP(i): bounded resolution + backtracking Complete algorithm: run BDR(i) as preprocessing before the Davis-Putnam backtracking algorithm. Empirical results: random vs. structured (low-w*) problems:
43
Spring 2007 ICS-275 43 DCDR(b): empirical results (Rish and Dechter 2000)
44
Spring 2007 ICS-275 44 Empirical evaluation of w-cutset Cycle-cutset for CSPs (dechter 1990) Alternating w-cutset for cnfs (Rish and Dechter 2000) Alternating w-cutset for Constraint optimization (Larrosa and Dechter, 2002)
45
Spring 2007 ICS-275 45 The idea of super-buckets Larger super-buckets (cliques) =>more time but less space Complexity: 1.Time: exponential in clique (super-bucket) size 2.Space: exponential in separator size
46
Spring 2007 ICS-275 46 Example
47
Spring 2007 ICS-275 47 Super-Bucket Elimination, SBE(k) Eliminate sets of variables such that: individual eliminations are too costly in space (namely, each variable in the set has degree larger than k) the join degree is lower than k
48
Spring 2007 ICS-275 48 Sep-based time-space tradeoff Let T be a tree-decomposition of hypergraph H. Let s_0,s_1,...,s_n be the sizes of the separators in T, listed in strictly descending order. With each separator size s_i we associate a secondary tree decomposition T_i, generated by combining adjacent nodes whose separator sizes are strictly greater than s_i. We denote by r_i the largest set of variables in any cluster of T_i. Note that as s_i decreases, r_i increase. Theorem: The complexity of CTE when applied to each T_i is O( n exp(r_i)) time, and O( n exp(s_i)) space.
49
Spring 2007 ICS-275 49 Complexity Theorem: If R = (X,D,C) is a constraint network whose constraint graph has non-separable components of at most size r, then the super-bucket elimination algorithm, whose buckets are the non- separable components, is time exponential O(n exp(r)) and is linear in space.
50
Spring 2007 ICS-275 50 Hybrids of hybrids hybrid(b_1,b_2): First, a tree-decomposition having separators bounded by b_1 is created, followed by application of the CTE algorithm, but each clique is processed by elim-cond(b_2). If c^*_{b_2} is the size of the maximum b_2-cutset in each clique of the b_1-tree- decomposition, the algorithm is space exponential in b_1 but time exponential in c^*_{b_2}. Special cases: hybrid(b_1,1): Applies cycle-cutset in each clique. b_1 = b_2. For b=1, hybrid(1,1) is the non-separable components utilizing the cycle-cutset in each component. The space complexity of this algorithm is linear but its time complexity can be much better than the cycle-cutsets cheme or the non-separable component approach alone.
51
Spring 2007 ICS-275 51 Case study: circuit diagnosis Problem: Given a circuit and its unexpected output, identify faulty components. The problem can be modeled as a constraint optimization problem and solved by bucket elimination.
52
Spring 2007 ICS-275 52 Case study: combinatorial circuits: benchmark used for fault diagnosis and testing community Problem: Given a circuit and its unexpected output, identify faulty components. The problem can be modeled as a constraint optimization problem and solved by bucket elimination.
53
Spring 2007 ICS-275 53 Case study: C432 A circuit’s primal graph For every gate we connect inputs and outputs Join-tree of c432 Seperator size is 23
54
Spring 2007 ICS-275 54 Join-tree of C3540 (1719 vars) max sep size 89
55
Spring 2007 ICS-275 55 Secondary trees for C432
56
Spring 2007 ICS-275 56 Time-space tradeoffsTime/Space tradeoff Time is measured by the maximum of the separator size and the cutset size and space by the maximum separator size.
57
Spring 2007 ICS-275 57 Constraint Optimization: Combinatorial Auction: Bucket-elimination vs Search b1 b2 b3 b4 b5 b6 Bucket-elimination = Dynamic programming Bucket-elimination: In a bucket sum costs and maximize over constrained assignments: b2b2 b1b1 b3b3 b4b4 b5b5 b6b6 Search: Branch and Bound or Best-first search.
58
Spring 2007 ICS-275 58 Recursive-search: a linear space search guided by a tree-decomposition Given a tree network, we identify a node x_1 which, when removed, generates two subtrees of size n/2 (approximately). T_n is the time to solve a binary tree starting at x_1. T_n obeys recurrence T_n = k 2 T_n/2, T_1 = k We get: T_n = n k^{logn +1} Given a tree-decomposition having induced-width w* this generalize to recursive conditioning of tree-decompositions: T_n = n k^({w*+1} log n) because the number of values k is replaced by th enumber of tuples k^w*
59
Spring 2007 ICS-275 59 Road Map Interleaving conditioning and elimination Cutset decomposition AND/OR cutset decomposition Super-bucket and super-clusters Time-space curves
60
Spring 2007 ICS-275 60 OR Conditioning A=yellow A=green B=redB=blueB=redB=blueB=greenB=yellow C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E C K G L D F H M J E Inference may require too much memory Condition on some of the variables A C B K G L D F H M J E Graph Coloring problem
61
Spring 2007 ICS-275 61 AND/OR w-cutset A C BK G L DF H M J E A C B K G L D F H M J E A C BK G L DF H M J E C B K G L D F H M J E 3-cutset A C BK G L DF H M J E C K G L D F H M J E 2-cutset A C BK G L DF H M J E L D F H M J E 1-cutset
62
Spring 2007 ICS-275 62 Selecting an AND/OR w-cutset A C B K G L D F H M J E A C B K G L D F H M J E A C B K G L D F H M J E pseudo tree1-cutset treegrahpical model
63
Spring 2007 ICS-275 63 Searching AND/OR Graphs AO(j): searches depth-first, cache i-context j = the max size of a cache table (i.e. number of variables in a context) j=0j=w* Space:O(n) Time:O(exp(w* log n)) Space:O(exp w*) Time:O(exp w*) Space:O(exp(j) ) Time:O(exp(m_j+j) m_j - AO depth of j-cutset j
64
Spring 2007 ICS-275 64 Conceptual difference Traditional (w) cutset: Minimize the number of nodes Improve by organizing in AND/OR tree AND/OR (w) cutset: Minimize depth of AND/OR cutset tree
65
Spring 2007 ICS-275 65 AND/OR (w) cutset properties THEOREM: Given a graphical model, |Cutset | ≥ depth(Cutset) ≥ depth(AO-Cutset) The inequalities are often strict. Cutset = minimal cardinality (w) cutset depth(Cutset)= minimal depth over pseudo trees for Cutset AO-Cutset= minimal depth AND/OR (w) cutset
66
Spring 2007 ICS-275 66 How to find the AND/OR (w) cutset Finding the minimal depth AND/OR (w) cutset is NP hard Heuristics for AND/OR w-cutset Min-Fill – (for large w) Min-Depth – (for small w)
67
Spring 2007 ICS-275 67 Quality of start pseudo trees CPCS 422 network
68
Spring 2007 ICS-275 68 Improved caching scheme context(X k ) = [X 1 X 2 …X k ] i-bound < k X1X1 A X K-i XkXk X K-i+1 context(X k ) = [X k-i+1 …X k ] in conditioned subproblem Cache for X k is purged for every new instantiation of X k-i X1X1 A X K-i XkXk X K-i+1
69
Spring 2007 ICS-275 69 Experimental results Measures Time Networks Random, Genetic Linkage Algorithms AO(i) – AO with caching full contexts (No Caching on Cutset) + (AO Search) (Caching on Cutset) + (AO Search) (Caching on Cutset) + (BE)
70
Spring 2007 ICS-275 70 Random networks (Time) N=40, K=3, P=2, 20 instances, w*=7 i-boundAlgorithmsdepthTime(sec) MinFillMinDepthMinFillMinDepth 1AO(i)129610.1427.12 (No Caching on Cutset) + (AO Search) 174.538.75 (Caching on Cutset) + (AO Search) 67.997.61 (Caching on Cutset) + (BE) 16.952.18 3AO(i)7671.688.13 (No Caching on Cutset) + (AO Search) 5.730.84 (Caching on Cutset) + (AO Search) 2.940.84 (Caching on Cutset) + (BE) 0.690.25 5AO(i)4311.282.77 (No Caching on Cutset) + (AO Search) 0.550.54 (Caching on Cutset) + (AO Search) 0.55 (Caching on Cutset) + (BE) 0.100.04
71
Spring 2007 ICS-275 71 Random networks (Time) N=60, K=3, P=2, 20 instances, w*=11 i-boundAlgorithmsdepthTime(sec) MinFillMinDepthMinFillMinDepth 6(No Caching on Cutset) + (AO Search)76159.7963.01 (Caching on Cutset) + (AO Search) 112.4362.98 (Caching on Cutset) + (BE) 27.335.50 9(No Caching on Cutset) + (AO Search)3324.4041.45 (Caching on Cutset) + (AO Search) 24.1540.93 (Caching on Cutset) + (BE) 4.272.89 11(No Caching on Cutset) + (AO Search)0117.3938.46 (Caching on Cutset) + (AO Search) 17.6638.22 (Caching on Cutset) + (BE) 1.292.81
72
Spring 2007 ICS-275 72 Genetic Linkage Networks EA4 - N=1173, K=5, w*=15 i-boundAlgorithmsdepthTime(sec) MinFillMinDepthMinFillMinDepth 6AO(i)232110.0103.4 (No Caching on Cutset) + (AO Search) 22.576.4 (Caching on Cutset) + (AO Search) 2.051.3 (Caching on Cutset) + (BE) 8.482.3 9AO(i)18173.39.3 (No Caching on Cutset) + (AO Search) 1.64.7 (Caching on Cutset) + (AO Search) 1.54.8 (Caching on Cutset) + (BE) 3.57.0 13AO(i)382.05.9 (No Caching on Cutset) + (AO Search) 1.43.6 (Caching on Cutset) + (AO Search) 1.63.4 (Caching on Cutset) + (BE) 0.75.3
73
Spring 2007 ICS-275 73 Time-Space complexity of of w-cutset Space: O(exp(w)) W -cutset: a set that when removed the induced- width is w. c(w): size of w-cutset. m(w): depth of AO w-cutset Time: O(exp(w+c(w))) on OR space Time: O(exp(w+m(w))) on AND/OR space
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.