Download presentation
Presentation is loading. Please wait.
1
Context-specific independence
Readings: K&F: 4.1, 4.2, 4.3, 4.4, 8.4, 8.5, 8.6 “Recursive Conditioning”, Adnan Darwiche. In Artificial Intelligence Journal, 125:1, pp. 5-41 Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16th, 2006
2
Global Structure: Treewidth w
3
Local Structure 1: Context specific indepencence
Battery Age Alternator Fan Belt Charge Delivered Battery Fuel Pump Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start
4
Local Structure 1: Context specific indepencence
Context Specific Independence (CSI) After observing a variable, some vars become independent Battery Age Alternator Fan Belt Charge Delivered Battery Fuel Pump Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start
5
CSI example: Tree CPD Represent P(Xi|PaXi) using a decision tree
Apply SAT Letter Job Represent P(Xi|PaXi) using a decision tree Path to leaf is an assignment to (a subset of) PaXi Leaves are distributions over Xi given assignment of PaXi on path to leaf Interpretation of leaf: For specific assignment of PaXi on path to this leaf – Xi is independent of other parents Representation can be exponentially smaller than equivalent table
6
Tabular VE with Tree CPDs
If we turn a tree CPD into table “Sparsity” lost! Need inference approach that deals with tree CPD directly!
7
Local Structure 2: Determinism
If Battery Power = Dead, then Lights = OFF Determinism Battery Age Alternator Fan Belt ON OFF OK WEAK DEAD Lights Battery Power .99 .01 .20 .80 1 Charge Delivered Battery Fuel Pump Fuel Line Starter Gas Distributor Battery Power Gas Gauge Spark Plugs Radio Lights Engine Turn Over Engine Start
8
Determinism and inference
Determinism gives a little sparsity in table, but much bigger impact on inference Multiplying deterministic factor with other factor introduces many new zeros Operations related to theorem proving, e.g., unit resolution ON OFF OK WEAK DEAD Lights Battery Power .99 .01 .20 .80 1
9
Today’s Models … Often characterized by: Enabled by:
Richness in local structure (determinism, CSI) Massiveness in size (10,000’s variables) High connectivity (treewidth) Enabled by: High level modeling tools: relational, first order Advances in machine learning New application areas (synthesis): Bioinformatics (e.g. linkage analysis) Sensor networks Exploiting local structure a must!
10
Exact inference in large models is possible…
BN from a relational model
11
Recursive Conditioning
Treewidth complexity (worst case) Better than treewidth complexity with local structure Provides a framework for time-space tradeoffs Only quick intuition today, details in readings
12
The Computational Power of Assumptions
Battery Age Alternator Fan Belt Leak Charge Delivered Battery Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start A. Darwiche
13
The Computational Power of Assumptions
Battery Age Alternator Fan Belt Leak Charge Delivered Battery Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start A. Darwiche
14
Decomposition Gas A. Darwiche Battery Age Alternator Fan Belt Leak
Charge Delivered Battery Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start A. Darwiche
15
Case Analysis + p p A. Darwiche Battery Age Alternator Fan Belt
Leak Leak Charge Delivered Battery Charge Delivered Fuel Line Battery Fuel Line Starter Gas Distributor Starter Gas Distributor Battery Power Battery Power Spark Plugs Gas Gauge Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start + p p A. Darwiche
16
Case Analysis * + pl pr p A. Darwiche Battery Age Alternator Fan Belt
Leak Leak Charge Delivered Battery Charge Delivered Fuel Line Battery Fuel Line Starter Gas Distributor Starter Gas Distributor Battery Power Battery Power Spark Plugs Gas Gauge Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start * + pl pr p A. Darwiche
17
Case Analysis * + * pl pr pl pr A. Darwiche Battery Age Alternator
Fan Belt Battery Age Alternator Fan Belt Leak Leak Charge Delivered Battery Charge Delivered Fuel Line Battery Fuel Line Starter Gas Distributor Starter Gas Distributor Battery Power Battery Power Spark Plugs Gas Gauge Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start * + * pl pr pl pr A. Darwiche
18
Case Analysis * + * pl pr pl pr A. Darwiche Alternator Fan Belt
Battery Age Battery Age Alternator Fan Belt Leak Charge Delivered Leak Battery Charge Delivered Fuel Line Battery Fuel Line Starter Gas Distributor Starter Gas Distributor Battery Power Battery Power Spark Plugs Gas Gauge Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start * + * pl pr pl pr A. Darwiche
19
Case Analysis * + * pl pr pl pr A. Darwiche Alternator Fan Belt
Battery Age Battery Age Alternator Fan Belt Leak Charge Delivered Leak Battery Charge Delivered Fuel Line Battery Fuel Line Starter Gas Distributor Starter Gas Distributor Battery Power Battery Power Spark Plugs Gas Gauge Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start * + * pl pr pl pr A. Darwiche
20
Decomposition Tree A B C D E B A B C D E Cutset f(A) f(A,B) f(B,C)
f(C,D) f(B,D,E) A. Darwiche
21
Decomposition Tree A B C D E B A A B B C B C D E D Cutset f(A) f(A,B)
f(B,C) B C D E D f(C,D) f(B,D,E) A. Darwiche
22
Decomposition Tree A B C D E B A A B C Time: O(n exp(w log n))
Cutset B A A B C f(A) f(A,B) f(B,C) Time: O(n exp(w log n)) Space: Linear (using appropriate dtree) C D E D f(C,D) f(B,D,E) A. Darwiche
23
RC1 If T is a leaf node Return Lookup(T,e) Else RC1(T,e)
// compute probability of evidence e on dtree T If T is a leaf node Return Lookup(T,e) Else p := 0 for each instantiation c of cutset(T)-E do p := p + RC1(Tl,ec) RC1(Tr,ec) return p A. Darwiche
24
Lookup(T,e) QX|U : CPT associated with leaf T
If X is instantiated in e, then x: value of X in e u: value of U in e Return qx|u Else return 1 = Sx qx|u A. Darwiche
25
Caching A B C A B C C A B C D E F .27 .39 ABC A B C D F E Context
A. Darwiche
26
Caching A B C A B C C A B C D E F
Recursive Conditioning An any-space algorithm with treewidth complexity Darwiche AIJ-01 A B C D E F C .27 .39 Context Time: O(n exp(w)) Space: O(n exp(w)) (using appropriate dtree) ABC A B C A. Darwiche
27
RC2 RC2(T,e) If T is a leaf node, return Lookup(T,e)
y := instantiation of context(T) If cacheT[y] <> nil, return cacheT[y] p := 0 For each instantiation c of cutset(T)-E do p := p + RC2(Tl,ec) RC2(Tr,ec) cacheT[y] := p Return p A. Darwiche
28
Decomposition with Local Structure
X Independent of B, C given A A, B, C A X B C A. Darwiche
29
Decomposition with Local Structure
X Independent of B, C given A A, B, C A X B C A. Darwiche
30
Decomposition with Local Structure
X Independent of B, C given A A, B, C No need to consider an exponential number of cases (in the cutset size) given local structure A X B C A. Darwiche
31
Caching with Local Structure
B A Structural cache Caching with Local Structure B,C A A,B,C B C X A A. Darwiche
32
Caching with Local Structure
B A Structural cache Caching with Local Structure B,C A A,B,C B C X A A. Darwiche
33
Caching with Local Structure
B A Structural cache Caching with Local Structure No need to cache an exponential number of results (in the context size) given local structure B,C A C B A Non- Structural cache A,B,C A X B C A. Darwiche
34
Determinism… A, B, C A natural setup to incorporate SAT technology: A
X A A, B, C A natural setup to incorporate SAT technology: Unit resolution to: Derive values of variables Detect/skip inconsistent cases Dependency directed backtracking Clause learning More powerful than logical pre-processing A. Darwiche
35
CSI Summary Exploit local structure Significantly speed-up inference
Context-specific independence Determinism Significantly speed-up inference Tackle problems with tree-width in the thousands Acknowledgements Recursive conditioning slides courtesy of Adnan Darwiche Implementation available:
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.