Presentation is loading. Please wait.

Presentation is loading. Please wait.

Context-specific independence

Published byViktor Kadlec Modified over 5 years ago

Similar presentations

Presentation on theme: "Context-specific independence"— Presentation transcript:

1 Context-specific independence
Readings: K&F: 4.1, 4.2, 4.3, 4.4, 8.4, 8.5, 8.6 “Recursive Conditioning”, Adnan Darwiche. In Artificial Intelligence Journal, 125:1, pp. 5-41 Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16th, 2006

2 Global Structure: Treewidth w

3 Local Structure 1: Context specific indepencence
Battery Age Alternator Fan Belt Charge Delivered Battery Fuel Pump Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start

4 Local Structure 1: Context specific indepencence
Context Specific Independence (CSI) After observing a variable, some vars become independent Battery Age Alternator Fan Belt Charge Delivered Battery Fuel Pump Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start

5 CSI example: Tree CPD Represent P(Xi|PaXi) using a decision tree
Apply SAT Letter Job Represent P(Xi|PaXi) using a decision tree Path to leaf is an assignment to (a subset of) PaXi Leaves are distributions over Xi given assignment of PaXi on path to leaf Interpretation of leaf: For specific assignment of PaXi on path to this leaf – Xi is independent of other parents Representation can be exponentially smaller than equivalent table

6 Tabular VE with Tree CPDs
If we turn a tree CPD into table “Sparsity” lost! Need inference approach that deals with tree CPD directly!

7 Local Structure 2: Determinism
If Battery Power = Dead, then Lights = OFF Determinism Battery Age Alternator Fan Belt ON OFF OK WEAK DEAD Lights Battery Power .99 .01 .20 .80 1 Charge Delivered Battery Fuel Pump Fuel Line Starter Gas Distributor Battery Power Gas Gauge Spark Plugs Radio Lights Engine Turn Over Engine Start

8 Determinism and inference
Determinism gives a little sparsity in table, but much bigger impact on inference Multiplying deterministic factor with other factor introduces many new zeros Operations related to theorem proving, e.g., unit resolution ON OFF OK WEAK DEAD Lights Battery Power .99 .01 .20 .80 1

9 Today’s Models … Often characterized by: Enabled by:
Richness in local structure (determinism, CSI) Massiveness in size (10,000’s variables) High connectivity (treewidth) Enabled by: High level modeling tools: relational, first order Advances in machine learning New application areas (synthesis): Bioinformatics (e.g. linkage analysis) Sensor networks Exploiting local structure a must!

10 Exact inference in large models is possible…
BN from a relational model

11 Recursive Conditioning
Treewidth complexity (worst case) Better than treewidth complexity with local structure Provides a framework for time-space tradeoffs Only quick intuition today, details in readings

12 The Computational Power of Assumptions
Battery Age Alternator Fan Belt Leak Charge Delivered Battery Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start A. Darwiche

13 The Computational Power of Assumptions
Battery Age Alternator Fan Belt Leak Charge Delivered Battery Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start A. Darwiche

14 Decomposition Gas A. Darwiche Battery Age Alternator Fan Belt Leak
Charge Delivered Battery Fuel Line Starter Gas Distributor Battery Power Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start A. Darwiche

15 Case Analysis + p p A. Darwiche Battery Age Alternator Fan Belt
Leak Leak Charge Delivered Battery Charge Delivered Fuel Line Battery Fuel Line Starter Gas Distributor Starter Gas Distributor Battery Power Battery Power Spark Plugs Gas Gauge Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start + p p A. Darwiche

16 Case Analysis * + pl pr p A. Darwiche Battery Age Alternator Fan Belt
Leak Leak Charge Delivered Battery Charge Delivered Fuel Line Battery Fuel Line Starter Gas Distributor Starter Gas Distributor Battery Power Battery Power Spark Plugs Gas Gauge Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start * + pl pr p A. Darwiche

17 Case Analysis * + * pl pr pl pr A. Darwiche Battery Age Alternator
Fan Belt Battery Age Alternator Fan Belt Leak Leak Charge Delivered Battery Charge Delivered Fuel Line Battery Fuel Line Starter Gas Distributor Starter Gas Distributor Battery Power Battery Power Spark Plugs Gas Gauge Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start * + * pl pr pl pr A. Darwiche

18 Case Analysis * + * pl pr pl pr A. Darwiche Alternator Fan Belt
Battery Age Battery Age Alternator Fan Belt Leak Charge Delivered Leak Battery Charge Delivered Fuel Line Battery Fuel Line Starter Gas Distributor Starter Gas Distributor Battery Power Battery Power Spark Plugs Gas Gauge Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start * + * pl pr pl pr A. Darwiche

19 Case Analysis * + * pl pr pl pr A. Darwiche Alternator Fan Belt
Battery Age Battery Age Alternator Fan Belt Leak Charge Delivered Leak Battery Charge Delivered Fuel Line Battery Fuel Line Starter Gas Distributor Starter Gas Distributor Battery Power Battery Power Spark Plugs Gas Gauge Spark Plugs Gas Gauge Radio Lights Engine Turn Over Engine Start Radio Lights Engine Turn Over Engine Start * + * pl pr pl pr A. Darwiche

20 Decomposition Tree A B C D E B A B C D E Cutset f(A) f(A,B) f(B,C)
f(C,D) f(B,D,E) A. Darwiche

21 Decomposition Tree A B C D E B A A B B C B C D E D Cutset f(A) f(A,B)
f(B,C) B C D E D f(C,D) f(B,D,E) A. Darwiche

22 Decomposition Tree A B C D E B A A B C Time: O(n exp(w log n))
Cutset B A A B C f(A) f(A,B) f(B,C) Time: O(n exp(w log n)) Space: Linear (using appropriate dtree) C D E D f(C,D) f(B,D,E) A. Darwiche

23 RC1 If T is a leaf node Return Lookup(T,e) Else RC1(T,e)
// compute probability of evidence e on dtree T If T is a leaf node Return Lookup(T,e) Else p := 0 for each instantiation c of cutset(T)-E do p := p + RC1(Tl,ec) RC1(Tr,ec) return p A. Darwiche

24 Lookup(T,e) QX|U : CPT associated with leaf T
If X is instantiated in e, then x: value of X in e u: value of U in e Return qx|u Else return 1 = Sx qx|u A. Darwiche

25 Caching A B C A B C C A B C D E F .27 .39 ABC A B C D F E Context
A. Darwiche

26 Caching A B C A B C C A B C D E F
Recursive Conditioning An any-space algorithm with treewidth complexity Darwiche AIJ-01 A B C D E F C .27 .39 Context Time: O(n exp(w)) Space: O(n exp(w)) (using appropriate dtree) ABC A B C A. Darwiche

27 RC2 RC2(T,e) If T is a leaf node, return Lookup(T,e)
y := instantiation of context(T) If cacheT[y] <> nil, return cacheT[y] p := 0 For each instantiation c of cutset(T)-E do p := p + RC2(Tl,ec) RC2(Tr,ec) cacheT[y] := p Return p A. Darwiche

28 Decomposition with Local Structure
X Independent of B, C given A A, B, C A X B C A. Darwiche

29 Decomposition with Local Structure
X Independent of B, C given A A, B, C A X B C A. Darwiche

30 Decomposition with Local Structure
X Independent of B, C given A A, B, C No need to consider an exponential number of cases (in the cutset size) given local structure A X B C A. Darwiche

31 Caching with Local Structure
B A Structural cache Caching with Local Structure B,C A A,B,C B C X A A. Darwiche

32 Caching with Local Structure
B A Structural cache Caching with Local Structure B,C A A,B,C B C X A A. Darwiche

33 Caching with Local Structure
B A Structural cache Caching with Local Structure No need to cache an exponential number of results (in the context size) given local structure B,C A C B A Non- Structural cache A,B,C A X B C A. Darwiche

34 Determinism… A, B, C A natural setup to incorporate SAT technology: A
X A A, B, C A natural setup to incorporate SAT technology: Unit resolution to: Derive values of variables Detect/skip inconsistent cases Dependency directed backtracking Clause learning More powerful than logical pre-processing A. Darwiche

35 CSI Summary Exploit local structure Significantly speed-up inference
Context-specific independence Determinism Significantly speed-up inference Tackle problems with tree-width in the thousands Acknowledgements Recursive conditioning slides courtesy of Adnan Darwiche Implementation available:

Download ppt "Context-specific independence"

Similar presentations

1 Undirected Graphical Models Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 29 th, 2008 Readings: K&F: 4.1, 4.2, 4.3, 4.4,

1 Undirected Graphical Models Graphical Models – Carlos Guestrin Carnegie Mellon University October 29 th, 2008 Readings: K&F: 4.1, 4.2, 4.3, 4.4,
A. Darwiche Sensitivity Analysis in Bayesian Networks Adnan Darwiche Computer Science Department

A. Darwiche Sensitivity Analysis in Bayesian Networks Adnan Darwiche Computer Science Department
Bayesian Networks CSE 473. © Daniel S. Weld 2 Last Time Basic notions Atomic events Probabilities Joint distribution Inference by enumeration Independence.

Bayesian Networks CSE 473. © Daniel S. Weld 2 Last Time Basic notions Atomic events Probabilities Joint distribution Inference by enumeration Independence.
A. Darwiche Bayesian Networks. A. Darwiche Reasoning Systems Diagnostics: Which component failed? Information retrieval: What document to retrieve? On-line.

A. Darwiche Bayesian Networks. A. Darwiche Reasoning Systems Diagnostics: Which component failed? Information retrieval: What document to retrieve? On-line.
1 Some Comments on Sebastiani et al Nature Genetics 37(4)2005.

1 Some Comments on Sebastiani et al Nature Genetics 37(4)2005.
MPE, MAP AND APPROXIMATIONS Lecture 10: Statistical Methods in AI/ML Vibhav Gogate The University of Texas at Dallas Readings: AD Chapter 10.

MPE, MAP AND APPROXIMATIONS Lecture 10: Statistical Methods in AI/ML Vibhav Gogate The University of Texas at Dallas Readings: AD Chapter 10.
BAYESIAN NETWORKS CHAPTER#4 Book: Modeling and Reasoning with Bayesian Networks Author : Adnan Darwiche Publisher: CambridgeUniversity Press 2009.

BAYESIAN NETWORKS CHAPTER#4 Book: Modeling and Reasoning with Bayesian Networks Author : Adnan Darwiche Publisher: CambridgeUniversity Press 2009.
Recursive Conditioning Adnan Darwiche Computer Science Department UCLA.

Recursive Conditioning Adnan Darwiche Computer Science Department UCLA.
From Rumelhart (1977) Toward Interactive Model of Reading

From Rumelhart (1977) "Toward Interactive Model of Reading"
A. Darwiche Inference in Bayesian Networks. A. Darwiche Query Types Pr: –Evidence: Pr(e) –Posterior marginals: Pr(x|e) for every X MPE: Most probable.

A. Darwiche Inference in Bayesian Networks. A. Darwiche Query Types Pr: –Evidence: Pr(e) –Posterior marginals: Pr(x|e) for every X MPE: Most probable.
Clique Trees Amr Ahmed October 23, 2008. Outline Clique Trees Representation Factorization Inference Relation with VE.

Clique Trees Amr Ahmed October 23, Outline Clique Trees Representation Factorization Inference Relation with VE.
Bayesian Network. Introduction Independence assumptions Seems to be necessary for probabilistic inference to be practical. Naïve Bayes Method Makes independence.

Bayesian Network. Introduction Independence assumptions Seems to be necessary for probabilistic inference to be practical. Naïve Bayes Method Makes independence.
A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graph.

A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graph.
A. Darwiche Bayesian Networks. A. Darwiche Bayesian Network Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine.

A. Darwiche Bayesian Networks. A. Darwiche Bayesian Network Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine.
. Approximate Inference Slides by Nir Friedman. When can we hope to approximate? Two situations: u Highly stochastic distributions “Far” evidence is discarded.

. Approximate Inference Slides by Nir Friedman. When can we hope to approximate? Two situations: u Highly stochastic distributions “Far” evidence is discarded.
A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graphs.

A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graphs.
. PGM 2002/3 – Tirgul6 Approximate Inference: Sampling.

. PGM 2002/3 – Tirgul6 Approximate Inference: Sampling.
Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16 th, 2006 Readings: K&F: 4.1, 4.2, 4.3, 4.4,

Context-specific independence Graphical Models – Carlos Guestrin Carnegie Mellon University October 16 th, 2006 Readings: K&F: 4.1, 4.2, 4.3, 4.4,
Knowledge Representation Use of logic. Artificial agents need Knowledge and reasoning power Can combine GK with current percepts Build up KB incrementally.

Knowledge Representation Use of logic. Artificial agents need Knowledge and reasoning power Can combine GK with current percepts Build up KB incrementally.
1 Bayesian Param. Learning Bayesian Structure Learning Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 6 th, 2008 Readings:

1 Bayesian Param. Learning Bayesian Structure Learning Graphical Models – Carlos Guestrin Carnegie Mellon University October 6 th, 2008 Readings:

Similar presentations

Ads by Google