Presentation is loading. Please wait.

Presentation is loading. Please wait.

Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16 th, 2006 Readings: K&F: 4.1, 4.2, 4.3, 4.4,

Published byDeirdre Mason Modified over 9 years ago

Similar presentations

Presentation on theme: "Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16 th, 2006 Readings: K&F: 4.1, 4.2, 4.3, 4.4,"— Presentation transcript:

1 Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16 th, 2006 Readings: K&F: 4.1, 4.2, 4.3, 4.4, 8.4, 8.5, 8.6 “Recursive Conditioning”, Adnan Darwiche. In Artificial Intelligence Journal, 125:1, pp. 5-41

2 Global Structure: Treewidth w

3 Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Local Structure 1: Context specific indepencence

4 Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Context Specific Independence (CSI) After observing a variable, some vars become independent Local Structure 1: Context specific indepencence

5 CSI example: Tree CPD Represent P(X i |Pa Xi ) using a decision tree  Path to leaf is an assignment to (a subset of) Pa Xi  Leaves are distributions over X i given assignment of Pa Xi on path to leaf Interpretation of leaf:  For specific assignment of Pa Xi on path to this leaf – X i is independent of other parents Representation can be exponentially smaller than equivalent table ApplySATLetter Job

6 Tabular VE with Tree CPDs If we turn a tree CPD into table  “Sparsity” lost! Need inference approach that deals with tree CPD directly!

7 Local Structure 2: Determinism Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start ON OFF OK WEAK DEAD Lights Battery Power.99.01.20.80 01 If Battery Power = Dead, then Lights = OFF Determinism

8 Determinism and inference Determinism gives a little sparsity in table, but much bigger impact on inference Multiplying deterministic factor with other factor introduces many new zeros  Operations related to theorem proving, e.g., unit resolution ON OFF OK WEAK DEAD Lights Battery Power.99.01.20.80 01

9 Today’s Models … Often characterized by:  Richness in local structure (determinism, CSI)  Massiveness in size (10,000’s variables)  High connectivity (treewidth) Enabled by:  High level modeling tools: relational, first order  Advances in machine learning  New application areas (synthesis): Bioinformatics (e.g. linkage analysis) Sensor networks Exploiting local structure a must!

10 Exact inference in large models is possible… BN from a relational model

11 Recursive Conditioning Treewidth complexity (worst case) Better than treewidth complexity with local structure Provides a framework for time-space tradeoffs Only quick intuition today, details in readings

12 A. Darwiche The Computational Power of Assumptions Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start

13 A. Darwiche Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start The Computational Power of Assumptions

14 A. Darwiche Decomposition Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start

15 A. Darwiche Case Analysis Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start pp +

16 A. Darwiche Case Analysis Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start plpl p + prpr *

17 A. Darwiche Case Analysis Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start plpl + prpr * plpl prpr *

18 A. Darwiche Case Analysis Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start plpl + prpr * plpl prpr *

19 A. Darwiche Case Analysis Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start plpl + prpr * plpl prpr *

20 A. Darwiche Decomposition Tree A BCDE A A B BC C D D B E B f(A) f(A,B) f(B,C) f(C,D) f(B,D,E) Cutset

21 A. Darwiche Decomposition Tree A BCDE A A B BC C D D B E B f(A) f(A,B) f(B,C) f(C,D) f(B,D,E) Cutset

22 A. Darwiche Decomposition Tree A BCDE A A B C C D D E B Time: O(n exp(w log n)) Space: Linear (using appropriate dtree) f(A) f(A,B) f(B,C) f(C,D) f(B,D,E) Cutset

23 A. Darwiche RC1(T,e) // compute probability of evidence e on dtree T If T is a leaf node Return Lookup(T,e) Else p := 0 for each instantiation c of cutset(T)-E do p := p + RC1(Tl,ec) RC1(Tr,ec) return p RC1

24 A. Darwiche Lookup(T,e)  X|U : CPT associated with leaf T If X is instantiated in e, then x: value of X in e u: value of U in e Return  x|u Else return 1 =  x  x|u

25 A. Darwiche Caching A BCDEF A A B BC C D D E E F A B C ABCABC ABCABC A B C C C.27.39 Context

26 A. Darwiche Caching A BCDEF A A B BC C D D E E F A B C ABCABC ABCABC Time: O(n exp(w)) Space: O(n exp(w)) (using appropriate dtree) A B C C C.27.39 Context Recursive Conditioning An any-space algorithm with treewidth complexity Darwiche AIJ-01

27 A. Darwiche RC2(T,e) If T is a leaf node, return Lookup(T,e) y := instantiation of context(T) If cache T [y] <> nil, return cache T [y] p := 0 For each instantiation c of cutset(T)-E do p := p + RC2(T l,ec) RC2(T r,ec) cache T [y] := p Return p RC2

28 A. Darwiche Decomposition with Local Structure B C X A A, B, C X Independent of B, C given A

29 A. Darwiche Decomposition with Local Structure B C X A A, B, C X Independent of B, C given A

30 A. Darwiche Decomposition with Local Structure B C X A A, B, C X Independent of B, C given A No need to consider an exponential number of cases (in the cutset size) given local structure

31 A. Darwiche Caching with Local Structure B C X A A,B,C B,C A CBA CBA CBA CBA CBA CBA CBA CBA Structural cache

32 A. Darwiche Caching with Local Structure B C X A A,B,C B,C A CBA CBA CBA CBA CBA CBA CBA CBA Structural cache

33 A. Darwiche Caching with Local Structure B C X A A,B,C B,C A CBA CBA CBA CBA A Non- Structural cache CBA CBA CBA CBA CBA CBA CBA CBA Structural cache No need to cache an exponential number of results (in the context size) given local structure

34 A. Darwiche Determinism… B C X A A, B, C A natural setup to incorporate SAT technology: Unit resolution to: Derive values of variables Detect/skip inconsistent cases Dependency directed backtracking Clause learning

35 CSI Summary Exploit local structure  Context-specific independence  Determinism Significantly speed-up inference  Tackle problems with tree-width in the thousands Acknowledgements  Recursive conditioning slides courtesy of Adnan Darwiche  Implementation available: http://reasoning.cs.ucla.edu/ace

Download ppt "Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16 th, 2006 Readings: K&F: 4.1, 4.2, 4.3, 4.4,"

Similar presentations

1 Undirected Graphical Models Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 29 th, 2008 Readings: K&F: 4.1, 4.2, 4.3, 4.4,

1 Undirected Graphical Models Graphical Models – Carlos Guestrin Carnegie Mellon University October 29 th, 2008 Readings: K&F: 4.1, 4.2, 4.3, 4.4,
A. Darwiche Sensitivity Analysis in Bayesian Networks Adnan Darwiche Computer Science Department

A. Darwiche Sensitivity Analysis in Bayesian Networks Adnan Darwiche Computer Science Department
A. Darwiche Bayesian Networks. A. Darwiche Reasoning Systems Diagnostics: Which component failed? Information retrieval: What document to retrieve? On-line.

A. Darwiche Bayesian Networks. A. Darwiche Reasoning Systems Diagnostics: Which component failed? Information retrieval: What document to retrieve? On-line.
1 Some Comments on Sebastiani et al Nature Genetics 37(4)2005.

1 Some Comments on Sebastiani et al Nature Genetics 37(4)2005.
BAYESIAN NETWORKS CHAPTER#4 Book: Modeling and Reasoning with Bayesian Networks Author : Adnan Darwiche Publisher: CambridgeUniversity Press 2009.

BAYESIAN NETWORKS CHAPTER#4 Book: Modeling and Reasoning with Bayesian Networks Author : Adnan Darwiche Publisher: CambridgeUniversity Press 2009.
Identifying Conditional Independencies in Bayes Nets Lecture 4.

Identifying Conditional Independencies in Bayes Nets Lecture 4.
Recursive Conditioning Adnan Darwiche Computer Science Department UCLA.

Recursive Conditioning Adnan Darwiche Computer Science Department UCLA.
From Rumelhart (1977) Toward Interactive Model of Reading

From Rumelhart (1977) "Toward Interactive Model of Reading"
A. Darwiche Inference in Bayesian Networks. A. Darwiche Query Types Pr: –Evidence: Pr(e) –Posterior marginals: Pr(x|e) for every X MPE: Most probable.

A. Darwiche Inference in Bayesian Networks. A. Darwiche Query Types Pr: –Evidence: Pr(e) –Posterior marginals: Pr(x|e) for every X MPE: Most probable.
Clique Trees Amr Ahmed October 23, 2008. Outline Clique Trees Representation Factorization Inference Relation with VE.

Clique Trees Amr Ahmed October 23, Outline Clique Trees Representation Factorization Inference Relation with VE.
Bayesian Network. Introduction Independence assumptions Seems to be necessary for probabilistic inference to be practical. Naïve Bayes Method Makes independence.

Bayesian Network. Introduction Independence assumptions Seems to be necessary for probabilistic inference to be practical. Naïve Bayes Method Makes independence.
A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graph.

A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graph.
A. Darwiche Bayesian Networks. A. Darwiche Bayesian Network Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine.

A. Darwiche Bayesian Networks. A. Darwiche Bayesian Network Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine.
5/25/2005EE562 EE562 ARTIFICIAL INTELLIGENCE FOR ENGINEERS Lecture 16, 6/1/2005 University of Washington, Department of Electrical Engineering Spring 2005.

5/25/2005EE562 EE562 ARTIFICIAL INTELLIGENCE FOR ENGINEERS Lecture 16, 6/1/2005 University of Washington, Department of Electrical Engineering Spring 2005.
. Approximate Inference Slides by Nir Friedman. When can we hope to approximate? Two situations: u Highly stochastic distributions “Far” evidence is discarded.

. Approximate Inference Slides by Nir Friedman. When can we hope to approximate? Two situations: u Highly stochastic distributions “Far” evidence is discarded.
A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graphs.

A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graphs.
1 Bayesian Networks Chapter 14.1-14.2; 14.4 CS 63 Adapted from slides by Tim Finin and Marie desJardins. Some material borrowed from Lise Getoor.

1 Bayesian Networks Chapter ; 14.4 CS 63 Adapted from slides by Tim Finin and Marie desJardins. Some material borrowed from Lise Getoor.
Knowledge Representation Use of logic. Artificial agents need Knowledge and reasoning power Can combine GK with current percepts Build up KB incrementally.

Knowledge Representation Use of logic. Artificial agents need Knowledge and reasoning power Can combine GK with current percepts Build up KB incrementally.
1 Bayesian Param. Learning Bayesian Structure Learning Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 6 th, 2008 Readings:

1 Bayesian Param. Learning Bayesian Structure Learning Graphical Models – Carlos Guestrin Carnegie Mellon University October 6 th, 2008 Readings:
1 Variable Elimination Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 11 th, 2006 Readings: K&F: 8.1, 8.2, 8.3, 8.7.1.

1 Variable Elimination Graphical Models – Carlos Guestrin Carnegie Mellon University October 11 th, 2006 Readings: K&F: 8.1, 8.2, 8.3,

Similar presentations

Ads by Google