Presentation is loading. Please wait.

Presentation is loading. Please wait.

Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16 th, 2006 Readings: K&F: 4.1, 4.2, 4.3, 4.4,

Similar presentations


Presentation on theme: "Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16 th, 2006 Readings: K&F: 4.1, 4.2, 4.3, 4.4,"— Presentation transcript:

1 Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16 th, 2006 Readings: K&F: 4.1, 4.2, 4.3, 4.4, 8.4, 8.5, 8.6 “Recursive Conditioning”, Adnan Darwiche. In Artificial Intelligence Journal, 125:1, pp. 5-41

2 Global Structure: Treewidth w

3 Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Local Structure 1: Context specific indepencence

4 Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start Context Specific Independence (CSI) After observing a variable, some vars become independent Local Structure 1: Context specific indepencence

5 CSI example: Tree CPD Represent P(X i |Pa Xi ) using a decision tree  Path to leaf is an assignment to (a subset of) Pa Xi  Leaves are distributions over X i given assignment of Pa Xi on path to leaf Interpretation of leaf:  For specific assignment of Pa Xi on path to this leaf – X i is independent of other parents Representation can be exponentially smaller than equivalent table ApplySATLetter Job

6 Tabular VE with Tree CPDs If we turn a tree CPD into table  “Sparsity” lost! Need inference approach that deals with tree CPD directly!

7 Local Structure 2: Determinism Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine Turn Over Gas Gauge Gas Fuel Pump Fuel Line Distributor Spark Plugs Engine Start ON OFF OK WEAK DEAD Lights Battery Power.99.01.20.80 01 If Battery Power = Dead, then Lights = OFF Determinism

8 Determinism and inference Determinism gives a little sparsity in table, but much bigger impact on inference Multiplying deterministic factor with other factor introduces many new zeros  Operations related to theorem proving, e.g., unit resolution ON OFF OK WEAK DEAD Lights Battery Power.99.01.20.80 01

9 Today’s Models … Often characterized by:  Richness in local structure (determinism, CSI)  Massiveness in size (10,000’s variables)  High connectivity (treewidth) Enabled by:  High level modeling tools: relational, first order  Advances in machine learning  New application areas (synthesis): Bioinformatics (e.g. linkage analysis) Sensor networks Exploiting local structure a must!

10 Exact inference in large models is possible… BN from a relational model

11 Recursive Conditioning Treewidth complexity (worst case) Better than treewidth complexity with local structure Provides a framework for time-space tradeoffs Only quick intuition today, details in readings

12 A. Darwiche The Computational Power of Assumptions Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start

13 A. Darwiche Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start The Computational Power of Assumptions

14 A. Darwiche Decomposition Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start

15 A. Darwiche Case Analysis Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start pp +

16 A. Darwiche Case Analysis Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start plpl p + prpr *

17 A. Darwiche Case Analysis Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start plpl + prpr * plpl prpr *

18 A. Darwiche Case Analysis Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start plpl + prpr * plpl prpr *

19 A. Darwiche Case Analysis Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start Battery Age AlternatorFan Belt Battery Charge Delivered Battery Power Starter Radio Lights Engine Turn Over Gas Gauge Gas Leak Fuel Line Distributor Spark Plugs Engine Start plpl + prpr * plpl prpr *

20 A. Darwiche Decomposition Tree A BCDE A A B BC C D D B E B f(A) f(A,B) f(B,C) f(C,D) f(B,D,E) Cutset

21 A. Darwiche Decomposition Tree A BCDE A A B BC C D D B E B f(A) f(A,B) f(B,C) f(C,D) f(B,D,E) Cutset

22 A. Darwiche Decomposition Tree A BCDE A A B C C D D E B Time: O(n exp(w log n)) Space: Linear (using appropriate dtree) f(A) f(A,B) f(B,C) f(C,D) f(B,D,E) Cutset

23 A. Darwiche RC1(T,e) // compute probability of evidence e on dtree T If T is a leaf node Return Lookup(T,e) Else p := 0 for each instantiation c of cutset(T)-E do p := p + RC1(Tl,ec) RC1(Tr,ec) return p RC1

24 A. Darwiche Lookup(T,e)  X|U : CPT associated with leaf T If X is instantiated in e, then x: value of X in e u: value of U in e Return  x|u Else return 1 =  x  x|u

25 A. Darwiche Caching A BCDEF A A B BC C D D E E F A B C ABCABC ABCABC A B C C C.27.39 Context

26 A. Darwiche Caching A BCDEF A A B BC C D D E E F A B C ABCABC ABCABC Time: O(n exp(w)) Space: O(n exp(w)) (using appropriate dtree) A B C C C.27.39 Context Recursive Conditioning An any-space algorithm with treewidth complexity Darwiche AIJ-01

27 A. Darwiche RC2(T,e) If T is a leaf node, return Lookup(T,e) y := instantiation of context(T) If cache T [y] <> nil, return cache T [y] p := 0 For each instantiation c of cutset(T)-E do p := p + RC2(T l,ec) RC2(T r,ec) cache T [y] := p Return p RC2

28 A. Darwiche Decomposition with Local Structure B C X A A, B, C X Independent of B, C given A

29 A. Darwiche Decomposition with Local Structure B C X A A, B, C X Independent of B, C given A

30 A. Darwiche Decomposition with Local Structure B C X A A, B, C X Independent of B, C given A No need to consider an exponential number of cases (in the cutset size) given local structure

31 A. Darwiche Caching with Local Structure B C X A A,B,C B,C A CBA CBA CBA CBA CBA CBA CBA CBA Structural cache

32 A. Darwiche Caching with Local Structure B C X A A,B,C B,C A CBA CBA CBA CBA CBA CBA CBA CBA Structural cache

33 A. Darwiche Caching with Local Structure B C X A A,B,C B,C A CBA CBA CBA CBA A Non- Structural cache CBA CBA CBA CBA CBA CBA CBA CBA Structural cache No need to cache an exponential number of results (in the context size) given local structure

34 A. Darwiche Determinism… B C X A A, B, C A natural setup to incorporate SAT technology: Unit resolution to: Derive values of variables Detect/skip inconsistent cases Dependency directed backtracking Clause learning

35 CSI Summary Exploit local structure  Context-specific independence  Determinism Significantly speed-up inference  Tackle problems with tree-width in the thousands Acknowledgements  Recursive conditioning slides courtesy of Adnan Darwiche  Implementation available: http://reasoning.cs.ucla.edu/ace


Download ppt "Context-specific independence Graphical Models – 10708 Carlos Guestrin Carnegie Mellon University October 16 th, 2006 Readings: K&F: 4.1, 4.2, 4.3, 4.4,"

Similar presentations


Ads by Google