Download presentation
Presentation is loading. Please wait.
Published byAugustine Edwards Modified over 9 years ago
1
B AYESIAN N ETWORKS
2
S OME A PPLICATIONS OF BN Medical diagnosis Troubleshooting of hardware/software systems Fraud/uncollectible debt detection Data mining Analysis of genetic sequences Data interpretation, computer vision, image understanding
3
M ORE C OMPLICATED S INGLY -C ONNECTED B ELIEF N ET Radio Battery SparkPlugs Starts Gas Moves
4
Region = {Sky, Tree, Grass, Rock} R2 R4 R3 R1 Above
6
C ALCULATION OF J OINT P ROBABILITY BEP(A| … ) TTFFTTFF TFTFTFTF 0.95 0.94 0.29 0.001 BurglaryEarthquake Alarm MaryCallsJohnCalls P(B) 0.001 P(E) 0.002 AP(J|…) TFTF 0.90 0.05 AP(M|…) TFTF 0.70 0.01 P(J M A B E) = P(J|A)P(M|A)P(A| B, E)P( B)P( E) = 0.9 x 0.7 x 0.001 x 0.999 x 0.998 = 0.00062 P(x 1 x 2 … x n ) = i=1,…,n P(x i |parents(X i )) full joint distribution table
7
W HAT DOES THE BN ENCODE ? Burglary Earthquake JohnCalls MaryCalls | Alarm JohnCalls Burglary | Alarm JohnCalls Earthquake | Alarm MaryCalls Burglary | Alarm MaryCalls Earthquake | Alarm BurglaryEarthquake Alarm MaryCallsJohnCalls A node is independent of its non-descendents, given its parents
8
P ROBABILISTIC I NFERENCE Is the following problem…. Given: A belief state P(X 1,…,X n ) in some form (e.g., a Bayes net or a joint probability table) A query variable indexed by q Some subset of evidence variables indexed by e 1,…,e k Find: P(X q | X e1,…, X ek )
9
BEP(A| … ) TTFFTTFF TFTFTFTF 0.95 0.94 0.29 0.001 BurglaryEarthquake Alarm MaryCallsJohnCalls P(B) 0.001 P(E) 0.002 AP(J|…) TFTF 0.90 0.05 AP(M|…) TFTF 0.70 0.01 T OP -D OWN INFERENCE : RECURSIVE COMPUTATION OF A LL M ARGINALS D OWNSTREAM OF E VIDENCE P(A|E) = P(A|B,E)P(B) + P(A| B,E)P( B) P(J|E) = P(J|A,E)P(A) + P(J| A,E)P( A) P(M|E) = P(M|A,E)P(A) + P(M| A,E)P( A)
10
T OP -D OWN INFERENCE Only works if the graph of ancestors of a variable is a polytree Evidence given on ancestor(s) of the query variable Efficient: O(d 2 k ) time, where d is the number of ancestors of a variable, with k a bound on # of parents Evidence on an ancestor cuts off influence of portion of graph above evidence node
11
Q UERYING THE BN The BN gives P(T|C) P(C|T) can be computed using Bayes rule: P(A|B) = P(B|A) P(A) / P(B) Cavity Toothache P(C) 0.1 CP(T|C) TFTF 0.4 0.01111
12
Q UERYING THE BN The BN gives P(T|C) What about P(C|T)? P(Cavity|Toothache) = P(Toothache|Cavity) P(Cavity) P(Toothache) [Bayes’ rule] Querying a BN is just applying Bayes’ rule on a larger scale… Cavity Toothache P(C) 0.1 CP(T|C) TFTF 0.4 0.01111 Denominator computed by summing out numerator over Cavity and Cavity
13
N AÏVE B AYES M ODELS P(Cause,Effect 1,…,Effect n ) = P(Cause) i P(Effect i | Cause) Cause Effect 1 Effect 2 Effect n
14
N AÏVE B AYES C LASSIFIER P(Class,Feature 1,…,Feature n ) = P(Class) i P(Feature i | Class) Class Feature 1 Feature 2 Feature n P(C|F 1,….,F k ) = P(C,F 1,….,F k )/P(F 1,….,F k ) = 1/Z P(C) i P(Fi|C) Given features, what class? Spam / Not Spam English / French/ Latin … Word occurrences
15
C OMMENTS ON N AÏVE B AYES MODELS Very scalable (thousands or millions of features!), easy to implement Easily handles missing data: just ignore the feature Conditional independence of features is main weakness. What if two features were actually correlated? Many features?
16
V ARIABLE E LIMINATION : P ROBABILISTIC I NFERENCE IN G ENERAL N ETWORKS Coherence DifficultyIntelligence Happy GradeSAT Letter Job Basic idea: Eliminate “nuisance” variables one at a time via marginalization Example: P(J) Elimination order: C,D,I,H,G,S,L
17
Coherence DifficultyIntelligence Happy GradeSAT Letter Job P(D|C) P(C) P(I) P(G|I,D) P(H|G,J) P(J|S,L) P(S|I)
18
Coherence DifficultyIntelligence Happy GradeSAT Letter Job P(D|C) P(C) E LIMINATING C
19
DifficultyIntelligence Happy GradeSAT Letter Job P(D)= c P(D|C)P(C) C IS E LIMINATED, GIVING A NEW FACTOR OVER D
20
DifficultyIntelligence Happy GradeSAT Letter Job P(D) E LIMINATING D P(G|I,D)
21
Intelligence Happy GradeSAT Letter Job D IS E LIMINATED, GIVING A NEW F ACTOR OVER G, I P(G|I)= d P(G|I,d)P(d)
22
Intelligence Happy GradeSAT Letter Job E LIMINATING I P(G|I)P(S|I) P(I)
23
Happy GradeSAT Letter Job I IS E LIMINATED, PRODUCING A NEW F ILL E DGE AND F ACTOR OVER G AND S P(G,S)= i P(i)P(G|i)P(S|i) New undirected fill edge
24
Happy GradeSAT Letter Job E LIMINATING H P(H|G,J)
25
Happy GradeSAT Letter Job E LIMINATING H P(H|G,J) f GJ (G,J)= h P(h|G,J)=1
26
GradeSAT Letter Job H IS E LIMINATED, PRODUCING A NEW F ILL E DGE AND F ACTOR OVER G, J f GJ (G,J)
27
GradeSAT Letter Job E LIMINATING G f GJ (G,J) P(G,S) P(L|G)
28
GradeSAT Letter Job G IS E LIMINATED, M AKING A NEW T RINARY F ACTOR OVER S,L,J AND A NEW F ILL E DGE f GJ (G,J) P(G,S) P(L|G) f SLJ (S,L,J) = g P(g,S) P(L|g) f GJ (g,J)
29
SAT Letter Job E LIMINATING S f SLJ (S,L,J) P(J|S,L)
30
SAT Letter Job S IS E LIMINATED, CREATING A NEW FACTOR OVER L, J f SLJ (S,L,J) P(J|S,L) f LJ (L,J) = s f SLJ (s,L,J) P(J|s,L)
31
Letter Job E LIMINATING L f LJ (L,J)
32
Letter Job L IS ELIMINATED, GIVING A NEW FACTOR OVER J ( WHICH TURNS OUT TO BE P(J)) f LJ (L,J) P(J)= l f LJ (l,J)
33
Job L IS ELIMINATED, GIVING A NEW FACTOR OVER J ( WHICH TURNS OUT TO BE P(J)) P(J)
34
J OINT DISTRIBUTION P( X ) = P(C)P(D|C)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J) Apply elimination ordering C,D,I,H,G,S,L
35
G OING THROUGH VE P( X ) = P(C)P(D|C)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J) Apply elimination ordering C,D,I,H,G,S,L f D (D)= C P(C)P(D|C)
36
G OING THROUGH VE C P( X ) = f D (D)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J) Apply elimination ordering C,D,I,H,G,S,L f D (D)= C P(C)P(D|C)
37
G OING THROUGH VE C P( X ) = f D (D)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J) Apply elimination ordering C,D,I,H,G,S,L f GI (G,I)= D f D (D)P(G|I,D)
38
G OING THROUGH VE C,D P( X ) = f GI (G,I)P(I)P(S|I)P(L|G) P(J|L,S)P(H|G,J) Apply elimination ordering C,D,I,H,G,S,L f GI (G,I)= D f D (D)P(G|I,D)
39
G OING THROUGH VE C,D P( X ) = f GI (G,I)P(I)P(S|I)P(L|G) P(J|L,S)P(H|G,J) Apply elimination ordering C,D,I,H,G,S,L f GS (G,S)= I f GI (G,I)P(I)P(S|I)
40
G OING THROUGH VE C,D,I P( X ) = f GS (G,S)P(L|G)P(J|L,S)P(H|G,J) Apply elimination ordering C,D,I,H,G,S,L f GS (G,S)= I f GI (G,I)P(I)P(S|I)
41
G OING THROUGH VE C,D,I P( X ) = f GS (G,S)P(L|G)P(J|L,S)P(H|G,J) Apply elimination ordering C,D,I,H,G,S,L f GJ (G,J)= H P(H|G,J) What values does this factor store?
42
G OING THROUGH VE C,D,I,H P( X ) = f GS (G,S)P(L|G)P(J|L,S)f GJ (G,J) Apply elimination ordering C,D,I,H,G,S,L f GJ (G,J)= H P(H|G,J)
43
G OING THROUGH VE C,D,I,H P( X ) = f GS (G,S)P(L|G)P(J|L,S)f GJ (G,J) Apply elimination ordering C,D,I,H,G,S,L f SLJ (S,L,J)= G f GS (G,S)P(L|G)f GJ (G,J)
44
G OING THROUGH VE C,D,I,H,G P( X ) = f SLJ (S,L,J)P(J|L,S) Apply elimination ordering C,D,I,H,G,S,L f SLJ (S,L,J)= G f GS (G,S)P(L|G)f GJ (G,J)
45
G OING THROUGH VE C,D,I,H,G P( X ) = f SLJ (S,L,J)P(J|L,S) Apply elimination ordering C,D,I,H,G,S,L f LJ (L,J)= S f SLJ (S,L,J)P(J|L,S)
46
G OING THROUGH VE C,D,I,H,G,S P( X ) = f LJ (L,J) Apply elimination ordering C,D,I,H,G,S,L f LJ (L,J)= S f SLJ (S,L,J)
47
G OING THROUGH VE C,D,I,H,G,S P( X ) = f LJ (L,J) Apply elimination ordering C,D,I,H,G,S,L f J (J)= L f LJ (L,J)
48
G OING THROUGH VE C,D,I,H,G,S,L P( X ) = f J (J) Apply elimination ordering C,D,I,H,G,S,L f J (J)= L f LJ (L,J)
49
O RDER -D EPENDENCE
50
O RDER MATTERS Coherence DifficultyIntelligence Happy GradeSAT Letter Job If we were to eliminate G first, we’d create a factor over D, I, L, and H (their distribution becomes coupled)
51
E LIMINATION O RDER M ATTERS Coherence DifficultyIntelligence Happy SAT Letter Job If we were to eliminate G first, we’d create a factor over D, I, L, and H (their distribution becomes coupled) f DILH (D,I,L,H) = g P(g|D,I) *P(L|g)*P(H|g)
52
C OMPLEXITY In polytree networks where each node has at most k parents, O(n2 k ) with top-down ordering In other networks, intermediate factors may involve more than k terms Worst case O(n) Good ordering heuristics exist, e.g. min-neighbors, min-fill Exact inference on non-polytree networks is NP- hard!
53
V ARIABLE E LIMINATION WITH E VIDENCE Coherence DifficultyIntelligence Happy GradeSAT Letter Job Two-step process: 1. Find P(X,e) with VE 2. Normalize by P(e) Example: P(J|H) 1.Run VE, enforcing H=T when H is eliminated. 2.This produces P(J,H=T) (a factor over J) 3.P(J=T|H=T) = P(J=T,H=T) / (P(J=T,H=T)+P(J=F,H=T))
54
R ECAP Exact inference techniques Top-down inference: linear time when ancestors of query variable are polytree, evidence is on ancestors Bottom-up inference in Naïve Bayes models General inference using Variable Elimination (We’ll come back to approximation techniques in a week.)
55
N EXT TIME Learning Bayes nets R&N 20.1-2
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.