1. CSE 5731 Lecture 24B Markov Chain Monte Carlo CSE 573 Artificial Intelligence I Henry Kautz Fall 2001

2. CSE 5732 Announcements Today – Bayesian stuff Next week – decision trees & neural nets Monday Dec 10 th – guest lecture by Eric Horvitz, head of Microsoft Research Adaptive Systems group Wednesday Dec 12 th – wrap up, and preview of 574 by Oren Etzioni (written hm #4 due) Monday Dec 15 th – rest of hm #4 due Wednesday Dec 19 th, 8:30-10:20 am – final exam, open book & notebooks. Some people asked to take the exam on the 18 th instead. If you are one of these, send me email to confirm this, with any constraints on your time on the 18 th.

3. CSE 5733 Gibbs Sampling Fix the values of observed variables Set the values of all non-observed variables randomly Perform a random walk through the space of complete variable assignments. On each move: 1.Pick a variable X 2.Calculate Pr(X=true | all other variables) 3.Set X to true with that probability Repeat many times. Frequency with which any variable X is true is it’s posterior probability. Converges to true posterior when frequencies stop changing significantly stable distribution, mixing

4. CSE 5734 Markov Blanket Sampling How to calculate Pr(X=true | all other variables) ? Recall: a variable is independent of all others given it’s Markov Blanket parents children other parents of children So problem becomes calculating Pr(X=true | MB(X)) We solve this sub-problem exactly Fortunately, it is easy to solve

5. CSE 5735 Intuition A X B C

6. CSE 5736 Example Evidence: S=true, B=true smoking heart disease lung disease shortness of breath SP(s) T0.6 F0.1 SP(l) T0.8 F0.1 HLP(b) TT0.9 TF0.8 FT0.7 FF0.1 P(s) 0.2

7. CSE 5737 Example 2 Evidence: S=true, B=true Randomly set H=false, L=true smoking heart disease lung disease shortness of breath SP(h) T0.6 F0.1 SP(l) T0.8 F0.1 HLP(b) TT0.9 TF0.8 FT0.7 FF0.1 P(s) 0.2

8. CSE 5738 Example 3 Sample H: P(h|s,l,b)=  P(h|s)P(b|h,l) =  (0.6)(0.9)=  0.54 P(  h|s,l,b)=  P(  h|s)P(b|  h,l) =  (0.4)(0.7)=  0.28 Normalize: 0.54/(0.54+0.28)=0.66 Flip coin: H becomes true (maybe) smoking heart disease lung disease shortness of breath SP(h) T0.6 F0.1 SP(l) T0.8 F0.1 HLP(b) TT0.9 TF0.8 FT0.7 FF0.1 P(s) 0.2

9. CSE 5739 Example 4 Sample L: P(l|s,h,b)=  P(l|s)P(b|h,l) =  (0.8)(0.9)=  0.72 P(  l|s,h,b)=  P(  l|s)P(b| h,  l) =  (0.2)(0.8)=  0.16 Normalize: 0.72/(0.72+0.16)=0.82 Flip coin: … smoking heart disease lung disease shortness of breath SP(h) T0.6 F0.1 SP(l) T0.8 F0.1 HLP(b) TT0.9 TF0.8 FT0.7 FF0.1 P(s) 0.2

10. CSE 57310 Example 5: Different Evidence Evidence: S=true, B=false smoking heart disease lung disease shortness of breath SP(s) T0.6 F0.1 SP(l) T0.8 F0.1 HLP(b) TT0.9 TF0.8 FT0.7 FF0.1 P(s) 0.2

11. CSE 57311 Example 6 Evidence: S=true, B=false Randomly set H=false, L=true smoking heart disease lung disease shortness of breath SP(h) T0.6 F0.1 SP(l) T0.8 F0.1 HLP(b) TT0.9 TF0.8 FT0.7 FF0.1 P(s) 0.2

12. CSE 57312 Example 7 Sample H: P(h|s,l,  b)=  P(h|s)P(  b|h,l) =  (0.6)(0.1)=  0.06 P(  h|s,l,  b)=  P(  h|s)P(  b|  h,l) =  (0.4)(0.3)=  0.12 Normalize: 0.06/(0.06+0.12)=0.33 Flip coin: H stays false (maybe) smoking heart disease lung disease shortness of breath SP(h) T0.6 F0.1 SP(l) T0.8 F0.1 HLP(b) TT0.9 TF0.8 FT0.7 FF0.1 P(s) 0.2

13. CSE 57313 Example 8 Sample L: P(l|s,  h,  b)=  P(l|s)P(  b|  h,l) =  (0.8)(0.3)=  0.24 P(  l|s,  h,  b)=  P(  l|s)P(  b|  h,  l) =  (0.2)(0.9)=  0.18 Normalize: 0.24/(0.24+0.18)=0.75 Flip coin: … smoking heart disease lung disease shortness of breath SP(h) T0.6 F0.1 SP(l) T0.8 F0.1 HLP(b) TT0.9 TF0.8 FT0.7 FF0.1 P(s) 0.2

14. CSE 57314 Next slide goes with Lecture 26 – Bayesian learning

15. CSE 57315 Gradient Ascent Example Note: P h (X) shorthand for P(X | h), where h is the current Bayes net