1. Factor Graphs, Variable Elimination, MLEs Joseph Gonzalez TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A AA A A

2. Review

3. Draw the Bayesian Network: A A B B C C D D E E Parameters Binary Variables ABCDEABCDE

4. Draw the Factor Graph A A B B C C D D E E f1f2f3f4f6f5 Factors Variables

5. Write down the Equation A A B B C C D D E E f1f2f3f4

7. Compute Z A A B B C C D D E E f1f2f3f4 What’s wrong with?

8. Compute Z ABCDEProduct 00000 00001 00010 00011 00100 00101 00110 00111 01000 01001 01010 01011 01100 01101 01110 01111 00000 00001 00010 00011 00100 00101 00110 00111 01000 01001 01010 01011 01100 01101 01110 01111

9. Variable Elimination = Being Clever A A B B C C D D E E f1f2f3f4 DEf4(d,e) 004 012 101 115 Dg1 (d) 0 1

10. Variable Elimination = Being Clever A A B B C C D D E E f1f2f3f4 BCf2(b,c) 003 012 101 114 CDf3(c,d) 002 011 103 114 BDg1(b,d) 00 01 10 11

11. Variable Elimination and Conditioning Query:

12. Pictorial Depiction of Elimination

13. Gaussian Distributions Random variables that approximately have Gaussian distributions: Goto Mathematica

14. The Likelihood X X Observe i.i.d. data (independent and identically distributed): Likelihood

15. Maximizing the Likelihood of the Data Observe i.i.d. data (independent and identically distributed): Maximizing with respect to µ: This is difficult! What can we do?

16. Log is a Monotonic increasing function See Mathematica Notebook

17. Maximizing a concave function: See plot in Mathematica

18. Concave Functions: Not Concave or Convex and therefore difficult to maximize and minimize Easy to maximize and minimize