1. Advanced Artificial Intelligence
Lecture 2A: Probability Theory Review

2. Outline Axioms of Probability Product and chain rules Bayes Theorem
Random variables
PDFs and CDFs
Expected value and variance

3. Introduction
Sample space set of all possible outcomes of a random experiment
Dice roll: {1, 2, 3, 4, 5, 6}
Coin toss: {Tails, Heads}
Event space subsets of elements in a sample space
Dice roll: {1, 2, 3} or {2, 4, 6}
Coin toss: {Tails}

5. examples Coin flip P(H) P(T) P(H,H,H) P(x1=x2=x3=x4)
P({x1,x2,x3,x4} contains more than 3 heads)

6. Set operations

7. Conditional Probability

8. Conditional Probability

9. examples Coin flip P(x1=H)=1/2 P(x2=H|x1=H)=0.9 P(x2=T|x1=T)=0.8

10. Conditional Probability

11. Conditional Probability
P(A, B)
0.005
P(B)
0.02
P(A|B)
0.25

12. Quiz P(D1=sunny)=0.9 P(D2=sunny|D1=sunny)=0.8 P(D2=rainy|D1=sunny)=?
P(D2=sunny|D1=rainy)=0.6
P(D2=rainy|D1=rainy)=?
P(D2=sunny)=?
P(D3=sunny)=?
0.2,0.4,0.78,0.756

13. Joint Probability Multiple events: cancer, test result Has cancer?
Test positive?
P(C,TP)
yes
0.018 no
0.002
0.196
0.784

14. Joint Probability The problem with joint distributions
It takes 2D-1 numbers to specify them!

15. Conditional Probability
Describes the cancer test:
Put this together with: Prior probability

16. Conditional Probability
We have:
We can now calculate joint probabilities
Has cancer?
Test positive?
P(TP, C)
yes
0.018 no
0.002
0.196
0.784
Has cancer?
Test positive?
P(TP, C)
yes no

17. Conditional Probability
“Diagnostic” question: How likely do is cancer given a positive test?
Has cancer?
Test positive?
P(TP, C)
yes
0.018 no
0.002
0.196
0.784

18. Bayes Theorem

19. Posterior Probability
Bayes Theorem
Posterior Probability
A in unobserved, but B is observed
Likelihood
Prior Probability
Normalizing Constant

20. Bayes Theorem
A in unobserved, but B is observed

21. Random Variables

22. Cumulative Distribution Functions
F(x) is monotonically non-decreasing

23. Probability Density Functions
PDF is also called probability mass function when applied to discrete random variables

24. Probability Density Functions
PDF is also called probability mass function when applied to discrete random variables

25. Probability Density Functions
PDF is also called probability mass function when applied to discrete random variables

26. Probability Density Functions
f(X) X
PDF is also called probability mass function when applied to discrete random variables

27. Probability Density Functions
f(X) X
PDF is also called probability mass function when applied to discrete random variables

28. Probability Density Functions
f(x) x
PDF is also called probability mass function when applied to discrete random variables
F(x) 1 x

29. Probability Density Functions
f(x) x
PDF is also called probability mass function when applied to discrete random variables
F(x) 1 x

30. Expectation
PDF is also called probability mass function when applied to discrete random variables

31. Expectation
PDF is also called probability mass function when applied to discrete random variables

32. Variance

33. Gaussian Distributions