1. More Parameter Learning, Multinomial and Continuous Variables
Baran Barut
CSE 970 – PATTERN RECOGNITION

2. OUTLINE Multinomial Variables - Learning a Relative Frequency
- Probability Intervals and Regions
- Learning Parameters in a Bayesian Network
- Missing Data Items
- Variances in Computed Relative Frequencies
Continuous Variables
- Normally Distributed Variable
- Multivariate Normally Distributed Variable
- Gaussian Bayesian Networks

3. Dirichlet:
where
Modeling our beliefs concerning relative frequencies!

5. Introductory formulas:
If we knew that the relative frequency of k’th outcome is fk :

6. The probability of data set:
- D is a multinomial sample of size M governed by F
- sk is the number of outcomes in d equal k

7. How to update the distribution function using a data set:
Updated probabilities of outcomes:

8. How confident are we about the estimate of the relative frequency fk?

9. A multinomial Bayesian network has Xi ’s with space i>2

10. Global independence of Fi’s
Local independence of Fij’s

12. Equivalent sample size, N
If G,F and N are specified, then or

13. Normal Distribution

14. Unknown Mean and Known Variance

15. Sample of size M
1. Each outcome has real numbers as range
2. F = {A,r}
and
D is called a normal sample of size M with parameter
{A,r}

16. posterior density of A

17. assumptions about a hypothetical sample
r = 1 case
v = 0 case (no prior belief)

18. Probality of the next outcome
remember, initially:
and

19. Gamma Distribution
X1, X2,..., Xk are k-independent random variables with N(x;0,σ2) and V= X 21+X X 2k , then:
V has distribution gamma(v,k/2,1/2 σ2)

20. Known Mean and Unknown Variance

21. Sample of size M
1. Each outcome has real numbers as range
2. F = {a,R}
and
D is called a normal sample of size M with parameter
{a,R}

22. posterior density of r

23. t-distribution

24. Unknown Mean and Unknown Variance

25. How to update?
and

26. meaning of the parameters v, μ
μ is the mean of the hypothetical sample concerning value of A
v is the size of the hypothetical sample concerning value of A

27. meaning of the parameters β
β is s of the hypothetical sample

29. bivariate normal distribution

30. Vector Notation

31. Positive definite – positive semi definite
Symmetric n by n matrix A is positive definite if
Symmetric n by n matrix A is positive semidefinite if

32. Invertible
If symmetric n by n matrix A is positive definite, then it is nonsingular.
If Symmetric n by n matrix A is positive semidefinite but not positive definite, then it is singular.

33. Wishart-distribution

34. Multivariate t-distribution

35. Unknown Mean and Unknown Variance

36. How to update?
and

37. meaning of the parameters v, μ, β
μ is the mean of the hypothetical sample concerning value of A
v is the size of the hypothetical sample concerning value of A
β is s of the hypothetical sample

38. each node is a linear function of the values of the nodes that precede it in the ordering

39. How to find the precision matrix

40. Complete Gaussian Bayesian Network

41. Covariance Matrix
What if b’s are 0?

42. How to update?
and

43. Approximations!
Gaussian Bayesian Networks stand for
N(x; μ, T-1) whereas
x was given by t(x;α, μ,T)
We don’t assign distributions to Fi’s
We asses distributions for the random variables A, R