Download presentation
Presentation is loading. Please wait.
Published byDelilah Rice Modified over 9 years ago
1
Latent Variable Models Christopher M. Bishop
2
1. Density Modeling A standard approach: parametric models a number of adaptive parameters Gaussian distribution is widely used. Loglikelihood method Limitation too flexible: parameter is so excessive not too flexible: only uni-modal Considering mixture model, latent variable model
3
1.1. Latent Variables The number of parameters in normal distribution. : d(d+1)/2 + : d d 2. Assuming diagonal covariance matrix reduces : d, but this means that t are statistically independent. Latent variables Degree of freedom can be controlled, and correlation can be captured. Goal to express p(t) of the variable t 1,…,t d in terms of a smaller number of latent variables x=(x 1,…,x q ) where q < d.
4
Cont ’ d Joint distribution of p(t,x) Bayesian network express the factorization
5
Cont ’ d Express p(t|x) in terms of mapping from latent variables to data variables. The definition of latent variables model is completed by specifying distribution p(u), mapping y(x;w), marginal distributino p(x). The desired model for distribution p(t), but it is intractable in almost case. Factor analysis: One of the simplest latent variable models
6
Cont ’ d W, : adaptive parameters p(x): chosen to be N(0,I) u: chosen to be zero mean Gaussian with a diagonal covariance matrix . Then P(t) is Gaussian, with mean and covariance matrix +WW T. Degree of freedom: (d+1)(q+1)-q(q+1)/2 Can capture the dominant correlations between the data variables
7
1.2. Mixture Distributions Uni-modal mixture of M simpler parametric distributions p(t|i): usually normal distribution with its own i, i. i : mixing coefficients mixing coefficients: prior probabilities for the values of the label i. Considering indicator variable z ni. Posterior probabilities: R ni is expectation of z ni.
8
Cont ’ d EM-algorithm Mixture of latent-variable models Bayesian network representation of a mixture of latent variable models. Given the values of i and x, the variables t 1,…,t d are conditionally independent.
9
2. Probabilistic Principal Component Analysis Summary q principal axes v j, j {1,…,q} v j are q dominant eigenvectors of sample covariance matrix. q principal components: reconstruction vector: Disadvantage absence of a probability density model and associated likelihood measure
10
2.1. Relationship to Latent Variables
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.