1. Lecture 15
Factor Analysis

2. Factor Analysis
Like clustering is classification without knowing the labels, we have that Factor Analysis is like regression without the targets.
Attributes are noisy linear functions of factors.
Noise model p(x|y) is Gaussian and prior p(y) is gaussian.
All factors can cooperate to generate output (unlike MoG).
We use fewer factors than attributes: dimensionality reduction.
Its like a sphere of data that gets warped into an ellipse, after which we add independent noise to each attribute.
Final model is Gaussian, so we can remove the mean first: centering.

3. EM E-step: We compute the posteriors using algebra for Gaussians.
Since we know that the posteriors must Gaussian we only need to consider the quadratic form in the exponent.
There are also some tricks to replace dxd inverses by smaller kxk inverses. (k=#factors, d=#attributes).
M-step
take derivatives and set to zero: analytic updates
We can change A & y together using a rotation and the model will not change.