Expectation- Maximization

News o’ the day First “3-d” picture of sun Anybody got red/green sunglasses?

Administrivia No noose is good noose

Where we’re at Last time: E^3 Finished up (our brief survey of) RL Today: Intro to unsupervised learning The expectation-maximization “algorithm”

What’s with this EM thing? Nobody expects...

Unsupervised learning EM is (one form of) unsupervised learning: Given: data Find: “structure” of that data Clusters -- what points “group together”? (we’ll do this one today) Taxonomies -- what’s descended from/related to what? Parses -- grammatical structure of a sentence Hidden variables -- “behind the scenes”

Example task

We can see the clusters easily but the computer can’t. How can we get the computer to identify the clusters? Need: algorithm that takes data and returns a label (cluster ID) for each data point

Parsing example What’s the grammatical structure of this sentence? He never claimed to be a god.

Parsing example He never claimed to be a god. What’s the grammatical structure of this sentence? NVVNDetAdv NP VP NP VP S

Parsing example He never claimed to be a god. What’s the grammatical structure of this sentence? NVVNDetAdv NP VP NP VP S Note: entirely hidden information! Need to infer (guess) it in an ~unsupervised way.

EM assumptions All learning algorithms require data assumptions EM: generative model Description of process that generates your data Assumes: hidden (latent) variables Probability model: assigns probability to data + hidden variables Often think: generate hidden var, then generate data based on that hidden var

Classic latent var model Data generator looks like this: Behind a curtain: I flip a weighted coin Heads: I roll a 6-sided die Tails: I roll a 4-sided die I show you: Outcome of die

Your mission Data you get is sequence of die outcomes 6, 3, 3, 1, 5, 4, 2, 1, 6, 3, 1, 5, 2,... Your task: figure out what the coin flip was for each of these numbers Hidden variable: c≡outcome of coin flip What makes this hard?

A more “practical” example Robot navigating in physical world Locations in world can be occupied or unoccupied Robot wants occupancy map (so it doesn’t bump into things) Sensors are imperfect (noise, object variation, etc.) Given: sensor data Infer: occupied/unoccupied for each location

Classic latent var model This process describes (generates) prob distribution over numbers Hidden state: outcome of coin flip

Classic latent var model This process describes (generates) prob distribution over numbers Hidden state: outcome of coin flip

Classic latent var model This process describes (generates) prob distribution over numbers Hidden state: outcome of coin flip Observed state: outcome of die given (conditioned on) coin flip result

Classic latent var model This process describes (generates) prob distribution over numbers Hidden state: outcome of coin flip Observed state: outcome of die given (conditioned on) coin flip result

Classic latent var model This process describes (generates) prob distribution over numbers Hidden state: outcome of coin flip Observed state: outcome of die given (conditioned on) coin flip result

Probability of observations Final probability of outcome ( x ) is mixture of probability for each possible coin result:

Probability of observations Final probability of outcome ( x ) is mixture of probability for each possible coin result:

Probability of observations Final probability of outcome ( x ) is mixture of probability for each possible coin result:

Your goal Given model, and data, x 1, x 2,..., x n Find Pr[c i |x i ] So we need the model Model given by parameters: Θ= 〈 p,θ heads,θ tails 〉 Where θ heads and θ tails are die outcome probabilities; p is prob of heads

Where’s the problem? To get Pr[c i |x i ], you need Pr[x i |c i ] To get Pr[x i |c i ], you need model parameters To get model parameters, you need Pr[c i |x i ] Oh oh...

EM to the rescue! Turns out that you can run this “chicken and egg process” in a loop and eventually get the right * answer Make an initial guess about coin assignments Repeat: Use guesses to get parameters (M step) Use parameters to update coin guesses (E step) Until converged

EM to the rescue! function [Prc,Theta]=EM(X) // initialization Prc=pick_random_values() // the EM loop repeat { // M step: pick maximum likelihood // parameters: // argmax_theta(Pr[x,c|theta]) Theta=get_params_from_c(Prc) // E step: use complete model to get data // likelihood: Pr[c|x]=1/z*Pr[x|c,theta] Prc=get_labels_from_params(X,Theta) } until(converged)

Wierd, but true This is counterintuitive, but it works Essentially, you’re improving guesses on each step M step “maximizes” parameters, Θ, given data E step finds “expectation” of hidden data, given Θ Both are driving toward max likelihood joint soln Guaranteed to converge Not guaranteed to find global optimum...

Very easy example Two Gaussian (“bell curve”) clusters Well separated in space Two dimensions

In more detail Gaussian mixture w/ k “components” (clusters/blobs) Mixture probability:

In more detail Gaussian mixture w/ k “components” (clusters/blobs) Mixture probability: One for each component

In more detail Gaussian mixture w/ k “components” (clusters/blobs) Mixture probability: Weight (probability) of each component

In more detail Gaussian mixture w/ k “components” (clusters/blobs) Mixture probability: Gaussian distribution for each component w/ mean vector μ i and covariance matrix Σ i

In more detail Gaussian mixture w/ k “components” (clusters/blobs) Mixture probability: Normalizing term for Gaussian

In more detail Gaussian mixture w/ k “components” (clusters/blobs) Mixture probability: Squared distance of data point x from mean μ i (with respect to Σ i )

In more detail Gaussian mixture w/ k “components” (clusters/blobs) Mixture probability:

Hidden variables Introduce the “hidden variable”, c i (x) (or just c i for short) Denotes “amount by which data point x belongs to cluster i ” Sometimes called “cluster ownership”, “salience”, “relevance”, etc.

M step Need: parameters ( Θ ) given hidden variables ( c i ) and N data points, x 1, x 2,...,x N Q: what are the parameters of the model? (What do we need to learn?)

M step Need: parameters ( Θ ) given hidden variables ( c i ) and N data points, x 1, x 2,...,x N A: Θ= 〈 α i,μ i,Σ i 〉 i=1..k

M step Need: parameters ( Θ ) given hidden variables ( c i ) and N data points, x 1, x 2,...,x N A: Θ= 〈 α i,μ i,Σ i 〉 i=1..k

M step Need: parameters ( Θ ) given hidden variables ( c i ) and N data points, x 1, x 2,...,x N A: Θ= 〈 α i,μ i,Σ i 〉 i=1..k

M step Need: parameters ( Θ ) given hidden variables ( c i ) and N data points, x 1, x 2,...,x N A: Θ= 〈 α i,μ i,Σ i 〉 i=1..k

E step Need: probability of hidden variable ( c i ) given fixed parameters ( Θ ) and observed data ( x 1,...,x N )

Another example k=3 Gaussian clusters Different means, covariances Well separated

Restart Problem: EM has found a “minimum energy” solution It’s only “locally” optimal B/c of poor starting choice, it ended up in wrong local optimum -- not global optimum Default answer: pick a new random start and re-run

Final example More Gaussians. How many clusters here?

Note... Doesn’t always work out this well in practice Sometimes the machine is smarter than humans Usually, if it’s hard for us, it’s hard for the machine too... First ~7-10 times I ran this one, it lost one cluster altogether ( α 3 → )

Unresolved issues Notice: different cluster IDs (colors) end up on different blobs of points in each run Answer is “unique only up to permutation” I can swap around cluster IDs without changing solution Can’t tell what “right” cluster assignment is

Unresolved issues “Order” of model I.e., what k should you use? Hard to know, in general Can just try a bunch and find one that “works best” Problem: answer tends to get monotonically better w/ increasing k Best answer to date: Chinese restaurant process