1. The EM Algorithm With Applications To Image Epitome
Kiera Henning, Jiwon Kim,

2. EM Problem
Given data U determine the best parameters Q without knowing the values of the hidden variables J.
Example: color segmentation (U: color at each pixel, theta: mean color of each cluster, J: cluster assignment for each pixel)

3. EM Algorithm
Maximize the posterior probability of the parameters Q given U, marginalizing over J.
To do this, alternate between estimating J (the E-step) and Q (the M-step).
More precisely, E step: estimate “distribution over” J.
Alternatively, E step: compute tight lower bound to posterior; M step: maximize bound w.r.t. theta.

4. Using the EM Algorithm
Identify U, Q, J and derive update equations

5. Using the EM Algorithm Identify U, Q, J and derive update equations
Convergence guaranteed, but slow

6. Using the EM Algorithm Identify U, Q, J and derive update equations
Convergence guaranteed, but slow: variational EM
Instead of directly estimating distribution over J in the E step, use parameterized form for the distribution with a set of variational parameters.
This reduces the number of computation steps involved in the update and speeds up the algorithm.
However, may lose quality of solution (bound decreased, not maximized). Also up to the designer to choose parameterization.

7. Using the EM Algorithm Identify U, Q, J and derive update equations
Convergence guaranteed, but slow: variational EM
Each implementation is application specific
No general code that can be plugged in, like graph cuts.

8. Image Epitome
“The epitome of an image is its miniature, condensed version containing the essence of the textural and shape properties of the image.”

9. Applying EM Data: Parameters to be estimated: Hidden variables:
set of patches
Parameters to be estimated:
epitome
Hidden variables:
mappings from epitome to image

10. Examples High initial noise 1000 patches 4000 patches
After 10 iterations
Low initial noise
1000
patches
4000
patches
EM iteration

11. Examples
4000 Patches

12. Examples 4000 Patches (high initial noise)
4000 Patches (low initial noise)
Reconstructed image

13. Epitome Extensions Suggestions Courtesy of Nebojsa Jojjic.
Video Epitome
Motion Epitome
Video Epitome: image epitome extended to three dimensions
Motion Epitome: articulated object trajectories, as sequences of joint angles

14. Video Epitome
Idea:
Extend the concept of an Image Epitome to a Video. The epitome of a video would then be a much smaller video containing the textural, shape, and motion properties of the video.

15. Motion Epitome
Idea:
Apply the concept of image epitomes to motion data. The motion epitome would then be significantly smaller than the motion data while still containing most of the motion information.
The epitome of motion data is a significantly smaller amount of data which captures the essence of the motion.

16. Motion Epitome

17. References
Advances in Algorithm for Inference and Learning in Complex Probability Models, Brendan J. Frey and Nebojsa Jojic
Epitomic Analysis of Appearance and Shape, Nebojsa Jojic, Brendan J. Frey, and Anitha Kannan
Epitome Representation of Images and Notes on Epitome-based Silhouette Generation, Shinko Y. Cheng
The Expectation Maximization Algorithm, Frank Dellaert
Motion Segmentation using EM – a short Tutorial, Yair Weiss