1. Dictionary Learning on a Manifold
1Mingyuan Zhou, 2Hongxia Yang, 3Guillermo Sapiro,
2David Dunson and 1Lawrence Carin
1Department of Electrical & Computer Engineering, Duke University
2Department of Statistical Science, Duke University
3Department of Electrical & Computer Engineering, University of Minnesota
Zhou 2011

2. Outline Introduction Dependent Hierarchical Beta Process (dHBP)
Dictionary Learning with dHBP
Image Interpolation and Denoising
Conclusions

3. Introduction: Background
Dictionary learning and sparse coding
Sparse factor analysis model (Factor/feature/dish/dictionary atom)
Indian Buffet process and beta process

4. Introduction: Background
Beta process and Bernoulli process (Thibaux & Jordan AISTATS2007)
Indian Buffet process (Griffiths & Ghahramani 2005)

5. Introduction: Motivation
Exchangeability assumption is not true
Image interpolation with BPFA (Zhou et al. 2009)
80% pixels are missing at random

6. Introduction: Covariate Dependence
Dependent Dirichlet process
MacEachern (1999), Duan et al. (2007), Griffin & Steel (2006)
Non-exchangeable IBP
Phylogenetic IBP (Miller, Griffiths & Jordan 2008)
Dependent IBP (Williamson, Orbanz & Ghahramani 2010)
Bayesian density regression (Dunson & Pillai 2007)

7. Review of Beta Process (Thibaus & Jordan 2007)
A beta process is a positive Levy process, whose Levy measure lives on and can be expressed as
If the base measure is continuous, then is drawn from a degenerate beta distribution parameterized by
If , then

8. Dependent Hierarchical Beta Process
Random walk matrix
dHBP
Covariate-dependent correlations

9. Dictionary Learning with dHBP

10. Missing Data and Outliers
Full data:
Observed: , Missing:
Sparse spiky noise
Recoverd data:

11. MCMC Inference Independence chain Metropolis-Hastings Slice sampling
Gibbs sampling

12. Experiments Image interpolation Image denoising
Missing pixels
Locations of missing pixels are known
Image denoising
WGN + sparse spiky noise
Amplitudes unknown
Locations of spiky noise are unknown
Covariates: patch spatial locations

13. Image Interpolation: BP vs. dHBP
BP: 26.9 dB
dHBP: dB
80% pixels missing at random

14. Image Interpolation: BP vs. dHBP
Observed
BP recovery
dHBP recovery
Original
BP atom usage
dHBP atom usage
dHBP recovery
BP dictionary
dHBP dictionary
Dictionary atom activation probability map

15. Image Interpolation: BP vs. dHBP
dHBP recovery
Observed (20%)
BP recovery
dHBP recovery
Original

16. Image Interpolation: BP vs. dHBP
dHBP recovery
Observed (20%)
BP recovery
dHBP recovery
Original

17. Spiky Noise Removal: BP vs. dHBP
Original image
dHBP dictionary
dHBP denoised image
Noisy image (WGN + Sparse Spiky noise)
BP dictionary
BP denoised image

18. Spiky Noise Removal: BP vs. dHBP
Original image
dHBP dictionary
dHBP denoised image
Noisy image (WGN + Sparse Spiky noise)
BP dictionary
BP denoised image

19. Future Work Landmark-dHBP Locality constraint for manifold learning
J landmarks
J << N
Locality constraint for manifold learning
Covariates: cosine distance between samples
Dictionary atoms look like the data
Variational inference, online learning
Other applications
Super-resolution
Deblurring
Video background & foreground modeling

20. Conclusions A dependent hierarchical beta process is proposed
Efficient hybrid MCMC inference is presented
Encouraging performance is demonstrated on image-processing applications