Download presentation
Presentation is loading. Please wait.
Published byCharla Baldwin Modified over 6 years ago
1
Nonparametric Bayesian Texture Learning and Synthesis
Leo Zhu Yuanhao Chen William Freeman Antonio Torralba Presented at NIPS 2009 Discussion and Slides by Eric Wang August 13, 2010 Wang 2010
2
Outline Introduction Image Patches and Features HDP-2DHMM Inference
Texture Synthesis Image Segmentation and Synthesis Conclusion Wang 2010
3
Introduction The authors consider texture learning and synthesis with spatial dependence. Two basic approaches to this problem have been considered Represent texture using textons and spatial layout. This approach is sensitive to parameter settings, has low rendering quality, and is slow. Patch based approaches offer improved rendering quality and are faster, but do not have semantic understanding and texture modeling. In this paper, the authors adopt a patch based approach, and augment it with nonparametric Bayesian modeling and statistical learning. A spatial HMM (2D-HMM) whose states (texton vocabulary) are generated from an HDP. Once the parameters of the HDP-2DHMM are learned, large textures can be synthesized according to the spatial transition matrix and dictionary of textures. Wang 2010
4
Introduction Wang 2010
5
Image Patches and Features
Each patch is characterized by a set of filter responses that correspond to values b of image filter response h at location l. Specifically, each 24x24 patch is divided into 6x6 cells (l=1,…,36), each of which is size 4x4 pixels. For each pixel in cell l, the response to h=37 image filter responses are computed. For each 4x4 cell, the responses to each filter are averaged across all pixels and quantized (binned) into 1 of b=6 bins. Therefore, each patch is represented by 37x36x6=7992 dimensional feature responses. The authors point out that, unlike previous work that used Kmeans to first form a visual vocabulary, their approach directly applies a nonparametric model Wang 2010
6
HDP-2DHMM Let L(i), T(i), R(i), D(i) denote the four neighbors of node i. In the above graphical model, only L(i) and T(i) dependencies are shown. indexes the state of node i, and associates it with the cluster label of a texton. and are the emission and transition parameters, respeectively Wang 2010
7
HDP-2DHMM Wang 2010
8
HDP-2DHMM Wang 2010
9
HDP-2DHMM Wang 2010
10
HDP-2DHMM Wang 2010
11
HDP-2DHMM Wang 2010
12
HDP-2DHMM Wang 2010
13
HDP-2DHMM Wang 2010
14
HDP-2DHMM Wang 2010
15
HDP-2DHMM The authors also allow the textons to be slightly shifted from their original locations by introducing two hidden variables that indicate the displacements of textons relative to location i. For simplicity, the shift variables are given a uniform prior over a small shift neighborhood (10% in either direction). Wang 2010
16
Inference Collapsed Gibbs sampling is used to learn the parameters of HDP-2DHMM. The inference alternates between sampling the state indicators z and the global state probabilities . The state indicators z are sampled as The global state probabilities are sampled as in HDP, and the equation not given. Wang 2010
17
Texture Synthesis The primary objective of this paper was to synthesize textures from smaller texture observations. The authors note, however, that HDP-2DHMM is a generative model for image features and not actual image patches. To do this, they integrate image quilting, and define a dictionary of image patches (from the original image) from which the new image is synthesized (with much lower synthesis cost over standard image quilting). Wang 2010
18
Synthesis Examples
19
Synthesis Examples Wang 2010
20
Synthesis Examples Wang 2010
21
Image Segmentation and Synthesis
Wang 2010
22
Conclusion A novel nonparametric Bayesian model called HD-2DHMM for texture learning and synthesis was presented in this paper. The main contributions of this paper are that it learns the textons and spatial HMM structure jointly. Inference is performed using a collapsed Gibbs Sampler The clustering nature of the model allows for faster image quilting and synthesis with good qualitative results. Image segmentation and synthesis results were also presented. Wang 2010
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.