Outline S. C. Zhu, X. Liu, and Y. Wu, “Exploring Texture Ensembles by Efficient Markov Chain Monte Carlo”, IEEE Transactions On Pattern Analysis And Machine Intelligence, Vol. 22, No. 6, pp. 554-569, 2000

Limitations of Linear Representations Linear representations do not depend on the spatial relationships among pixels For example, if we shuffle the pixels and corresponding representations, then the classification results will remain the same But in images spatial relationships are important November 21, 2018 Computer Vision

Image Features November 21, 2018 Computer Vision

Spectral Representation of Images Spectral histogram Given a bank of filters F(a), a = 1, …, K, a spectral histogram is defined as the marginal distribution of filter responses November 21, 2018 Computer Vision

Spectral Representation of Images - continued An example of spectral histogram November 21, 2018 Computer Vision

Image Modeling - continued Given observed feature statistics {H(a)obs}, we associate an energy with any image I as Then the corresponding Gibbs distribution is The q(I) can be sampled using a Gibbs sampler or other Markov chain Monte-Carlo algorithms November 21, 2018 Computer Vision

Image Modeling - continued Image Synthesis Algorithm Compute {Hobs} from an observed texture image Initialize Isyn as any image, and T as T0 Repeat Randomly pick a pixel v in Isyn Calculate the conditional probability q(Isyn(v)| Isyn(-v)) Choose new Isyn(v) under q(Isyn(v)| Isyn(-v)) Reduce T gradually Until E(I) < e November 21, 2018 Computer Vision

A Texture Synthesis Example Observed image Initial synthesized image November 21, 2018 Computer Vision

A Texture Synthesis Example Image patch Energy Conditional probability Temperature Energy and conditional probability of the marked pixel November 21, 2018 Computer Vision

A Texture Synthesis Example - continued Average spectral histogram error A white noise image was transformed to a perceptually similar texture by matching the spectral histogram November 21, 2018 Computer Vision

A Texture Synthesis Example - continued Synthesized images from different initial conditions November 21, 2018 Computer Vision

Texture Synthesis Examples - continued Observed image Synthesized image A random texture image November 21, 2018 Computer Vision

Texture Synthesis Examples - continued Observed image Synthesized image An image with periodic structures November 21, 2018 Computer Vision

Texture Synthesis Examples - continued Mud image Synthesized image A mud image with some animal foot prints November 21, 2018 Computer Vision

Texture Synthesis Examples - continued Observed image Synthesized image A random texture image with elements November 21, 2018 Computer Vision

Texture Synthesis Examples - continued Observed image Synthesized image An image consisting of two regions Note that wrap-around boundary conditions were used November 21, 2018 Computer Vision

Texture Synthesis Examples - continued Original cheetah skin patch Synthesized image A cheetah skin image November 21, 2018 Computer Vision

Texture Synthesis Examples - continued Observed image Synthesized image An image consisting of circles November 21, 2018 Computer Vision

Texture Synthesis Examples - continued Observed image Synthesized image An image consisting of crosses November 21, 2018 Computer Vision

Texture Synthesis Examples - continued Observed image Synthesized image A pattern with long-range structures November 21, 2018 Computer Vision

Object Synthesis Examples As in texture synthesis, we start from a random image In addition, similar object images are used as boundary conditions in that the corresponding pixel values are not updated during sampling process November 21, 2018 Computer Vision

Object Synthesis Examples - continued November 21, 2018 Computer Vision

Object Synthesis Examples - continued November 21, 2018 Computer Vision

Linear Transformations of Images Linear transformations include Principal component analysis Independent component analysis Fisher discriminant analysis Optimal component analysis They have been widely used to reduce dimension of images for appearance-based recognition applications Each image is viewed as a long vector and projected into a set of bases that have certain properties November 21, 2018 Computer Vision

Principal Component Analysis Defined with respect to a training set such that the average reconstruction error is minimized November 21, 2018 Computer Vision

Principal Component Analysis - continued November 21, 2018 Computer Vision

Eigen Values of 400 Eigen Vectors November 21, 2018 Computer Vision

Principal Component Analysis - continued Original Image Reconstructed using 50 PCs Reconstructed using 200 PCs November 21, 2018 Computer Vision

Principal Component Analysis - continued Is PCA representation a good representation of images for recognition in that images that have similar principal representations are similar? Image generation through sampling Roughly speaking, we try to generate images that have the given coefficients along PCs November 21, 2018 Computer Vision

Principal Component Analysis - continued November 21, 2018 Computer Vision

Principal Component Analysis - continued November 21, 2018 Computer Vision

Difference Between Reconstruction and Sampling Reconstruction is not sufficient to show the adequacy of a representation and sampling from the set of images with same representation is more informational November 21, 2018 Computer Vision

Object Recognition Experiments We compare linear methods in the methods including Principal component analysis (PCA) Independent component analysis (ICA) Fisher discriminant analysis (FDA) Random component analysis (RCA) For fun and to show the actual gain of using different bases is relatively small Corresponding linear methods in the spectral histogram space including SPCA, SICA, SFDA, and SRCA November 21, 2018 Computer Vision

COIL Dataset November 21, 2018 Computer Vision

3D Recognition Results November 21, 2018 Computer Vision

Experimental Results - continued To further demonstrate the effectiveness of our method for different types of images, we create a dataset of combining the texture dataset, face dataset, and COIL dataset, resulting in a dataset of 180 categories with 10160 images in total November 21, 2018 Computer Vision

Linear Subspaces of Spectral Representation November 21, 2018 Computer Vision

Experimental Results - continued Combined dataset – continued Not only the recognition rate is very good, but also it is very reliable and robust, as the average entropy of the p0(i|I) is 0.60 bit (The corresponding uniform distribution’s entropy is 7.49 bits) November 21, 2018 Computer Vision

Experimental Results - continued Combined dataset – continued Not only the recognition rate is very good, but also it is very reliable and robust, as the average entropy of the p0(i|I) is 0.60 bit (The corresponding uniform distribution’s entropy is 7.49 bits) Entropy=0.60 bit Entropy=6.78bits November 21, 2018 Computer Vision