EE565 Advanced Image Processing Copyright Xin Li Statistical Modeling of Natural Images in the Wavelet Space Parametric models of wavelet coefficients  Univariate i.i.d. models  Spatially adaptive models Application into texture synthesis  Pyramid-based scheme (Heeger&Bergen’1995)  Projection-based scheme (Portilla&Simoncelli’2000)

EE565 Advanced Image Processing Copyright Xin Li Recall: Transform Facilitates Modeling x1x1 x2x2 y1y1 y2y2 x 1 and x 2 are highly correlated p(x 1 x 2 )  p(x 1 )p(x 2 ) y 1 and y 2 are less correlated p(y 1 y 2 )  p(y 1 )p(y 2 )

EE565 Advanced Image Processing Copyright Xin Li Empirical Observation H1H1 A single peak at zero

EE565 Advanced Image Processing Copyright Xin Li Univariate Probability Model Laplacian: Gaussian:

EE565 Advanced Image Processing Copyright Xin Li Gaussian Distribution

EE565 Advanced Image Processing Copyright Xin Li Laplacian Distribution

EE565 Advanced Image Processing Copyright Xin Li Statistical Testing How do we know which parametric model better fits the empirical distribution of wavelet coefficients? In addition to visual inspection (which is often subjective and less accurate), we can use various statistical testing tools to objectively evaluate the closeness of an empirical cumulative distribution function (ECDF) to the hypothesized one One of the most widely used techniques is Kolmogorov-Smirnov Test (MATLAB function: >help kstest).

EE565 Advanced Image Processing Copyright Xin Li Kolmogorov-Smirnov Test* The K-S test is based on the maximum distance between empirical CDF (ECDF) and hypothesized CDF (e.g., the normal distribution N(0,1)).

EE565 Advanced Image Processing Copyright Xin Li Example Usage: [H,P,KS,CV] = KSTEST(X,CDF) If CDF is omitted, it assumes pdf of N(0,1) x: computer-generated samples (0<P<1, the larger P, the more likely) Accept the hypothesis Reject the hypothesis d: high-band wavelet coefficients of lena image (note the normalization by signal variance)

EE565 Advanced Image Processing Copyright Xin Li Generalized Gaussian/Laplacian Distribution where Laplacian Gaussian P: shape parameter : variance parameter

EE565 Advanced Image Processing Copyright Xin Li Model Parameter Estimation* Maximum Likelihood Estimation Method of moments Linear regression method [1] Sharifi, K. and Leon-Garcia, A. “ Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video, ” IEEE T-CSVT, No. 1, February 1995, pp [2] Ref.

EE565 Advanced Image Processing Copyright Xin Li I.I.D. Assumption Challenged If wavelet coefficients of each subband are indeed i.i.d., then random permutation of pixels should produce another image of the same class (natural images) The fundamental question here: does WT completely decorrelate image signals?

EE565 Advanced Image Processing Copyright Xin Li Image Example High-band coefficients permutation You can run the MATLAB demo to check this experiment

EE565 Advanced Image Processing Copyright Xin Li Another Experiment Joint pdf of two uncorrelated random variables X and Y X Y

EE565 Advanced Image Processing Copyright Xin Li Joint PDF of Wavelet Coefficients Neighborhood I(Q): {Left,Up,cousin and aunt} X= Y= Joint pdf of two correlated random variables X and Y

EE565 Advanced Image Processing Copyright Xin Li Heeger&Bergen’1995: Histogram-based Pyramid-based (using steerable pyramids)  Facilitate the statistical modeling Histogram matching  Enforce the first-order statistical constraint Texture matching  Alternate histogram matching in spatial and wavelet domain Boundary handling: use periodic extension Color consistency: use color transformation Basic idea: two visually similar textures will also have similar statistics

EE565 Advanced Image Processing Copyright Xin Li Histogram Matching Generalization of histogram equalization (the target is the histogram of a given image instead of uniform distribution)

EE565 Advanced Image Processing Copyright Xin Li Histogram Equalization Uniform Quantization Note: L 1 x L y 0 cumulative probability function

EE565 Advanced Image Processing Copyright Xin Li MATLAB Implementation function y=hist_eq(x) [M,N]=size(x); for i=1:256 h(i)=sum(sum(x= =i-1)); End y=x;s=sum(h); for i=1:256 I=find(x= =i-1); y(I)=sum(h(1:i))/s*255; end Calculate the histogram of the input image Perform histogram equalization

EE565 Advanced Image Processing Copyright Xin Li Histogram Equalization Example

EE565 Advanced Image Processing Copyright Xin Li Histogram Specification ST S -1 * T histogram 1 histogram 2 ?

EE565 Advanced Image Processing Copyright Xin Li Texture Matching Objective: the histogram of both subbands and synthesized image matches the given template Basic hypothesis: if two texture images visually look similar, then they have similar histograms in both spatial and wavelet domain

EE565 Advanced Image Processing Copyright Xin Li Image Examples

EE565 Advanced Image Processing Copyright Xin Li Portilla&Simoncelli’2000: Parametric Instead of matching histogram (nonparametric models), we can build parametric models for wavelet coefficients and enforce the synthesized image to inherit the parameters of given image Model parameters: 710 parameters were used in Portilla and Simoncelli ’ s experiment (4 orientations, 4 scales, 7  7 neighborhood) Basic idea: two visually similar textures will also have similar statistics

EE565 Advanced Image Processing Copyright Xin Li Statistical Constraints Four types of constraints  Marginal Statistics  Raw coefficient correlation  Coefficient magnitude statistics  Cross-scale phase statistics Alternating Projections onto the four constraint sets  Projection-onto-convex-set (POCS)

EE565 Advanced Image Processing Copyright Xin Li Convex Set A set Ω is said to be convex if for any two point We have Convex set examples Non-convex set examples

EE565 Advanced Image Processing Copyright Xin Li Projection Operator f g Projection onto convex set C C In simple words, the projection of f onto a convex set C is the element in C that is closest to f in terms of Euclidean distance

EE565 Advanced Image Processing Copyright Xin Li Alternating Projection X0X0 X1X1 X2X2 X∞X∞ Projection-Onto-Convex-Set (POCS) Theorem: If C 1, …,C k are convex sets, then alternating projection P 1, …,P k will converge to the intersection of C 1, …,C k if it is not empty Alternating projection does not always converge in the case of non-convex set. Can you think of any counter-example? C1C1 C2C2

EE565 Advanced Image Processing Copyright Xin Li Convex Constraint Sets ● Non-negative set ● Bounded-value set ● Bounded-variance set A given signal or

EE565 Advanced Image Processing Copyright Xin Li Non-convex Constraint Set Histogram matching used in Heeger&Bergen’1995 Bounded Skewness and Kurtosis skewnesskurtosis The derivation of projection operators onto constraint sets are tedious are referred to the paper and MATLAB codes by Portilla&Simoncelli.

EE565 Advanced Image Processing Copyright Xin Li Image Examples original synthesized

EE565 Advanced Image Processing Copyright Xin Li Image Examples (Con’d) original synthesized

EE565 Advanced Image Processing Copyright Xin Li When Does It Fail? original synthesized

EE565 Advanced Image Processing Copyright Xin Li Summary Textures represent an important class of structures in natural images – unlike edges characterizing object boundaries, textures often associate with the homogeneous property of object surfaces Wavelet-domain parametric models provide a parsimonious representation of high-order statistical dependency within textural images