1. Pyramid-Based Texture Analysis / Synthesis
Presented By :
Keren Kahanov

2. Texture Synthesis
Generation of much texture as desired so that any object can be covered
Input texture Output size Output texture
The output texture is synthesized image that match the texture appearance of the given digitized image

3. General Techniques Texture mapping – repeating the texture
Can lead to unacceptable artifacts in visible seams , visible repetition
Distortion in map to 3-D – often no natural map from planar texture image to the geometry/ topology of the surface. (There are some partial solutions , but no universal solution for mapping an image onto arbitrarily shaped surface.)

4. General Techniques (cont.)
Create (paint) textures by hand directly onto the 3-d surface model
Very labor intensive
Requires considerable artistic skills
Computer-synthesized textures
as much texture can generated as needed
Some of the synthesis techniques produce textures that tile seamlessly.

5. Synthetic textures The distortion problem has been solved
in two different ways:
Some techniques work by synthesizing
texture directly on the object surface.
The second solution is to use solid textures
(no mapping no distortion)

6. Heeger & Bergen Pyramid-Based Texture Analysis / Synthesis
Our lecture today
A technique for synthesizing an image (or
solid texture) that matches the appearance of a given texture sample.
The output image (at any size) looks like the original image in its color and texture properties.
The texture is synthesized so that it tiles seamlessly.

7. Heeger & Bergen (cont.) First order statistics (histograms)
Focuses on the synthesis of stochastic textures
deterministic textures: characterized by
a set of primitives and a placement rule
(e.g., a tile floor ,bricks).
stochastic textures : do not have easily
identifiable primitives (e.g., granite, bark, sand).
Many real-world textures have
some mixture of these two (e.g.
woven fabric, plowed fields).

8. Heeger & Bergen’s Algorithm
The texture-matching algorithm relies on two main components:
a self-inverting steerable pyramid
a histogram matching process.

9. Pyramids - Reminder
Reminder from “image processing” course by Dr. Hagit Hel-Or

10. Pyramids – Reminder (cont.)

11. (1) Steerable Pyramid
a linear multi-scale, multi-orientation image decomposition
developed in order to overcome the limitations of orthogonal separable wavelet decompositions
The steerable pyramid, unlike wavelet, is non-orthogonal and overcomplete
Overcomplete = the number of pixels in the pyramid is much greater than the number of pixels in the input image

12. Steerable Pyramid (cont.)
The basis functions of the steerable pyramid are directional derivative operators, that come in different sizes and orientations
translation-invariant (i.e., the subbands are aliasing-free, or equivariant with respect to translation)
Translating the image the transform translates energy in each band is preserved (not in wavelet)

13. Aliasing Aliasing Properly sampled
In steerable pyramid only the low-pass band is subsampled
This is done to minimize the amount of aliasing within each subband. Avoiding aliasing is critical because the algorithm treats each subband independently.
Aliasing
Properly
sampled

14. Steerable Pyramid (cont.)
rotation-invariant = steerable (i.e., the subbands are steerable, or equivariant with respect to rotation)
rotating the image the transform rotates
A set of filters form a steerable basis if
(1) they are rotated copies of each other,
and (2) a copy of the filter at any orientation may be computed as a linear combination of the basis

15. Steerable filters example
2nd derivative of Gaussian
Linear combination
filters:
Original
Image:
Same
Linear combination
outputs:

16. Steerable Pyramid (cont.)
A self-inverting pyramid:
it provides a complete representation of an image
original image can be reconstructed by applying the same filters used to generate the pyramid
A steerable pyramid can capture the variation of a texture in both intensity and orientation

17. Steerable Pyramid Build
Original image
Steerable Pyramid Build
Needs 7 filters:

18. Steerable Pyramid Build (cont.)
Original image
4 oriented filters
The steerable pyramid

19. Steerable Pyramid- Fourier domain
The steerable pyramid performs a polar-separable decomposition in the frequency domain, thus allowing independent representation of scale and orientation 1 4 2 3 3 2 4 1
wavelet

20. (2) Histogram Matching A generalization of histogram equalization.
modify an image (im1) to have the same histogram as another image (im2).
Using two lookup tables:
the cumulative distribution function (cdf) of one image.
The inverse cumulative distribution function (inverse cdf) of the other image

21. Cdf / inverse Cdf The cdf
maps from the interval [0,256] to the interval [0,1]. (cdf = accumulated histogram)
The inverse cdf
maps back from [0,1] to [0,256].
constructed by resampling (with linear interpolation) the cdf so that its samples are evenly spaced on the [0,1] interval.
Note! the values in the noise and target images need not lie in the range[0,256]

22. Histogram Matching Algorithm

23. התאמת ההיסטוגרמה של תמונה זו
לזו של תמונה זו
Histogram matched
תוצאה סופית

25. Image 1 histogram matched to image 2 histogram

26. Enough with all the talks about Pyramids & Histograms !!!
It’s time to learn the real thing –
Heeger & Bergen’s Algorithm for Texture Analysis / Synthesis

27. Heeger & Bergen’s Algorithm

28. The Algorithm - steps Input1 = sample texture image
Input2 = sample of uniform random noise
( noise image)
Its dimensions = desired size of output image
Match noise image’s histogram to the input sample’s histogram. ( modified noise image)
The texture sample is decomposed into an oriented bandpass pyramid

29. The Algorithm – steps (cont.)
Loop for several iterations:
Make pyramid from the modified noise image
Histogram matching process is employed between corresponding subbands of the two pyramids.
The noise pyramid is collapsed
The histogram matching process is again applied to the new noise image and the texture sample
After a few iterations, the output image, which began as random noise, begins to resemble the input texture.

30. When are two textures similar?
Textures appear visually similar if…
they have similar distributions of filter outputs*.
(Filters : orientation and spatial-frequency)
FROM: “Statistical Texture Synthesis” Presented by:Zeev Litichever & Tal Hassner , Weizmann Institute

31. Laplassian \ steerable pyramid
FROM: “Statistical Texture Synthesis” Presented by:Zeev Litichever & Tal Hassner , Weizmann Institute

32. Using histograms for synthesis
Output image with similar distributions to sample.
Iteratively coarse noise image to have sample histogram via histogram matching.
Output pyramid
Noise image of output size:
Apply filters
And create histograms
FROM: “Statistical Texture Synthesis” Presented by:Zeev Litichever & Tal Hassner , Weizmann Institute

33. Results
Input1 = texture image
Input2 = noise image
Output

34. More Results
In each pair : left= input (original) , right =output (synthesized)

35. Edge Handling
Proper edge handling in the convolution operations is important.
synthesis pyramid: use circular convolution. Given that the synthesis starts with a random noise image, circular convolution guarantees that the resulting synthetic texture will tile seamlessly. (synthesis pyramid = noise pyramid)

36. Edge Handling (cont.)
analysis pyramid: circular convolution would typically result in spuriously large filter responses at the image borders. This would, in turn, introduce artifacts in the synthesized texture. (analysis pyramid = original image pyramid)
A reasonable border handler for the analysis pyramid is to pad the image with a reflected copy of itself. Reflecting at the border usually avoids spurious responses.

37. Color Heeger and Bergen’s Algorithm can be applied for both :
gray level texture images
color texture images

38. Color Space RGB RGB components of a typical image are not independent
cannot directly apply the algorithm to each channel individually. R G
correlation
Looking for B ?
B is on a vacation ,
in another dimension
(helped to simplify the graph)

39. Color Space Transform
First, transform the color space of the input texture to independent channels
SVD algorithm :
The decorrelation transform is :

40. Color Space Transform (cont.)
Diagonal matrix G
No correlation in transformed color space R

41. New Color Space each independent channel can be processed separately.
After processing, the channels are transformed back into RGB values, using
yielding the synthesized texture.

42. Solid Textures The algorithm can also be used to make
isotropic 3-d solid textures.
Same as before, except for the choice of pyramid: use a 2-d Laplacian pyramid for analysis and a 3-d Laplacian pyramid for synthesis.

43. Solid Textures - examples

44. Texture Mixtures
Texture mixtures synthesized by choosing the color palette from one image and the pattern from a second image.

45. Texture Mixtures (cont.)
tiling different textures as input to Heeger and Bergen’s algorithm
Output: combine the properties of multiple textures.

46. Algorithm – no. iterations
An Implementation of Heeger and Bergen’s Texture Analysis/Synthesis Algorithm by Thomas F. El-Maraghi
The human observer would find it difficult, or even impossible, to discern any change after the 4th iteration.

47. No. of orientation
The effect of changing the number of oriented subbands in the pyramid on the synthesized texture.
the synthesized texture looks more like the original as the number of orientations is increased.
original
1 orientation
2 orientations
4 orientations

48. Homogenous / Inhomogoneous input
Homogoneous input texture success
Inhomogoneous input texture produces blotchy synthetic texture failure

49. Failures

50. More Failures

51. summery
In their paper entitled “Pyramid-Based Texture Analysis/Synthesis” , Heeger and Bergen present an algorithm for generating synthetic copies of stochastic textures.
The algorithm is based on a self-inverting oriented bandpass pyramid and a histogram matching technique
The process is tantalizing both for the results it produces and for its intrinsic simplicity.
Given a texture sample, either grayscale or color, the algorithm can synthesize as much of it as desired.

52. Upsides Fast Simple Produces tileable textures
No parameters (requires only the “target” texture as input)
You just crop a textured region from a digitized image and run a program to produce as much of that texture as you want.
Produces tileable textures
Works well on stochastic textures

53. Downsides Doesn’t work well other textures Not guaranteed to converge
For example : inhomogeneous texture, quasi-periodic textures and on random mosaic textures…
Not guaranteed to converge

54. References
Based on articles :
Pyramid-Based Texture Analysis/Synthesisby David J. Heeger & James R. Bergen
An Implementation of Heeger and Bergen’s Texture Analysis/Synthesis Algorithm by Thomas F. El-Maraghi
More Referances:
THE STEERABLE PYRAMID: A FLEXIBLE ARCHITECTURE FOR
MULTI-SCALE DERIVATIVE COMPUTATION by Eero P Simoncelli & William T Freeman
“image processing” course ,Haifa University, by Dr. Hagit Hel-Or
“Advanced Topics in Computer Vision“ course, Spring 2003, Weizmann Institute

55. THE END