1. Outline
Texture modeling - continued
Julesz ensemble

2. Visual Perception Modeling
FRAME Model – review
FRAME model
Filtering, random field, and maximum entropy
A well-defined mathematical model for textures by combining filtering and random field models
Maximum entropy is used when constructing the probability distribution on the image space
Minimum entropy is used when selecting filters from a large bank of filters
Together this is called min-max entropy principle
2/24/2019
Visual Perception Modeling

3. Visual Perception Modeling
FRAME Model – review
Maximum Entropy Distribution
Given the expectations of some functions, the maximum entropy solution for p(x) is
where
2/24/2019
Visual Perception Modeling

4. Visual Perception Modeling
FRAME Model – review
Maximum Entropy – continued
are determined by the constraints
Gradient ascend to maximize
2/24/2019
Visual Perception Modeling

5. Visual Perception Modeling
Julesz Ensemble
The original texture modeling question
What features and statistics are characteristics of a texture pattern, so that texture pairs that share the same features and statistics cannot be told apart by pre-attentive human visual perception?
--- Julesz, 1962
2/24/2019
Visual Perception Modeling

6. Summary of Existing Texture Features
2/24/2019
Visual Perception Modeling

7. Existing Feature Statistics
2/24/2019
Visual Perception Modeling

8. Most General Feature Statistics
2/24/2019
Visual Perception Modeling

9. Visual Perception Modeling
Julesz Ensemble – cont.
Definition
Given a set of normalized statistics on lattice 
a Julesz ensemble W(h) is the limit of the following set as   Z2 and H  {h} under some boundary conditions
2/24/2019
Visual Perception Modeling

10. Visual Perception Modeling
Julesz Ensemble – cont.
Feature selection
A feature can be selected from a large set of features through information gain, or the decrease in entropy
2/24/2019
Visual Perception Modeling

11. Visual Perception Modeling
Julesz Ensemble – cont.
2/24/2019
Visual Perception Modeling

12. Visual Perception Modeling
Julesz Ensemble – cont.
Sampling the Julesz ensemble
In the Julesz ensemble, a texture type is defined as all the images sharing the observed statistics and features
It is an inverse problem in order to generate texture images or verify the statistics
The problem is again the dimensionality
If the image size is 256x256 and each pixel can have 8 values, there are different images
Markov chain Monte-Carlo algorithms
2/24/2019
Visual Perception Modeling

13. Visual Perception Modeling
Julesz Ensemble – cont.
Given observed feature statistics {H(a)obs}, we associate an energy with any image I as
Then the corresponding Gibbs distribution is
The q(I) can be sampled using a Gibbs sampler or other Markov chain Monte-Carlo algorithms
2/24/2019
Visual Perception Modeling

14. Image Synthesis Algorithm
Compute {Hobs} from an observed texture image
Initialize Isyn as any image, and T as T0
Repeat
Randomly pick a pixel v in Isyn
Calculate the conditional probability q(Isyn(v)| Isyn(-v))
Choose new Isyn(v) under q(Isyn(v)| Isyn(-v))
Reduce T gradually
Until E(I) < e
2/24/2019
Visual Perception Modeling

15. A Texture Synthesis Example
Observed image
Initial synthesized image
2/24/2019
Visual Perception Modeling

16. A Texture Synthesis Example
Image patch
Energy
Conditional probability
Temperature
2/24/2019
Visual Perception Modeling

17. A Texture Synthesis Example - continued
Average spectral histogram error
2/24/2019
Visual Perception Modeling

18. Texture Synthesis Examples - continued
Observed image
Synthesized image
2/24/2019
Visual Perception Modeling

19. Texture Synthesis Examples - continued
Observed image
Synthesized image
2/24/2019
Visual Perception Modeling

20. Texture Synthesis Examples - continued
Mud image
Synthesized image
2/24/2019
Visual Perception Modeling

21. Texture Synthesis Examples - continued
Observed image
Synthesized image
2/24/2019
Visual Perception Modeling

22. Texture Synthesis Examples - continued
Observed image
Synthesized image
2/24/2019
Visual Perception Modeling

23. Texture Synthesis Examples - continued
Synthesized image
Original cheetah skin patch
2/24/2019
Visual Perception Modeling

24. Texture Synthesis Examples - continued
Observed image
Synthesized image
2/24/2019
Visual Perception Modeling

25. Texture Synthesis Examples - continued
Observed image
Synthesized image
2/24/2019
Visual Perception Modeling

26. Texture Synthesis Examples - continued
Observed image
Synthesized image
2/24/2019
Visual Perception Modeling

27. An Synthesis Example for Fun
2/24/2019
Visual Perception Modeling

28. Comparison with Texture Synthesis Method - continued
An example from Heeger and Bergen’s algorithm
Cross image
Heeger and Bergen’s
Our result
2/24/2019
Visual Perception Modeling

29. Visual Perception Modeling
Julesz Ensemble – cont.
Remarks
The results shown here are based on histograms of filter responses
However, the Julesz ensemble applies to any features/statistics of your choice
You can also define Julesz ensemble for images other than textures
2/24/2019
Visual Perception Modeling

30. Visual Perception Modeling
Julesz Ensemble – cont.
Applications
This essentially provides a framework to systematically verify the sufficiency of chosen features/statistics
Normally, features/statistics are evaluated empirically. In other words, features are evaluated on a limited number of images
2/24/2019
Visual Perception Modeling