2. Proposes functions to measure Gestalt features of shapes Adapts [Zhu, Wu Mumford] FRAME method to shapes Exhibits effect of MRF model obtained by putting these together.

3. (à la [Lowe], and others) Colinearity Cocircularity Proximity Parallelism Symmetry Continuity Closure Familiarity

4. [Zhu, Wu, Mumford] Filters R andom fields A nd M aximum E ntropy A general procedure for constructing MRF models

5. Data Learn MRF models from data Test generative power of learned model

6. A set of images representative of the chosen application domain An adequate collection of feature measures or filters The (marginal) statistics of applying the feature measures or filters to the set of images

7. Zhu considers 22 animal shapes and their horizontal flips The resulting histograms are symmetric More data can be obtained But are there other effects?

9. Goal is to be generic But generic shape features are hard to find φ 1 = κ(s), the curvature κ(s) = 0 implies the linelets on either side of Γ(s) are colinear φ 2 = κ'(s), its derivative κ'(s) = 0 implies three sequential linelets are cocircular “Other contour-based shape filters can be defined in the same way”

10. Ψ(s) pairs linelets across medial axes Defined and computed by minimizing an energy functional constructed so that Paired linelets are as close, parallel and symmetric as possible, and There are as few discontinuities as possible

11. φ 3 (s) = dist(s, ψ(s)) Measures proximity of paired linelets across a region φ 4 (s) = φ 3 '(s), the derivative φ 4 (s) = 0 implies paired linelets are parallel φ 5 (s) = φ' 4 (s) = φ 3 ''(s) φ 5 (s) = 0 implies paired linelets are symmetric

12. φ 6 (s) = 1 where ψ(s) is discontinuous 0 otherwise Counts the number of “parts” a shape has Can Gestalt “familiarity” be (statistically?) measured?

13. The histogram of feature φ over curve Γ is H(z; φ k, Γ) = ∫ δ(z-φ k (s)) ds δ is the Dirac function: mass 1 at 0, and 0 otherwise μ(z; φ k ) denotes the average over all images Zhu claims μ is a close estimation of the marginal distribution of the “true distribution” over shape space, assuming the total number of linelets is small.

14. φ 1 at scales 0, 1, 2 φ3φ3 φ4φ4 φ5φ5 On 22 images and their flips

15. Ω is the space of shapes Φ is a finite subset of feature filters We seek a probability distribution p on Ω ∫ Ω p(Γ) dΓ = 1 (1) That reproduces the statistics for all φ in Ω ∫ Ω p(Γ) δ(z-φ(s)) dΓ = μ(z; φ) (2)

16. Idea: Choose the p with maximal entropy Seems reasonable and fair, but is it really the best target/energy function? Lagrange multipliers and calculus of variations lead to p(Γ; Φ, Λ) = exp ( – ∑ φ Є Φ ∫ λ φ (z) H(φ, Γ, z) dz ) / Z where Z is the usual normalizing factor Λ = { λ φ | φ Є Φ }

17. In other words, it has the form of a Gibbs distribution, and therefore determines a Markov Random Field (MRF) model.

18. Too hard to compute λ's and p analytically Idea: Sample Ω according to the distribution p, stochastically update Λ to update p, and repeat until p reproduces all μ(z; φ) for φ Є Φ Monte Carlo because of random walk Markov Chain in the nature of the loop

19. From the sampling produce μ'(z; φ) Same as μ(z; φ) except based on a random sample of shape space For the purposes of today's discussion, the details are not important For φ Є Φ μ'(z; φ) = μ(z; φ) Zhu et al. assume there exists a “true underlying distribution”

20. For φ' not in the set Φ we expect μ'(z; φ') ≠ μ(z; φ') μ'(z; φ') is the accidental statistic for φ' It is a measure of correlation between φ' and Φ The “distance” (L 1, L 2, or other) between μ'(z; φ') and μ(z; φ') is the nonaccidental statistic for φ' It is a measure of how much “additional information” φ' carries above what is already in Φ

21. Enter your set Γ = { γ } of shapes Enter a (large) set { φ } of candidate feature measures Compute μ(φ, Γ) for all φ in Φ Compute μ'(φ) relative to a uniform distribution on Ω Until the nonaccidental statistic of all unused features is small enough, repeat:

22. Of the remaining φ, add to Φ one with maximal nonaccidental statistic Update: Set of Lagrange multipliers Λ = { λ } Probability distribution model p(Φ, Λ) The μ'(φ) for remaining candidate features φ

23. Let my description of these experiments stimulate your thoughts on such issues as Are there better Gestalt feature measures? What is the best possible outcome of a generative model of shape? What feature measures should be added to the Gestalt ones? How useful were these experiments and what other might be worth doing?

24. When the only feature used is the curvature the model generated

25. A Gaussian model (with the same κ-variance) produced

26. Experiment 2 uses both κ and κ' The nonaccidental statistic of κ' with respect to the model based on κ can be seen here

27. This time the model generated these shapes, purported to be smoother and more scale invariant

28. The nonaccidental statistics of the three region- based shape features relative to the model produced in Experiment 2

29. So r'' was omitted, this model has Φ = { κ, κ', r, r' }

30. This model produced such shapes as

31. Zhu acknowledges that the selection of training shapes might introduce a bias; but

32. Zhu acknowledges that the paucity of Gestalt features limits the possible neighborhood structures used to define a MRF. Zhu acknowledges that these models do not account for high-level shape properties, and suggests that a composition system might address this problem.

33. Although it is in the nature of an MRF-model to propagate local properties, I think there needs to be a higher-level basis (than linelets) for measuring the Gestalt features of a shape! Are there better Gestalt feature measures? What feature measures should be added to the Gestalt ones?

34. What is the best possible outcome of a generative model of shape? Is such a thing worth pursuing? How useful were Zhu' experiments and what others might be worth doing?