Generative Models for Image Analysis Stuart Geman (with E. Borenstein, L.-B. Chang, W. Zhang)

I.Bayesian (generative) image models II.Feature distributions and data distributions III.Conditional modeling IV.Sampling and the choice of null distribution V.Other applications of conditional modeling

I. Bayesian (generative) image models Prior Conditional likelihood Posterior focus here on

II. Feature distributions and data distributions image patch Model patch through a feature model:

e.g. detection and recognition of eyes image patch actually:

The first is fine for estimating λ but not fine for estimating T Use maximum likelihood…but what is the likelihood? ?

III. Conditional modeling

Conditional modeling: a perturbation of the null distribution

Estimation Much Easier!

Example: learning eye templates image patch

Example: learning eye templates

Maximize the data likelihood for the mixing probabilities, the feature parameters, and the templates themselves…

Example: learning (right) eye templates

How good are the templates? A classification experiment…

Classify East Asian and South Asian * mixing over 4 scales, and 8 templates East Asian: (L) examples of training images (M) progression of EM (R) trained templates South Asian: (L) examples of training images (M) progression of EM (R) trained templates Classification Rate: 97%

Other examples: noses 16 templates multiple scales, shifts, and rotations samples from training setlearned templates

Other examples: mixture of noses and mouths samples from training set (1/2 noses, 1/2 mouths) 32 learned templates

Other examples: train on 58 faces …half with glasses…half without 32 learned templates samples from training set 8 learned templates

Other examples: train on 58 faces …half with glasses…half without 8 learned templates random eight of the 58 faces row 2 to 4, top to bottom: templates ordered by posterior likelihood

Other examples: train random patches (“sparse representation”) 500 random 15x15 training patches from random internet images 24 10x10 templates

Other examples: coarse representation training of 8 low-res (10x10) templates sample from training set (down-converted images)

IV. Sampling and the choice of null distribution

(approximate) sampling…

V. Other applications of conditional modeling

Markov model Markov property… Estimation Computation Representation

Markov model

characters, plate sides generic letter, generic number, L-junctions of sides license plates parts of characters, parts of plate sides plate boundaries, strings (2 letters, 3 digits, 3 letters, 4 digits) license numbers (3 digits + 3 letters, 4 digits + 2 letters) Hierarchical models and the Markov dilemma

Original imageZoomed license region Top object: Markov distribution Top object: perturbed (“content-sensitive”) distribution Hierarchical models and the Markov dilemma

PATTERN SYNTHESIS = PATTERN ANALYSIS Ulf Grenander