1. Generative learning methods for bags of features
Model the probability of a bag of features given a class
Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba

2. Generative methods
We will cover two models, both inspired by text document analysis:
Naïve Bayes
Probabilistic Latent Semantic Analysis

3. The Naïve Bayes model
Assume that each feature is conditionally independent given the class
wi: ith feature in the image
N: number of features in the image
Csurka et al. 2004

4. The Naïve Bayes model
Assume that each feature is conditionally independent given the class
wi: ith feature in the image
N: number of features in the image
W: size of visual vocabulary
n(w): number of features with index w in the image
Csurka et al. 2004

5. The Naïve Bayes model
Assume that each feature is conditionally independent given the class
No. of features of type w in training images of class c
Total no. of features in training images of class c
p(w | c) =
Csurka et al. 2004

6. The Naïve Bayes model
Assume that each feature is conditionally independent given the class
No. of features of type w in training images of class c + 1
Total no. of features in training images of class c + W
p(w | c) =
(Laplace smoothing to avoid zero counts)
Csurka et al. 2004

7. The Naïve Bayes model MAP decision:
(you should compute the log of the likelihood instead of the likelihood itself in order to avoid underflow)
Csurka et al. 2004

8. The Naïve Bayes model
“Graphical model”: c w N
Csurka et al. 2004

9. Probabilistic Latent Semantic Analysis
= α1
+ α2
+ α3
Image
zebra
grass
tree
“visual topics”
T. Hofmann, Probabilistic Latent Semantic Analysis, UAI 1999

10. Probabilistic Latent Semantic Analysis
Unsupervised technique
Two-level generative model: a document is a mixture of topics, and each topic has its own characteristic word distribution w d z
document
topic
word
P(z|d)
P(w|z)
T. Hofmann, Probabilistic Latent Semantic Analysis, UAI 1999

11. Probabilistic Latent Semantic Analysis
Unsupervised technique
Two-level generative model: a document is a mixture of topics, and each topic has its own characteristic word distribution w d z
T. Hofmann, Probabilistic Latent Semantic Analysis, UAI 1999

12. The pLSA model Probability of word i in document j (known)
Probability of word i given topic k (unknown)
Probability of topic k given document j (unknown)
For that we will use a method called probabilistic latent semantic analysis.
pLSA can be thought of as a matrix decomposition.
Here is our term-document matrix (documents, words) and we want to
find topic vectors common to all documents and mixture coefficients P(z|d) specific to each document.
Note that these are all probabilites which sum to one.
So a column here is here expressed as a convex combination of the topic vectors.
What we would like to see is that the topics correspond to objects.
So an image will be expressed as a mixture of different objects and backgrounds.

13. Codeword distributions
The pLSA model
documents
topics
documents
p(wi|dj)
p(wi|zk)
p(zk|dj)
words
words
topics =
For that we will use a method called probabilistic latent semantic analysis.
pLSA can be thought of as a matrix decomposition.
Here is our term-document matrix (documents, words) and we want to
find topic vectors common to all documents and mixture coefficients P(z|d) specific to each document.
Note that these are all probabilites which sum to one.
So a column here is here expressed as a convex combination of the topic vectors.
What we would like to see is that the topics correspond to objects.
So an image will be expressed as a mixture of different objects and backgrounds.
Observed codeword
distributions
(M×N)
Codeword distributions
per topic (class)
(M×K)
Class distributions
per image
(K×N)

14. Learning pLSA parameters
Maximize likelihood of data:
Observed counts of word i in document j
We use maximum likelihood approach to fit parameters of the model.
We write down the likelihood of the whole collection and maximize it using EM.
M,N goes over all visual words and all documents.
P(w|d) factorizes as on previous slide the entries in those matrices are our parameters.
n(w,d) are the observed counts in our data
M … number of codewords
N … number of images
Slide credit: Josef Sivic

15. Inference Finding the most likely topic (class) for an image:
We use maximum likelihood approach to fit parameters of the model.
We write down the likelihood of the whole collection and maximize it using EM.
M,N goes over all visual words and all documents.
P(w|d) factorizes as on previous slide the entries in those matrices are our parameters.
n(w,d) are the observed counts in our data

16. Inference Finding the most likely topic (class) for an image:
Finding the most likely topic (class) for a visual word in a given image:
We use maximum likelihood approach to fit parameters of the model.
We write down the likelihood of the whole collection and maximize it using EM.
M,N goes over all visual words and all documents.
P(w|d) factorizes as on previous slide the entries in those matrices are our parameters.
n(w,d) are the observed counts in our data

17. Topic discovery in images
J. Sivic, B. Russell, A. Efros, A. Zisserman, B. Freeman, Discovering Objects and their Location in Images, ICCV 2005

18. From single features to “doublets”
Run pLSA on a regular visual vocabulary
Identify a small number of top visual words for each topic
Form a “doublet” vocabulary from these top visual words
Run pLSA again on the augmented vocabulary
J. Sivic, B. Russell, A. Efros, A. Zisserman, B. Freeman, Discovering Objects and their Location in Images, ICCV 2005

19. From single features to “doublets”
Ground truth
All features
“Face” features initially found by pLSA
One doublet
Another doublet
“Face” doublets
J. Sivic, B. Russell, A. Efros, A. Zisserman, B. Freeman, Discovering Objects and their Location in Images, ICCV 2005

20. Summary: Generative models
Naïve Bayes
Unigram models in document analysis
Assumes conditional independence of words given class
Parameter estimation: frequency counting
Probabilistic Latent Semantic Analysis
Unsupervised technique
Each document is a mixture of topics (image is a mixture of classes)
Can be thought of as matrix decomposition
Parameter estimation: Expectation-Maximization