1. Research Methods Lecturer: Steve Maybank
Department of Computer Science and Information Systems
Autumn 2015
Data Research Methods in Computer Vision
11 November 2015
Birkbeck College, U. London

2. Birkbeck College, U. London
Digital Images 95
110 40 34
125
108 25 91
158
116 59
112
166
132
101
124
A digital image is a rectangular array
of pixels. Each pixel has a position
and a value.
Original colour image from the Efficient
Content Based Retrieval Group, University
of Washington
11 November 2015
Birkbeck College, U. London

3. Birkbeck College, U. London
Size of Images
Digital camera, 5,000x5,000 pixels, 3 bytes/pixel -> 75 MB.
Surveillance camera at 25 f/s ->
1875 MB/s.
1000 surveillance cameras -> ~1.9 TB/s.
Not all of these images are useful!
11 November 2015
Birkbeck College, U. London

4. Birkbeck College, U. London
Image Compression
Divide the image into blocks, and compress each block separately, e.g. JPEG uses 8x8 blocks.
Lossfree compression: the original image can be recovered exactly from the compressed image.
Lossy compression: the original image cannot be recovered.
11 November 2015
Birkbeck College, U. London

5. Why is Compression Possible?
Natural image: values of
neighbouring pixels are
strongly correlated.
White noise image: values of
neighbouring pixels are not
correlated. Compression discards
information.
11 November 2015
Birkbeck College, U. London

6. Birkbeck College, U. London
Measurement Space
Vectors from 8x8 blocks
R64
Each 8x8 block yields a vector in R64. The vectors
from natural images tend to lie in a low dimensional
subspace of R64.
11 November 2015
Birkbeck College, U. London

7. Strategy for Compression
vectors from 8x8 blocks
Choose a basis for R64 in which the low dimensional
subspace is spanned by the first few coordinate vectors.
Retain these coordinates and discard the rest.
11 November 2015
Birkbeck College, U. London

8. Discrete Cosine Transform
11 November 2015
Birkbeck College, U. London

9. Basis Images for the DCT
UTe(1)
UTe(2)
UTe(3)
UTe(4)
11 November 2015
Birkbeck College, U. London

10. Example of Compression using DCT
Image constructed from 3 DCT
coefficients in each 8x8 block.
Original image
11 November 2015
Birkbeck College, U. London

11. Linear Classificationy y
Given two sets X, Y of
measurement vectors from
different classes, find a hyperplane
that separates X and Y. y y x x x x x x
A new vector is assigned to the
class of X or to the class of Y,
depending on its position relative
to the hyperplane.
11 November 2015
Birkbeck College, U. London

12. Fisher Linear Discriminant
11 November 2015
Birkbeck College, U. London

13. Birkbeck College, U. London
Maximisation of J(v)
11 November 2015
Birkbeck College, U. London

14. Edge Regions and Regions Without a Peak
3x3 blocks such that the grey level of
the central pixel equals the mean grey
level of the 9 pixels
3x3 blocks with large Sobel gradients
11 November 2015
Birkbeck College, U. London

15. Projections Onto a Random Plane
Sobel edges
Regions without
a peak
Superposed plots
11 November 2015
Birkbeck College, U. London

16. Projections Onto a 1-Dimensional FLD
Sobel edges
Regions without
a peak
Combined
histograms
11 November 2015
Birkbeck College, U. London

17. Discrete Distribution
A probability distribution on a discrete set S={1, 2,…, n} is a set of numbers 𝑝 𝑖 such that
0≤ 𝑝 𝑖 ≤1
𝑖=1 𝑛 𝑝 𝑖 =1
11 November 2015
Birkbeck College, U. London

18. Birkbeck College, U. London
Interpretations
Bayes: 𝑝 𝑖 is a measure of our knowledge that item i will be chosen from S.
Frequentist: if a large number m of independent samples from S are chosen then i occurs approximately 𝑚 𝑝 𝑖 times
11 November 2015
Birkbeck College, U. London

19. Birkbeck College, U. London
Terminology
Event: subset of S
Probability of event E:
P E = 𝑖𝜖𝐸 𝑝 𝑖
Conditional Probability:
𝑃 𝐸 𝐹 = 𝑃(𝐸∩𝐹) 𝑃(𝐹)
11 November 2015
Birkbeck College, U. London

20. Birkbeck College, U. London
Example
Roll two dice. F=event that total is 8.
S={(i, j), 1<=i, j<=6}
The pairs (i, j) all have the same probability, thus
P({i, j})=1/36, 1<=i<=36
F={(6,2), (2,6), (3,5), (5,3), (4,4)}
P(F) = 5/36
11 November 2015
Birkbeck College, U. London

21. Example of a Conditional Probability
E={(6,2)}. What is the probability of E given F?
𝑃 𝐸 𝐹 = 𝑃 𝐸∩𝐹 𝑃 𝐹 = 𝑃 𝐸 𝑃 𝐹 = /36 =1/5
11 November 2015
Birkbeck College, U. London

22. Birkbeck College, U. London
Independent Events
The events E, F are independent if
𝑃 𝐸∩𝐹 =𝑃 𝐸 𝑃 𝐹
Example: E=first number is 6
F=second number is 5
𝑃 𝐸∩𝐹 = 1 36
P(E) = 1/6, P(F)=1/6
11 November 2015
Birkbeck College, U. London

23. Probability Density Function
A pdf of the real line R is a function
𝑓:𝑅→𝑅
such that
𝑓 𝑥 ≥0,𝑥𝜖𝑅
−∞ ∞ 𝑓 𝑥 𝑑𝑥=1
A pdf is used to assign probabilities to subsets of R:
P(A) = 𝐴 𝑓𝑑𝑥
11 November 2015
Birkbeck College, U. London

24. Birkbeck College, U. London
The Gaussian PDF
𝑓 𝑥 = (2𝜋) −1/2 𝑒 − 𝑥 2 /2
Mean value: −∞ ∞ 𝑥𝑓 𝑥 𝑑𝑥
Variance: −∞ ∞ 𝑥 2 𝑓 𝑥 𝑑𝑥
11 November 2011
Birkbeck College, U. London

25. Estimation of Parameters
Given samples 𝑥 1 , 𝑥 2 , … 𝑥 𝑛 in R from a probability distribution, estimate the pdf, assuming it is Gaussian
Mean value: 𝜇= 1 𝑛 𝑖=1 𝑛 𝑥 𝑖
Variance: 𝜎 2 = 1 𝑛 𝑖=1 𝑛 ( 𝑥 𝑖 −𝜇) 2
𝑓 𝑥 =(2𝜋 𝜎 2 ) −1/2 𝑒 − 𝑥−𝜇 2 /(2 𝜎 2 )
11 November 2015
Birkbeck College, U. London

26. Birkbeck College, U. London
Gaussian pdf in 2D
𝑓 𝑥,𝑦 = (2𝜋) −1 𝑒 −( 𝑥 2 + 𝑦 2 )/2
11 November 2015
Birkbeck College, U. London

27. Estimation of a Multivariate Gaussian
Mean value in 𝑅 𝑑 : μ= 1 𝑛 𝑖=1 𝑛 𝑥 𝑖
Covariance matrix:
𝐶 𝑗𝑘 = 1 𝑛 𝑖=1 𝑛 ( 𝑥 𝑖 −𝜇) 𝑗 ( 𝑥 𝑖 −𝜇) 𝑘
𝑓 𝑥 = 2𝜋 det 𝐶 − 𝑑 2 𝐸𝑥𝑝 − 𝑥−𝜇 𝑇 𝐶 −1 (𝑥−𝜇)
11 November 2015
Birkbeck College, U. London

28. Histogram of a DCT Coefficient
The pdf is leptokurtic, i.e. it has a peak at 0
and “fat tails”.
11 November 2015
Birkbeck College, U. London

29. Sparseness of the DCT Coefficients
For a given 8×8 block, only a few DCT coefficients are significantly different from 0.
Given a DCT coefficient of low to moderate frequency, there exist some blocks for which it is large.
11 November 2015
Birkbeck College, U. London

30. Birkbeck College, U. London
Whitening the Data
11 November 2015
Birkbeck College, U. London

31. Independence and Correlation
11 November 2015
Birkbeck College, U. London

32. Independent Components Analysis
Let z be a rv with values in R^n such that cov(z)=I.
Find an orthogonal matrix V such that the components of s =Vz are as independent as possible.
11 November 2015
Birkbeck College, U. London

33. Birkbeck College, U. London
PDF for Image Blocks
Assume a particular leptokurtic pdf p for the components si, e.g. a two sided Laplace density.
Find the orthogonal matrix V with rows v1,…,vn that maximises the product
p(s1)…p(sn) = p(v1.z)…p(vn.z)
11 November 2015
Birkbeck College, U. London

34. Birkbeck College, U. London
Example
Natural Image Statistics: a probabilistic approach to early computational vision, by A. Hyvarinen, J. Hurri and P.O. Hoyer. Page 161.
ICA applied to image patches yields Gabor type filters similar to those found in the human visual system.
11 November 2015
Birkbeck College, U. London

35. Birkbeck College, U. London
Stereo Pair of Images
UC Berkeley looking west
11 November 2015
Birkbeck College, U. London

36. Birkbeck College, U. London
Salience
A left hand image region is salient if the correct right hand matching region can be found with a high probability.
H: hypothesis that (v(1),v(2)) is a matching pair.
B: hypothesis that (v(1),v(2)) is a background pair
11 November 2015
Birkbeck College, U. London

37. Formulation using PDFs
The region yielding v(1) is salient if
p(v(2)|v(1), H)
differs significantly from
p(v(2)|v(1), B)
Measurement of difference: the Kullback-Leibler divergence:
11 November 2015
Birkbeck College, U. London

38. Illustration of the Kullback-Leibler Divergence
KL divergence = 0.125
KL divergence = 0.659
See S.J. Maybank (2013) A probabilistic definition of salient
regions for image matching. Neurocomputing v. 120, pp
11 November 2015
Birkbeck College, U. London