1. Research Methods Lecturer: Steve Maybank
Department of Computer Science and Information Systems
Autumn 2013
Data Research Methods in Computer Vision
13 November 2013
Birkbeck College, U. London

2. Birkbeck College, U. London
Digital Images 95
110 40 34
125
108 25 91
158
116 59
112
166
132
101
124
A digital image is a rectangular array
of pixels. Each pixel has a position
and a value.
Original colour image from the Efficient
Content Based Retrieval Group, University
of Washington
13 November 2013
Birkbeck College, U. London

3. Birkbeck College, U. London
Size of Images
Digital camera, 5,000x5,000 pixels, 3 bytes/pixel -> 75 MB.
Surveillance camera at 25 f/s ->
1875 MB/s.
1000 surveillance cameras -> ~1.9 TB/s.
Not all of these images are useful!
13 November 2013
Birkbeck College, U. London

4. Birkbeck College, U. London
Image Compression
Divide the image into blocks, and compress each block separately, e.g. JPEG uses 8x8 blocks.
Lossfree compression: the original image can be recovered exactly from the compressed image.
Lossy compression: the original image cannot be recovered.
13 November 2013
Birkbeck College, U. London

5. Why is Compression Possible?
Natural image: values of
neighbouring pixels are
strongly correlated.
White noise image: values of
neighbouring pixels are not
correlated. Compression discards
information.
13 November 2013
Birkbeck College, U. London

6. Birkbeck College, U. London
Measurement Space
Vectors from 8x8 blocks
R64
Each 8x8 block yields a vector in R64. The vectors
from natural images tend to lie in a low dimensional
subspace of R64.
13 November 2013
Birkbeck College, U. London

7. Strategy for Compression
vectors from 8x8 blocks
Choose a basis for R64 in which the low dimensional
subspace is spanned by the first few coordinate vectors.
Retain these coordinates and discard the rest.
13 November 2013
Birkbeck College, U. London

8. Discrete Cosine Transform
13 November 2013
Birkbeck College, U. London

9. Basis Images for the DCT
UTe(1)
UTe(2)
UTe(3)
UTe(4)
13 November 2013
Birkbeck College, U. London

10. Example of Compression using DCT
Image constructed from 3 DCT
coefficients in each 8x8 block.
Original image
13 November 2013
Birkbeck College, U. London

11. Linear Classificationy y
Given two sets X, Y of
measurement vectors from
different classes, find a hyperplane
that separates X and Y. y y x x x x x x
A new vector is assigned to the
class of X or to the class of Y,
depending on its position relative
to the hyperplane.
13 November 2013
Birkbeck College, U. London

12. Fisher Linear Discriminant
13 November 2013
Birkbeck College, U. London

13. Birkbeck College, U. London
Maximisation of J(v)
13 November 2013
Birkbeck College, U. London

14. Edge Regions and Regions Without a Peak
3x3 blocks such that the grey level of
the central pixel equals the mean grey
level of the 9 pixels
3x3 blocks with large Sobel gradients
13 November 2013
Birkbeck College, U. London

15. Projections Onto a Random Plane
Sobel edges
Regions without
a peak
Superposed plots
13 November 2013
Birkbeck College, U. London

16. Projections Onto a 1-Dimensional FLD
Sobel edges
Regions without
a peak
Combined
histograms
13 November 2013
Birkbeck College, U. London

17. Histogram of a DCT Coefficient
The pdf is leptokurtic, i.e. it has a peak at 0
and “fat tails”.
13 November 2013
Birkbeck College, U. London

18. Sparseness of the DCT Coefficients
For a given 8×8 block, only a few DCT coefficients are significantly different from 0.
Given a DCT coefficient of low to moderate frequency, there exist some blocks for which it is large.
13 November 2013
Birkbeck College, U. London

19. Birkbeck College, U. London
Whitening the Data
13 November 2013
Birkbeck College, U. London

20. Independence and Correlation
13 November 2013
Birkbeck College, U. London

21. Independent Components Analysis
Let z be a rv with values in R^n such that cov(z)=I.
Find an orthogonal matrix V such that the components of s =Vz are as independent as possible.
13 November 2013
Birkbeck College, U. London

22. Birkbeck College, U. London
Method
Assume a particular leptokurtic pdf p for the components si, e.g. a two sided Laplace density.
Find the orthogonal matrix V with rows v1,…,vn that maximises the product
p(s1)…p(sn) = p(v1.z)…p(vn.z)
13 November 2013
Birkbeck College, U. London

23. Birkbeck College, U. London
Example
Natural Image Statistics: a probabilistic approach to early computational vision, by A. Hyvarinen, J. Hurri and P.O. Hoyer. Page 161.
ICA applied to image patches yields Gabor type filters similar to those found in the human visual system.
13 November 2013
Birkbeck College, U. London

24. Birkbeck College, U. London
Stereo Pair of Images
UC Berkeley looking west
13 November 2013
Birkbeck College, U. London

25. Birkbeck College, U. London
Salience
A left hand image region is salient if the correct right hand matching region can be found with a high probability.
H: hypothesis that (v(1),v(2)) is a matching pair.
B: hypothesis that (v(1),v(2)) is a background pair
13 November 2013
Birkbeck College, U. London

26. Formulation using PDFs
The region yielding v(1) is salient if
p(v(2)|v(1), H)
differs significantly from
p(v(2)|v(1), B)
Measurement of difference: the Kullback-Leibler divergence:
13 November 2013
Birkbeck College, U. London

27. Illustration of the Kullback-Leibler Divergence
KL divergence = 0.125
KL divergence = 0.659
Preprint on salience available from SJM
13 November 2013
Birkbeck College, U. London