Research Methods Lecturer: Steve Maybank Department of Computer Science and Information Systems sjmaybank@dcs.bbk.ac.uk Autumn 2015 Data Research Methods in Computer Vision 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Digital Images 95 110 40 34 125 108 25 91 158 116 59 112 166 132 101 124 A digital image is a rectangular array of pixels. Each pixel has a position and a value. Original colour image from the Efficient Content Based Retrieval Group, University of Washington 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Size of Images Digital camera, 5,000x5,000 pixels, 3 bytes/pixel -> 75 MB. Surveillance camera at 25 f/s -> 1875 MB/s. 1000 surveillance cameras -> ~1.9 TB/s. Not all of these images are useful! 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Image Compression Divide the image into blocks, and compress each block separately, e.g. JPEG uses 8x8 blocks. Lossfree compression: the original image can be recovered exactly from the compressed image. Lossy compression: the original image cannot be recovered. 11 November 2015 Birkbeck College, U. London

Why is Compression Possible? Natural image: values of neighbouring pixels are strongly correlated. White noise image: values of neighbouring pixels are not correlated. Compression discards information. 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Measurement Space Vectors from 8x8 blocks R64 Each 8x8 block yields a vector in R64. The vectors from natural images tend to lie in a low dimensional subspace of R64. 11 November 2015 Birkbeck College, U. London

Strategy for Compression vectors from 8x8 blocks Choose a basis for R64 in which the low dimensional subspace is spanned by the first few coordinate vectors. Retain these coordinates and discard the rest. 11 November 2015 Birkbeck College, U. London

Discrete Cosine Transform 11 November 2015 Birkbeck College, U. London

Basis Images for the DCT UTe(1) UTe(2) UTe(3) UTe(4) 11 November 2015 Birkbeck College, U. London

Example of Compression using DCT Image constructed from 3 DCT coefficients in each 8x8 block. Original image 11 November 2015 Birkbeck College, U. London

Linear Classification y y Given two sets X, Y of measurement vectors from different classes, find a hyperplane that separates X and Y. y y x x x x x x A new vector is assigned to the class of X or to the class of Y, depending on its position relative to the hyperplane. 11 November 2015 Birkbeck College, U. London

Fisher Linear Discriminant 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Maximisation of J(v) 11 November 2015 Birkbeck College, U. London

Edge Regions and Regions Without a Peak 3x3 blocks such that the grey level of the central pixel equals the mean grey level of the 9 pixels 3x3 blocks with large Sobel gradients 11 November 2015 Birkbeck College, U. London

Projections Onto a Random Plane Sobel edges Regions without a peak Superposed plots 11 November 2015 Birkbeck College, U. London

Projections Onto a 1-Dimensional FLD Sobel edges Regions without a peak Combined histograms 11 November 2015 Birkbeck College, U. London

Discrete Distribution A probability distribution on a discrete set S={1, 2,…, n} is a set of numbers 𝑝 𝑖 such that 0≤ 𝑝 𝑖 ≤1 𝑖=1 𝑛 𝑝 𝑖 =1 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Interpretations Bayes: 𝑝 𝑖 is a measure of our knowledge that item i will be chosen from S. Frequentist: if a large number m of independent samples from S are chosen then i occurs approximately 𝑚 𝑝 𝑖 times 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Terminology Event: subset of S Probability of event E: P E = 𝑖𝜖𝐸 𝑝 𝑖 Conditional Probability: 𝑃 𝐸 𝐹 = 𝑃(𝐸∩𝐹) 𝑃(𝐹) 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Example Roll two dice. F=event that total is 8. S={(i, j), 1<=i, j<=6} The pairs (i, j) all have the same probability, thus P({i, j})=1/36, 1<=i<=36 F={(6,2), (2,6), (3,5), (5,3), (4,4)} P(F) = 5/36 11 November 2015 Birkbeck College, U. London

Example of a Conditional Probability E={(6,2)}. What is the probability of E given F? 𝑃 𝐸 𝐹 = 𝑃 𝐸∩𝐹 𝑃 𝐹 = 𝑃 𝐸 𝑃 𝐹 = 1 36 5/36 =1/5 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Independent Events The events E, F are independent if 𝑃 𝐸∩𝐹 =𝑃 𝐸 𝑃 𝐹 Example: E=first number is 6 F=second number is 5 𝑃 𝐸∩𝐹 = 1 36 P(E) = 1/6, P(F)=1/6 11 November 2015 Birkbeck College, U. London

Probability Density Function A pdf of the real line R is a function 𝑓:𝑅→𝑅 such that 𝑓 𝑥 ≥0,𝑥𝜖𝑅 −∞ ∞ 𝑓 𝑥 𝑑𝑥=1 A pdf is used to assign probabilities to subsets of R: P(A) = 𝐴 𝑓𝑑𝑥 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London The Gaussian PDF 𝑓 𝑥 = (2𝜋) −1/2 𝑒 − 𝑥 2 /2 Mean value: −∞ ∞ 𝑥𝑓 𝑥 𝑑𝑥 Variance: −∞ ∞ 𝑥 2 𝑓 𝑥 𝑑𝑥 11 November 2011 Birkbeck College, U. London

Estimation of Parameters Given samples 𝑥 1 , 𝑥 2 , … 𝑥 𝑛 in R from a probability distribution, estimate the pdf, assuming it is Gaussian Mean value: 𝜇= 1 𝑛 𝑖=1 𝑛 𝑥 𝑖 Variance: 𝜎 2 = 1 𝑛 𝑖=1 𝑛 ( 𝑥 𝑖 −𝜇) 2 𝑓 𝑥 =(2𝜋 𝜎 2 ) −1/2 𝑒 − 𝑥−𝜇 2 /(2 𝜎 2 ) 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Gaussian pdf in 2D 𝑓 𝑥,𝑦 = (2𝜋) −1 𝑒 −( 𝑥 2 + 𝑦 2 )/2 11 November 2015 Birkbeck College, U. London

Estimation of a Multivariate Gaussian Mean value in 𝑅 𝑑 : μ= 1 𝑛 𝑖=1 𝑛 𝑥 𝑖 Covariance matrix: 𝐶 𝑗𝑘 = 1 𝑛 𝑖=1 𝑛 ( 𝑥 𝑖 −𝜇) 𝑗 ( 𝑥 𝑖 −𝜇) 𝑘 𝑓 𝑥 = 2𝜋 det 𝐶 − 𝑑 2 𝐸𝑥𝑝 − 1 2 𝑥−𝜇 𝑇 𝐶 −1 (𝑥−𝜇) 11 November 2015 Birkbeck College, U. London

Histogram of a DCT Coefficient The pdf is leptokurtic, i.e. it has a peak at 0 and “fat tails”. 11 November 2015 Birkbeck College, U. London

Sparseness of the DCT Coefficients For a given 8×8 block, only a few DCT coefficients are significantly different from 0. Given a DCT coefficient of low to moderate frequency, there exist some blocks for which it is large. 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Whitening the Data 11 November 2015 Birkbeck College, U. London

Independence and Correlation 11 November 2015 Birkbeck College, U. London

Independent Components Analysis Let z be a rv with values in R^n such that cov(z)=I. Find an orthogonal matrix V such that the components of s =Vz are as independent as possible. 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London PDF for Image Blocks Assume a particular leptokurtic pdf p for the components si, e.g. a two sided Laplace density. Find the orthogonal matrix V with rows v1,…,vn that maximises the product p(s1)…p(sn) = p(v1.z)…p(vn.z) 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Example Natural Image Statistics: a probabilistic approach to early computational vision, by A. Hyvarinen, J. Hurri and P.O. Hoyer. Page 161. ICA applied to image patches yields Gabor type filters similar to those found in the human visual system. 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Stereo Pair of Images UC Berkeley looking west http://arch.ced.berkeley.edu/kap/wind/?p=32 11 November 2015 Birkbeck College, U. London

Birkbeck College, U. London Salience A left hand image region is salient if the correct right hand matching region can be found with a high probability. H: hypothesis that (v(1),v(2)) is a matching pair. B: hypothesis that (v(1),v(2)) is a background pair 11 November 2015 Birkbeck College, U. London

Formulation using PDFs The region yielding v(1) is salient if p(v(2)|v(1), H) differs significantly from p(v(2)|v(1), B) Measurement of difference: the Kullback-Leibler divergence: 11 November 2015 Birkbeck College, U. London

Illustration of the Kullback-Leibler Divergence KL divergence = 0.125 KL divergence = 0.659 See S.J. Maybank (2013) A probabilistic definition of salient regions for image matching. Neurocomputing v. 120, pp. 4-14. 11 November 2015 Birkbeck College, U. London