1. Likelihood Models for Template Matching Using the PDF Projection Theorem Arasanathan Thayananthan Ramanan Navaratnam Dr. Phil Torr Prof. Roberto Cipolla

2. Problem The correct template The minimum chamfer score Chamfer score3.493.07

3. Overview Problem Motivation PDF Projection Theorem Likelihood Modelling for Chamfer Matching Experiments Conclusion

4. Motivation Template matching widely used in computer vision Similarity measures are obtained from matching a template to a new image e.g. chamfer score, cross-correlation, etc. A likelihood value need to be calculated from the similarity measures. Chamfer score 3.58 Likelihood ?

5. Motivation Is the similarity measure alone enough to calculate the likelihood ? What are the probabilities of matching to a correct image and an incorrect image at this specific matching measure ?

6. Feature Likelihood Feature likelihood distributions, obtained by matching the templates to the real images they represent They differ according to the shape and scale of the templates.

7. Feature Likelihoods chamfer 6.0 likelihood 0.14likelihood 0.03

8. Clutter Likelihoods Clutter likelihood distributions are obtained by matching the template to the background clutter

9. Likelihood Ratios The ratio of the feature and clutter likelihood provides a robust likelihood measure. Likelihood Ratio Tests (LRT) are often used in many classification problems Jones & Ray [99], skin-colour classification Sidenbladh & Black [01], limb-detector

10. Modelling the likelihood Need a principled framework for modelling the likelihood for template matching Probability Distribution Function Projection Theorem ( Baggenstoss [99]) provides such a framework

11. Overview Problem Motivation PDF Projection Theorem Likelihood Modelling for Chamfer Matching Experiments Conclusion

12. PDF Projection Theorem PDF Projection Theorem Provides a mechanism to work in raw data space, I, instead of extracted feature space, z. This is done by projecting the PDF estimates from the feature space back to the raw data space

13. PDF Projection Theorem Neyman-Fisher factorisation states that if is a sufficient statistic for H, p(I|H) can be factored as Applying Eq(1) for a hypothesis, H, and a reference Hypothesis, H 0,

14. PDF Projection Theorem Image space, I I

15. PDF Projection Theorem Image space, I Feature space, z I z

16. PDF Projection Theorem Image space, I Feature space, z I z

17. Class-specific features PDF Projection Theorem extends to class- specific features Each hypothesis or class can have its feature set Yet, we get consistent and comparable raw image likelihoods Reference hypothesis H 0 remains the same for all hypothesis

18. Class-specific features I

19. Overview Problem Motivation PDF Projection Theorem Likelihood Modelling for Chamfer Matching Experiments Conclusion

20. Chamfer Matching Input imageCanny edges Distance transform Template

21. Chamfer Matching We apply PDF projection Theorem to model likelihood in a chamfer matching scheme Each template chooses its own subset of edge features, z j

22. Chamfer Matching A common reference hypothesis is chosen for all templates p(z j |H 0 ) provides the probability of template matching to any image. Difficulty is in learning p(z j |H j ) and p(z j |H 0 ) for each template T j

23. Learning the PDFs Time-consuming to obtain real images for learning the PDFs Software like Poser can create near real images Becoming popular for learning image statistics e.g. Shakhnarovich [03] For each template T j, we learn p(z j |H j ) and p(z j |H 0 ) from synthetic images.

24. Learning the PDFs Example learning images for the template For learning the feature likelihood p(z j |H j ) For learning the reference likelihood p(z j |H 0 )

25. Overview Problem Motivation PDF Projection Theorem Likelihood Modelling for Chamfer Matching Experiments Conclusion

26. Experiments 35 hand templates from a 3D hand model with 5 gestures at 7 different scales Hypothesis, H j, is that the image contains a hand pose similar to Template T j, (in scale and gesture). The distributions p(z j |H j ) and p(z j |H 0 ) were learned off- line for each template.

27. Experiments Aim of the experiment is to compare the matching performances of 1. Z j, the chamfer score obtained by matching the template T j to the image 2. P(z j |H j ), the feature likelihood of Template T j 3. P(I|H j ), the data likelihood value using the PDF projection theorem.

28. Experiments Template matching on 1000 randomly created synthetic images. Each synthetic image contains a hand pose similar in scale and pose to a randomly chosen template. Three ROC curves were obtained for each matching measure.

29. Results

30. Results PDF Projection Theorem Chamfer Chamfer score4.964.06 feature likelihood14.59 x 10 -2 8.62 x 10 -2 reference likelihood88.69 x 10 -5 383.32 x 10 -5 data likelihood0.164 x 10 3 0.022 x 10 3

31. Results PDF Projection Theorem Chamfer Chamfer score4.964.06 feature likelihood14.59 x 10 -2 8.62 x 10 -2 reference likelihood88.69 x 10 -5 383.32 x 10 -5 data likelihood0.164 x 10 3 0.022 x 10 3

32. Results PDF Projection Theorem Chamfer Chamfer score4.964.06 feature likelihood14.59 x 10 -2 8.62 x 10 -2 reference likelihood88.69 x 10 -5 383.32 x 10 -5 data likelihood0.164 x 10 3 0.022 x 10 3

33. Results PDF Projection Theorem Chamfer Chamfer score4.964.06 feature likelihood14.59 x 10 -2 8.62 x 10 -2 reference likelihood88.69 x 10 -5 383.32 x 10 -5 data likelihood0.164 x 10 3 0.022 x 10 3

34. Results PDF Projection Theorem Chamfer Chamfer score3.493.07 feature likelihood24.94x 10 -2 27.88 x 10 -2 reference likelihood4.73 x 10 -5 24.7 x 10 -5 data likelihood5.27 x 10 3 1.126 x 10 3

35. Results PDF Projection Theorem Chamfer Chamfer score3.493.07 feature likelihood24.94x 10 -2 27.88 x 10 -2 reference likelihood4.73 x 10 -5 24.7 x 10 -5 data likelihood5.27 x 10 3 1.126 x 10 3

36. Results PDF Projection Theorem Chamfer Chamfer score3.493.07 feature likelihood24.94x 10 -2 27.88 x 10 -2 reference likelihood4.73 x 10 -5 24.7 x 10 -5 data likelihood5.27 x 10 3 1.126 x 10 3

37. Results PDF Projection Theorem Chamfer Chamfer score3.493.07 feature likelihood24.94x 10 -2 27.88 x 10 -2 reference likelihood4.73 x 10 -5 24.7 x 10 -5 data likelihood5.27 x 10 3 1.126 x 10 3

38. Results PDF Projection Theorem Chamfer Chamfer score3.723.54 feature likelihood13.15 x 10 -2 20.5 x 10 -2 reference likelihood8.5 x 10 -5 108.0 x 10 -5 data likelihood1.547 x 10 3 0.191 x 10 3

39. Conclusion Depending on raw matching score is less reliable in template matching PDF Projection theorem provides a principled framework for modelling the likelihood in raw image data space. Consistent and comparable likelihoods obtained through PDF projection theorem improves the efficiency of template matching scheme