1. Latent SVMs for Human Detection with a Locally Affine Deformation Field
Ľubor Ladický1 Phil Torr2 Andrew Zisserman1
1 University of Oxford Oxford Brookes University 1 1 1

2. Human Detection Find all objects of interest
Enclose them tightly in a bounding box 2

3. Human Detection Find all objects of interest
Enclose them tightly in a bounding box 3

4. HOG Detector Sliding window using learnt HOG template
Post-processing using non-maxima suppression
Dalal & Triggs CVPR05

5. HOG Detector Sliding window using learnt HOG template
Post-processing using non-maxima suppression
Dalal & Triggs CVPR05

6. HOG Detector Sliding window using learnt HOG template
Post-processing using non-maxima suppression
Dalal & Triggs CVPR05

7. HOG Detector Sliding window using learnt HOG template
Post-processing using non-maxima suppression
Dalal & Triggs CVPR05

8. HOG Detector Sliding window using learnt HOG template
Post-processing using non-maxima suppression
Dalal & Triggs CVPR05

9. HOG Detector Sliding window using learnt HOG template
Post-processing using non-maxima suppression
Dalal & Triggs CVPR05

10. HOG Detector Sliding window using learnt HOG template
Post-processing using non-maxima suppression
Dalal & Triggs CVPR05

11. HOG Detector Does not fit well !
Sliding window using learnt HOG template
Post-processing using non-maxima suppression
Dalal & Triggs CVPR05

12. Deformable Part-based Model
Allows parts to move relative to the centre
Effectively allows the template to deform
Multiple models based on an aspect ratio
Felzenszwalb et al. CVPR08

13. Deformable Part-based Model
Allows parts to move relative to the centre
Effectively allows the template to deform
Multiple models based on an aspect ratio
Felzenszwalb et al. CVPR08

14. Comparison with other approaches
HOG template
(no deformation)
Part-based model
(rigid movable parts)
Our model
(deformation field) 14

15. HOG Detector Classifier response : Dalal & Triggs CVPR05 weights bias
HOG feature
over the set of cells c
Dalal & Triggs CVPR05

16. HOG Detector Classifier response :
The weights w* and the bias b* learnt using the Linear SVM as:
regularization
number of training samples
hinge loss
ground truth labels
Dalal & Triggs CVPR05

17. Detector with Deformation Field
Cells c displaced by the deformation field d:

18. over the deformed template Regularization of the deformation field
Detector with Deformation Field
Cells c displaced by the deformation field d:
Classifier response :
HOG feature
over the deformed template
Regularization of the deformation field

19. Detector with Deformation Field
Cells c displaced by the deformation field d:
Classifier response :
Regularisation takes the form of a smoothness MRF prior:
Pairwise weight
Pairwise cost

20. Detector with Deformation Field
Cells c displaced by the deformation field d:
Classifier response :
The weights w* and the bias b* learnt using the Latent Linear SVM as:

21. Why hasn’t anyone tried it before?
Detector with Deformation Field
Why hasn’t anyone tried it before?

22. Why hasn’t anyone tried it before?
Detector with Deformation Field
Why hasn’t anyone tried it before?
Latent models with many latent variables tend to over-fit
Inference not feasible for a sliding window

23. To resolve these problems we propose:
Detector with Deformation Field
To resolve these problems we propose:
Flexible constraints on the deformation field which avoid over-fitting
Feasible inference method under these constraints
Clustering of the training data into multiple models

24. Locally Affine Deformation Field
We restrict the deformation field to be locally affine ( ):

25. Locally Affine Deformation Field
We restrict the deformation field to be locally affine ( ):

26. Locally Affine Deformation Field
We restrict the deformation field to be locally affine ( ):

27. Locally Affine Deformation Field
We restrict the deformation field to be locally affine ( ):

28. Locally Affine Deformation Field
We restrict the deformation field to be locally affine ( ):

29. Locally Affine Deformation Field
We restrict the deformation field to be locally affine ( ):

30. Locally Affine Deformation Field
We restrict the deformation field to be locally affine ( ):

31. Locally Affine Deformation Field
We restrict the deformation field to be locally affine ( ):

32. Locally Affine Deformation Field
We restrict the deformation field to be locally affine ( ):

33. Locally Affine Deformation Field
We restrict the deformation field to be locally affine ( ):

34. Optimisation
Weights / bias (w*, b*) and the deformation fields dk estimated iteratively

35. Optimisation
Weights / bias (w*, b*) and the deformation fields dk estimated iteratively
Given the deformation fields the problem is a standard linear SVM:

36. Optimisation
Given (w*, b*) the problem is a constrained MRF optimisation:

37. Optimisation
Given (w*, b*) the problem is a constrained MRF optimisation:
The last can be decomposed as :

38. Optimisation
Given (w*, b*) the problem is a constrained MRF optimisation:
The last can be decomposed as :
By defining
the optimisation becomes:

39. Didn’t we make the problem harder ?
Optimisation
Given (w*, b*) the problem is a constrained MRF optimisation:
The last can be decomposed as :
By defining
the optimisation becomes:
Didn’t we make the problem harder ?

40. Optimisation
The location of the cells in the first row and in the first column fully determine the location of each cell

41. Optimisation
The location of the cells in the first row and in the first column fully determine the location of each cell
Any locally affine deformation field can be reached by two moves :
move all columns where each column i can move by (Δcdix ,Δcdiy)
move all rows where each row j can move by (Δrdjx ,Δrdjy)

42. Optimisation
Columns move

43. Optimisation
Rows move

44. Optimisation
The location of the cells in the first row and in the first column fully determine the location of each cell
Any locally affine deformation field can be reached by two moves :
move all columns where each column i can move by (Δcdix ,Δcdiy)
move all rows where each row j can move by (Δrdjx ,Δrdjy)
These moves do not alter the local affinity
Columns move
Rows move

45. Optimisation
The location of the cells in the first row and in the first column fully determine the location of each cell
Any locally affine deformation field can be reached by two moves :
move all columns where each column i can move by (Δcdix ,Δcdiy)
move all rows where each row j can move by (Δrdjx ,Δrdjy)
These moves do not alter the local affinity
Both moves can be solved quickly using dynamic programming

46. Optimisation
The location of the cells in the first row and in the first column fully determine the location of each cell
Any locally affine deformation field can be reached by two moves :
move all columns where each column i can move by (Δcdix ,Δcdiy)
move all rows where each row j can move by (Δrdjx ,Δrdjy)
These moves do not alter the local affinity
Both moves can be solved quickly using dynamic programming
Works for any form of pairwise potentials

47. Learning multiple poses / viewpoints
We define a similarity measure between two training samples as :
where

48. Learning multiple poses / viewpoints
We define a similarity measure between two training samples as :
where
K-medoid clustering of S matrix clusters the data into multi models

49. Experiments Buffy dataset (typically used for pose estimation)
Contains large variety of poses, viewpoints and aspect ratios
Consists of 748 images
Episode s5e3 used for training
Episode s5e4 used for validation
Episodes s5e2, s5e5 and s5e6 used for testing
Ferrari et al. CVPR08

50. Clustering of training samples
Each row corresponds to one model (out of 10 models)

51. Qualitative results

52. Qualitative results

53. Qualitative results

54. Quantitative results

55. Conclusion and Further Work
We propose
Novel inference for locally affine deformation field (LADF)
Object detector using LADF
Clustering using LADF
Further work
Explore usability for other vision problems (tracking, flow)
Explore generalisations of LADF where the same inference method is applicable

56. Thank you
Questions? 56 56