1. Robust Statistics Robust Statistics Why do we use the norms we do? Henrik Aanæs IMM,DTU haa@imm.dtu.dk A good general reference is: Robust Statistics: Theory and Methods, by Maronna, Martin and Yohai. Wiley Series in Probability and Statistics TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA

2. How Tall are You ?

3. Idea of Robust Statistics To fit or describe the bulk of a data set well without being perturbed (influenced to much) by a small portion of outliers. This should be done without a pre-processing segmentation of the data. We thus now model our data set as consisting of inliers, that follow some distribution, at outliers which do not. InliersOutliers Outliers can be interesting too!

4. Line Example..............

5. Robust Statistics in Computer Vision Robust Statistics in Computer Vision Image Smoothing Image by Frederico D'Almeida

6. Robust Statistics in Computer Vision Robust Statistics in Computer Vision Image Smoothing

7. Robust Statistics in Computer Vision Robust Statistics in Computer Vision optical flow Play Sequence MIT BCS Perceptual Science Group. Demo by John Y. A. Wang.John Y. A. Wang

8. Robust Statistics in Computer Vision Robust Statistics in Computer Vision tracking via view geometry Image 1 Image 2

12. Gaussian/ Normal Distribution The Distribution We Usually Use Nice Properties: Central Limit Theorem. Induces two norm. Leads to linear computations. But: Is fiercely influenced by outliers. Empirical distributions often have ‘fatter’ tails.

13. Gaussians Just are Models Too Alternative title of this talk

14. Error or ρ -functions Error or ρ -functions Converting from Model-Data Deviation to Objective Function.

15. ρ -functions and ML ρ -functions and ML A typical way of forming ρ- functions

16. ρ -functions and ML II ρ -functions and ML II A typical way of forming ρ- functions

18. Typical ρ-functions Typical ρ-functions Where the Robustness in Practice Comes From 2-norm 1-norm Huber norm Truncated quadratic Bi-Squared General Idea: Down weigh outliers, i.e. ρ(x) should be ‘smaller’ for large |x|.

19. Typical ρ-functions Typical ρ-functions Where the Robustness in Practice Comes From 2-norm 1-norm Huber norm Truncated quadratic Bi-Squared Induced by Gaussian. Very non-robust. ‘Standard’ distribution.

20. Typical ρ-functions Typical ρ-functions Where the Robustness in Practice Comes From 2-norm 1-norm Huber norm Truncated quadratic Bi-Squared Quite Robust. Convex. Corresponds to Median.

21. The Median and the 1-Norm

22. The Median and the 1-Norm The Median and the 1-Norm Example with 2 observations

24. The Median and the 1-Norm The Median and the 1-Norm Example with more observations

25. Typical ρ-functions Typical ρ-functions Where the Robustness in Practice Comes From 2-norm 1-norm Huber norm Truncated quadratic Bi-Squared Quite Robust. Convex. Corresponds to Median.

26. Typical ρ-functions Typical ρ-functions Where the Robustness in Practice Comes From 2-norm 1-norm Huber norm Truncated quadratic Bi-Squared Mixture of 1 and two norm. Convex. Has nice theoretical properties.

27. Typical ρ-functions Typical ρ-functions Where the Robustness in Practice Comes From 2-norm 1-norm Huber norm Truncated quadratic Bi-Squared Discards Outlier’s. For inliers works as Gaussian. Has discontinues derivative.

28. Typical ρ-functions Typical ρ-functions Where the Robustness in Practice Comes From 2-norm 1-norm Huber norm Truncated quadratic Bi-Squared Discards Outlier’s. Smooth.

29. Quantifying Robustness Quantifying Robustness A peak at tools for analysis Bias vs. Variance

30. Quantifying Robustness Quantifying Robustness A peak at tools for analysis Related to variance, on the previous slide

31. Quantifying Robustness Quantifying Robustness You want to be robust over a range of models

32. Quantifying Robustness Quantifying Robustness A peak at tools for analysis Other measures (Similar): Breakage Point: How many outliers can an estimator handle and still give ‘reasonable’ results. Asymptotic bias: What bias does an outlier impose.

33. Back to Images here we have multiple ‘models’ To fit or describe the bulk of a data set well without being perturbed (influenced to much) by a small portion of outliers. This should be done without a pre-processing segmentation of the data.

34. Optimization Methods Typical Approach: 1.Find initial estimate. 2.Use Non-linear optimization and/or EM- algorithm. NB: In this course we have and will seen other methods e.g. with guaranteed convergence

35. Hough Transform Hough Transform One off the oldest robust methods in ‘vision’ Often used for initial estimate. Example from MatLab help Curse of Dimesionality = PROBLEM

36. RanSaC RanSaC Sampling in Hough space, better for higher dimensions In a Hough setting: 1. and 2. corresponds to finding a ‘good’ bin in Hough space. 3. Corresponds to calculating the value. RANdom SAmpling Consensus, RANSAC Iterate: 1.Draw minimal sample. 2.Fit model. 3.Evaluate model by Consensus. Run RanDemo.m

37. Ransac Ransac How many iterations InliersOutliers Need to sample only Inliers to ‘succed’. Naïve scheme; try all combinations i.e. all E.g. For 100 points and a sample size of 7, this is 8.0678e+013 trials. Preferred stopping scheme: Stop when there is a e.g. 99% chance of getting all inliers. Chance of getting an inlier Use consesus of best fit as estimate of N_in See e.g. Hartley and Zisserman: “Multiple view geometry”

38. Iteratively Reweighted Least Squares IRLS Iteratively Reweighted Least Squares IRLS EM-type or chicken and egg optimization