Random Sampling Estimation Without Prior Scale Charles V. Stewart Department of Computer Science Rensselaer Polytechnic Institute Troy, NY 12180-3590 USA

Motivating Example Is it possible to automatically estimate the structures in the data without knowing the noise properties and without any one structure containing at least a minimum fraction of the data?

Notation: Set of data points, e.g. (x,y,z) measures Corresponding pixel coordinates Parameter vector - the goal of the estimation Function giving the “fitting error” or “residual” The form of the model, e.g. a line, a plane, a homography, is implicit here Objective function to be minimized Order statistics of the residuals, usually unsigned (non-negative).

Outline of Generic Random Sampling Algorithm For j = 1,…,S Select a minimal subset {xj,1,…,xj,k} of the {x1,…,xN} points Generate an estimate j from the minimal subset Evaluate h(j) on the N-k points not in the minimal subset Retain j as the best if h(j) is the minimum thus far. Denote the best as j*. Gather the inliers to the best estimate, j*, and refine using (weighted) least-squares Variations on this algorithm have focused on efficiency improvements

Objective Functions of “Classical” Estimators Least-Median (Rousseeuw 1984) objective function uses the median of the (square) order statistics: RANSAC (Fischler and Bolles 1981) objective function: This reverses the original form of RANSAC. As written here, RANSAC is designed to minimize the number of points outside an inlier bound, T

Enhancements MSAC and MLESAC Kernel-based methods

Underlying Assumptions LMS: Minimum fraction of inliers is known RANSAC: Inlier bound is known

Why Might This Be Unsatisfying? Structures may be “seen” in data despite unknown scale and large outlier fractions Potential unknown properties: Sensor characteristics Scene complexity Performance of low-level operations Problems: Handling unknown scale Handling varying scale (heteroscedasticity) 45% random outliers 11% from 2nd structure 44% from 1st structure

Goal A robust objective function, suitable for use in random-sampling algorithm, that is Invariant to scale, Does not require a prior lower bound on the fraction of inliers

Approaches MINPRAN (IEEE T-PAMI Oct 1995) Discussed briefly today MUSE (IEEE CVPR 1996, Jim Miller’s PhD 1997), with subsequent, unpublished improvements Based on order statistics of residuals Focus of today’s presentation Code available in VXL and on the web Other order-statistics based methods: Lee, Meer and Park, PAMI 1998 Bab-Hadiashar and Suter, Robotica 1999 Kernel-density techniques Chen-Meer ECCV 2002 Wang and Suter, PAMI 2004 Subbarao and Meer, RANSAC-25 2006

MINPRAN: Minimize Probability of Randomness 26 inliers within +/- 8 units of random-sample-generated line 72 inliers within +/- 7 units of random-sample-generated line 55 inliers within +/- 2 units of random-sample-generated line 65% outliers

MINPRAN: Probability Measure Probability of having at least k points within error distance +/- r if all errors follow a uniform distribution within distance +/- Z0: Lower values imply it is less likely that the residuals are uniform Good estimates, with appropriately chosen values of r (inlier bound) and k (number of inliers), have extremely low probability values r

MINPRAN: Objective Function For a given estimate, j, monotonicity properties restrict the necessary evaluations to just the order statistics

MINPRAN: Discussion O(S N log N + N2) algorithm Excellent results for single structure Limitations: Requires a background distribution Tends to “bridge” discontinuities

Toward MUSE: Ordered Residuals of Good and Bad Estimates Objective function should capture: Ordered residuals are lower for inliers to “good” estimate than for “bad” estimate Transition from inliers to outliers in “good” estimate

Statistics of Order Statistics Density and distribution of absolute residuals Scale normalized residuals and order statistics To a good approximation, the expected value of kth normalized order statistic is simply The variance of the kth normalized order statistic is obtained using a Taylor expansion of F-1 Details omitted! See James V. Miller’s 1997 PhD thesis

Scale Estimates from Order Statistics Repeating, the kth normalized order statistic, assuming all residuals follow distribution F with unknown , is: Taking expected values on both sides: Rearranging isolates the unknown  Finally, we can obtain an unbiased estimate of , one for each order statistic:

Scale Estimates from Order Statistics: “Good” and “Bad” Estimates We have only assumed that the residuals follow a Gaussian distribution Scale estimates are much lower for the inliers to the good estimate Q: How do we determine which scale estimate to choose? Before addressing this we consider a scale estimate based on trimmed statistics

Trimmed Statistics Scale Estimates Expected value of sum of first k order statistics (remember, these are unsigned) Generating a scale estimate Computing the denominator based on a Gaussian distribution yields This estimate is more stable than the quantile version and is used in practice

Which of the O(N) Possible Scale Estimates? Original idea: choose smallest scale estimate. Problem: variance in scale estimates leads to instability and reintroduces bias toward small estimates This shows the ratio of the expected residual of the order statistic to its standard deviation.

Minimum (Variance) Unbiased Scale Estimate Choose the scale estimate from a set of order statistics that has the smallest standard deviation: There is a strong tendency for this choice to occur just before the transition from inliers to outliers. std[k] vs. k for “good” estimate std[k] vs. k for “bad” estimate

MUSE Objective Function Given parameter vector  calculate the (unsigned) residuals and their order statistics, Calculate the scale estimates (for the quantile scale estimate) Choose k that minimizes call this k* The objective function is

More About MUSE Remainder of the basic random sampling method is unchanged In practice: Use trimmed scale estimate with quantile-based standard deviation calculation Evaluate the standard deviation at a small set of k values, e.g. 0.2N, 0.25N, …, 0.85N Expected values and standard deviations of the uk:N are cached. Overall cost is O(SN logN)

S-K Refinement For given k*, find the number of residuals within Call this number N* Re-evaluate This produces less bias in “good” scale estimates while leaving “bad” scale estimates essentially unchanged.

Simulation: Scale Estimation Unit-mean Gaussian plus outliers

Simulations (Done in 2002) Structures: Compare against Single or multiple planar surfaces in R3 Single or multiple plane homography estimation Compare against LMS MSAC Chen-Meer, ECCV 2002, kernel density algorithm Weak dependence on prior scale estimate Bias is measured as the integral of the square estimation error, normalized by area Experiments done by Ying-Lin (Bess) Lee

60/40 Split, Known Scale: Step Best scenario for MUSE

60/40 Split, Known Scale: Crease MUSE and Chen-Meer comparable

60/40 Split, Known Scale: Perpendicular Planes The poorest scenario for MUSE

The Effect of Incorrect Scale Estimates on MSAC The effect of incorrect scale estimates is significant!

Results Discussion Similar results with plane homography estimation In most cases: Performance of MUSE is comparable to MSAC (RANSAC or MLESAC) when scale is known Performance of MUSE is better than MSAC (RANSAC / MLESAC) when scale is unknown Performance of kernel-density based algorithms is comparable to MUSE These have a very weak dependence on prior scale or use Median Absolute Deviation (MAD) to provide rough scale estimates.

Our Current Work at Rensselaer “Dual-Bootstrap” estimation of 2d registration with Keypoint indexing Start from single keypoint match Grow and refine estimate using statistically-controlled region growing and re-estimation MUSE as initial scale estimator M-estimator as parameter estimator Reliable decision criteria Algorithm substantially outperforms keypoint matching with RANSAC!

Exploiting Locality in the Dual-Bootstrap Algorithm

Exploiting Locality in the Dual-Bootstrap Algorithm

Exploiting Locality in the Dual-Bootstrap Algorithm

Locality vs. Globality Can you exploit local structure? Yes? No? Careful scale estimation M-estimator (really gradient-based) No? RANSAC with appropriately chosen cost function MSAC or MLESAC when scale is known MUSE or kernel-based method when scale is unknown Efficient algorithms Lack of local exploration is both the “blessed” and the “curse” of RANSAC

Back to MUSE Robust estimation when scale is unknown Issues: Accuracy comparable to MSAC / MLESAC Issues: Stopping criteria Efficiency improvements Heteroscedasticity

C++ Code Available http://www.vision.cs.rpi.edu Stand-alone MUSE objective functions, including statistics of order statistics Dual-Bootstrap executable and test suite vxl.sourceforge.net rrel library contributed by Rensselaer