1. Optimal Bandwidth Selection for MLS Surfaces
Hao Wang
Carlos E. Scheidegger
Claudio T. Silva
SCI Institute – University of Utah
Shape Modeling International 2008 – Stony Brook University

2. Point Set Surfaces Levin’s MLS formulation
Shape Modeling International 2008 – Stony Brook University

3. Neighborhood and Bandwidth
Three parameters in both steps of Levin’s MLS:
Weight function
Neighborhood
Bandwidth
Second Step of Levin’s MLS is weighted least squares polynomial fitting
Bandwidth determination is important because of the overfitting/underfitting problem
Overfitting
Underfitting
Shape Modeling International 2008 – Stony Brook University

4. Neighborhood and Bandwidth
Common practice
Weight function: Exponential
Neighborhood: Spherical
Bandwidth: Heuristics
Problems
Optimality
Anisotropic Dataset
Weight function: Exponential function is computationally intensive. Other function may lead to higher computational efficiency or better reconstruction quality. The APSS paper (SIGGRAPH 2007) uses (1-x^2)^4 as the weight function.
Neighborhood: Spherical neighborhood may not deal with anisotropic data properly. Other neighborhood shape may lead to better reconstruction quality.
Bandwidth: Empirically choose bandwidth based on trial and error involves human interaction. In addition, even though heuristic methods can produce visually acceptable reconstruction, the result might not be geometrically accurate.
Shape Modeling International 2008 – Stony Brook University

5. Related Work Other MLS Formulations Alexa et al.
Guennebaud et al.
Robust Feature Extraction
Fleishman et al.
Bandwidth Determination
Adamson et al.
Lipman et al.
Guennebaud et al. “APSS” SIGGRAPH 2007

6. Locally Weighted Kernel Regression
Problem
Points sampled from functional with white noise added
White noise are i.i.d. random variables
Reconstruct the functional with least squares criterion
Approach
Consider each point p individually
p is reconstructed by utilizing information of its neighborhood
Influence of each neighboring point is related to its distance from p
Shape Modeling International 2008 – Stony Brook University

7. Kernel Regression v.s MLS Surfaces
Kernel Regression is mostly the same as the second step in Levin’s MLS.
The only difference is between kernel weighting and MLS weighting.
Shape Modeling International 2008 – Stony Brook University

8. Kernel Regression v.s MLS Surfaces
Difference
Kernel weighting for functional data
MLS weighting for manifold data
Advantages of Kernel Regression
More mature technique for processing noisy sample points
Behavior of the neighborhood and kernel better studied
Goal
Adapt techniques in kernel regression to MLS surfaces
Extend theoretical results of kernel regression to MLS surfaces
Shape Modeling International 2008 – Stony Brook University

9. Weight Function
Common choices of weight functions in kernel regression:
Epanechnikov
Normal
Biweight
Optimal weight function: Epanechnikov
Choice of weight function not important
Implication:
Optimality
Most weight functions produce results with ignorable differences
Implications:
Exponential weight function in Levin’s MLS can be replaced by weight functions requiring less computational effort
Weight function in Levin’s MLS is indeed a near optimal choice
Shape Modeling International 2008 – Stony Brook University

10. Evaluation of Kernel Regression
MSE
MSE = Mean Squared Error
Evaluate result of the functional fitting at each point
Shape Modeling International 2008 – Stony Brook University

11. Evaluation of Kernel Regression
MISE
Integration of MSE over the domain
Evaluate the global performance of kernel regression
Shape Modeling International 2008 – Stony Brook University

12. Optimal Bandwidth Optimality Computation Leading to minimum MSE / MISE
Each point with a different optimal bandwidth
Computation
MSE / MISE approximated by Taylor Polynomial
Solve for the minimizing bandwidth
Computation
MSE / MISE can be approximated by a Taylor polynomial
The Taylor polynomial involves bandwidth and thus can be considered as a function of bandwidth
Solve analytically for minimizing bandwidth of the polynomial by setting its derivative to be 0 and solve the equation
Shape Modeling International 2008 – Stony Brook University

13. Optimal Bandwidth Approach Unknown quantities in computation
Derivatives of underlying functional
Variance of random noise variables
Density of point set
Approach
Derivatives: Ordinary Least Squares Fitting
Variance: Statistical Inference
Density: Kernel Density Estimation
Computation of bandwidth is difficult because it involves unknown quantities:
Taylor polynomial involves derivatives of underlying functional
Taylor polynomial involves variance of random noise variables
Taylor polynomial may involve density of point set
Solution
Underlying functional approximated by ordinary least squares fitting
Derivatives approximated by derivatives of approximated functional
Variance of random noise variables estimated by statistical inference
Density of point set estimated by kernel density estimation
Shape Modeling International 2008 – Stony Brook University

14. Optimal Bandwidth in 2-D
Optimal bandwidth based on MSE:
Interpretation
Higher noise level : larger bandwidth
Higher curvature : smaller bandwidth
Higher density : smaller bandwidth
More point samples : smaller bandwidth
Shape Modeling International 2008 – Stony Brook University

15. Optimal Bandwidth in 3-D
Kernel Function:
with
Kernel Shape:
Shape Modeling International 2008 – Stony Brook University

16. Optimal Bandwidth in 3-D
Optimal spherical bandwidth based on MSE:
Optimal spherical bandwidth based on MISE:
Shape Modeling International 2008 – Stony Brook University

17. Experiments Bandwidth selectors choose near optimal bandwidths
Shape Modeling International 2008 – Stony Brook University

18. Experiments
Shape Modeling International 2008 – Stony Brook University

19. Experiments
Shape Modeling International 2008 – Stony Brook University

20. Optimal Bandwidth in MLS
From functional domain to manifold domain
Choose a functional domain
Use kernel regression with modification
There are 2 ways to do this:
1. Choose functional domain using k-NN or constant radius sphere and apply kernel regression with optimal bandwidth
2. Use kernel regression with modification
Approximate underlying function using MLS with heuristically chosen bandwidths
Approximate unknown quantities in optimal bandwidth formulas using approximated MLS surface
Shape Modeling International 2008 – Stony Brook University

21. Robustness Insensitivity to error in first step of Levin’s MLS
Reference plane found by the first step is rotated by angle theta, as shown on x-axis. The y-axis shows the mean value of the bandwidth / reconstruction error. The angle v.s bandwidth plot shows a curve similar to plot of cosine function because by rotating theta, roughly cos(theta) of original point set remains.
Shape Modeling International 2008 – Stony Brook University

22. Comparison Constant h: uniform v.s non-uniform sampling
k-NN: sampling v.s feature
MSE/MISE based plug-in method: most robust and flexible
Constant h: Not suitable for non-uniformly point samples because of its lack of local adaptation.
k-NN: Can not distinguish high noise level and high curvature
MSE/MISE based plug-in method:
Work for both regularly and irregularly sampled points
Capable of distinguishing noise and curvature
Shape Modeling International 2008 – Stony Brook University

23. Comparison MSE/MISE-based plug-in method better than heuristic methods
Shape Modeling International 2008 – Stony Brook University

24. Comparison
Heuristic methods can produce visually acceptable but not geometrically accurate reconstruction.
Shape Modeling International 2008 – Stony Brook University

25. Future Work Nonlinear kernel regression bandwidth selector in 3-D
Compute optimal bandwidth implicitly
Extend the method to other MLS formulations
Nonlinear kernel regression bandwidth selector in 3-D
Derivation mathematically intense
Formulas need to be simplified
Compute optimal bandwidth implicitly
Avoiding computing bandwidth analytically may lead to do simpler method
Extend the method to other MLS formulations
Shape Modeling International 2008 – Stony Brook University