1. Flexible Metric NN Classification based on Friedman (1995) David Madigan

2. Nearest-Neighbor Methods k-NN assigns an unknown object to the most common class of its k nearest neighbors Choice of k? (bias-variance tradeoff again) Choice of metric? Need all the training to be present to classify a new point (“lazy methods”) Surprisingly strong asymptotic results (e.g. no decision rule is more than twice as accurate as 1-NN)

4. Suppose a Regression Surface Looks like this: Flexible-metric NN Methods try to capture this idea… want this not this

5. FMNN Predictors may not all be equally relevant for classifying a new object Furthermore, this differential relevance may depend on the location of the new object FMNN attempts to model this phenomenon

6. Local Relevance Consider an arbitrary function f on R p If no values of x are known, have: Suppose x i =z, then:

7. Local Relevance cont. The improvement in squared error provided by knowing x i is: I 2 i (z) reflects the importance of the ith variable on the variation of f(x) at x i =z

8. Local Relevance cont. Now consider an arbitrary point z=(z 1,…,z p ) The relative importance of x i to the variation of f at x=z is: R 2 i (z)=0 when f(x) is independent of x i at z R 2 i (z)=1 when f(x) depends only on x i at z

9. Estimation Recall:

10. On To Classification For J-class classification have {y j }, j=1,…,J output variables, y j  {0,1},  y j =1. Can compute: Technical point: need to weight the observations to rectify unequal variances

11. The Machete Start with all data points R 0 Compute Then: Continue until R i contains K points M 1 th order statistic

12. Results on Artificial Data

13. Results on Real Data