1. The point is class B via 3NNC.
2018/11/17
Concept of KNNC
Steps:
1. Find the first k nearest neighbors of a given point.
2. Determine the class of the given point by voting among these k nearest neighbors.
: class-A point
: class-B point
: point with unknown class
3-nearest neighbors
The point is class B via 3NNC.
Feature 2
2018/11/17
Feature 1

2. Flowchart for KNNC General flowchart of PR: KNNC: Feature extraction
2018/11/17
Flowchart for KNNC
General flowchart of PR:
KNNC:
Feature
extraction
From raw data to features
Data
modeling
Clustering (optional)
Evaluation
Distance Computation
For KNNC
2018/11/17

3. Decision Boundary for 1NNC
2018/11/17
Decision Boundary for 1NNC
Voronoi diagram: piecewise linear boundary
2018/11/17

4. Characteristics of KNNC
Strengths of KNNC
Most Intuitive
No data modeling required
Drawbacks of KNNC
Massive computation required when dataset is big
No straightforward way to determine the value of K
No straightforward way to rescale the dataset along each dimension
2018/11/17

5. Preprocessing/Variants for KNNC
2018/11/17
Preprocessing/Variants for KNNC
Preprocessing:
Data rescaling to have zero mean and unit variance along each feature
Value of K obtained via trials and errors
Variants:
Weighted votes
Nearest prototype classification
Edited nearest neighbor classification
k+k-nearest neighbor
2018/11/17

6. Demos by Cleve Cleve’s Demos of Delaunay triangles and Voronoi diagram
books/dcpr/example/cleve/vshow.m
2018/11/17

7. 1NNC Decision Boundaries
2018/11/17

8. 1NNC Distance as Surfaces and Contours
2018/11/17

9. Using Prototypes in KNNC
No. of prototypes for each class is 4.
2018/11/17

10. Decision Boundaries of Different Classifiers
Quadratic classifier 1KNNC classifeir
Naive Bayes classifier
2018/11/17