Nearest-Neighbor Classifiers

Slides:

Advertisements

Similar presentations

Christoph F. Eick Questions and Topics Review Nov. 30, Give an example of a problem that might benefit from feature creation 2.How does DENCLUE.

Advertisements

DECISION TREES. Decision trees  One possible representation for hypotheses.

Lecture 3-4: Classification & Clustering

Machine Learning Instance Based Learning & Case Based Reasoning Exercise Solutions.

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

K-NEAREST NEIGHBORS AND DECISION TREE Nonparametric Supervised Learning.

Data Mining Classification: Alternative Techniques

Data Mining Classification: Alternative Techniques

K-means method for Signal Compression: Vector Quantization

Lazy vs. Eager Learning Lazy vs. eager learning

Classification and Decision Boundaries

Navneet Goyal. Instance Based Learning  Rote Classifier  K- nearest neighbors (K-NN)  Case Based Resoning (CBR)

MACHINE LEARNING 9. Nonparametric Methods. Introduction Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2 

1 Classification: Definition Given a collection of records (training set ) Each record contains a set of attributes, one of the attributes is the class.

Flexible Metric NN Classification based on Friedman (1995) David Madigan.

Data Mining Classification: Alternative Techniques

These slides are based on Tom Mitchell’s book “Machine Learning” Lazy learning vs. eager learning Processing is delayed until a new instance must be classified.

1 Nearest Neighbor Learning Greg Grudic (Notes borrowed from Thomas G. Dietterich and Tom Mitchell) Intro AI.

An Intelligent & Incremental Approach to kNN using R-trees DJ Oneil & Esten Rye (G01)

CES 514 – Data Mining Lec 9 April 14 Mid-term k nearest neighbor.

Aprendizagem baseada em instâncias (K vizinhos mais próximos)

KNN, LVQ, SOM. Instance Based Learning K-Nearest Neighbor Algorithm (LVQ) Learning Vector Quantization (SOM) Self Organizing Maps.

Nearest Neighbour Condensing and Editing David Claus February 27, 2004 Computer Vision Reading Group Oxford.

CS Instance Based Learning1 Instance Based Learning.

K Nearest Neighborhood (KNNs)

DATA MINING LECTURE 10 Classification k-nearest neighbor classifier Naïve Bayes Logistic Regression Support Vector Machines.

COMMON EVALUATION FINAL PROJECT Vira Oleksyuk ECE 8110: Introduction to machine Learning and Pattern Recognition.

1 Data Mining Lecture 5: KNN and Bayes Classifiers.

Nearest Neighbor (NN) Rule & k-Nearest Neighbor (k-NN) Rule Non-parametric : Can be used with arbitrary distributions, No need to assume that the form.

Overview of Supervised Learning Overview of Supervised Learning2 Outline Linear Regression and Nearest Neighbors method Statistical Decision.

Pattern Recognition April 19, 2007 Suggested Reading: Horn Chapter 14.

NEAREST NEIGHBORS ALGORITHM Lecturer: Yishay Mansour Presentation: Adi Haviv and Guy Lev 1.

David Claus and Christoph F. Eick: Nearest Neighbor Editing and Condensing Techniques Nearest Neighbor Editing and Condensing Techniques 1.Nearest Neighbor.

METU Informatics Institute Min720 Pattern Classification with Bio-Medical Applications Part 6: Nearest and k-nearest Neighbor Classification.

KNN & Naïve Bayes Hongning Wang Today’s lecture Instance-based classifiers – k nearest neighbors – Non-parametric learning algorithm Model-based.

Chapter 13 (Prototype Methods and Nearest-Neighbors )

Vector Quantization CAP5015 Fall 2005.

DATA MINING LECTURE 10b Classification k-nearest neighbor classifier

Overview Data Mining - classification and clustering

CS Machine Learning Instance Based Learning (Adapted from various sources)

Eick: kNN kNN: A Non-parametric Classification and Prediction Technique Goals of this set of transparencies: 1.Introduce kNN---a popular non-parameric.

Debrup Chakraborty Non Parametric Methods Pattern Recognition and Machine Learning.

KNN & Naïve Bayes Hongning Wang

Nonparametric Density Estimation – k-nearest neighbor (kNN) 02/20/17

Non-Parameter Estimation

MIRA, SVM, k-NN Lirong Xia. MIRA, SVM, k-NN Lirong Xia.

Instance Based Learning

Ch8: Nonparametric Methods

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Lecture 05: K-nearest neighbors

Classification Nearest Neighbor

Instance Based Learning (Adapted from various sources)

K Nearest Neighbor Classification

The point is class B via 3NNC.

Classification Nearest Neighbor

Prepared by: Mahmoud Rafeek Al-Farra

Instance Based Learning

COSC 4335: Other Classification Techniques

DATA MINING LECTURE 10 Classification k-nearest neighbor classifier

Chap 8. Instance Based Learning

Lecture 7: Simple Classifier (KNN)

Nearest Neighbors CSC 576: Data Mining.

Lecture 03: K-nearest neighbors

Data Mining Classification: Alternative Techniques

Nearest Neighbor Classifiers

CSE4334/5334 Data Mining Lecture 7: Classification (4)

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

MIRA, SVM, k-NN Lirong Xia. MIRA, SVM, k-NN Lirong Xia.

ECE – Pattern Recognition Lecture 10 – Nonparametric Density Estimation – k-nearest-neighbor (kNN) Hairong Qi, Gonzalez Family Professor Electrical.

Data Mining CSCI 307, Spring 2019 Lecture 11

Presentation transcript:

Nearest-Neighbor Classifiers Requires three things The set of stored records Distance Metric to compute distance between records The value of k, the number of nearest neighbors to retrieve To classify an unknown record: Compute distance to other training records Identify k nearest neighbors Use class labels of nearest neighbors to determine the class label of unknown record (e.g., by taking majority vote)

Definition of Nearest Neighbor K-nearest neighbors of a record x are data points that have the k smallest distance to x

Voronoi Diagrams for NN-Classifiers Each cell contains one sample, and every location within the cell is closer to that sample than to any other sample. A Voronoi diagram divides the space into such cells. Every query point will be assigned the classification of the sample within that cell. The decision boundary separates the class regions based on the 1-NN decision rule. Knowledge of this boundary is sufficient to classify new points. Remarks: Voronoi diagrams can be computed in lower dimensional spaces; in feasible for higher dimensional spaced. They also represent models for clusters that have been generate by representative-based clustering algorithms.

K-NN:More Complex Decision Boundaries

What is interesting about kNN? No real model the “data is the model” Parametric approaches: Learn model from data Non-parametric approaches: Data is the model Lazy Capable to create quite convex decision boundaries Having a good distance function is important.