K nearest neighbors algorithm Parallelization on Cuda PROF. VELJKO MILUTINOVIĆ MAŠA KNEŽEVIĆ 3037/2015.

Slides:

Advertisements

Similar presentations

1. Find the cost of each of the following using the Nearest Neighbor Algorithm. a)Start at Vertex M.

Advertisements

DECISION TREES. Decision trees  One possible representation for hypotheses.

Naïve-Bayes Classifiers Business Intelligence for Managers.

Data Mining Classification: Alternative Techniques

K-means method for Signal Compression: Vector Quantization

1 CS 391L: Machine Learning: Instance Based Learning Raymond J. Mooney University of Texas at Austin.

Indian Statistical Institute Kolkata

1 Machine Learning: Lecture 7 Instance-Based Learning (IBL) (Based on Chapter 8 of Mitchell T.., Machine Learning, 1997)

Lazy vs. Eager Learning Lazy vs. eager learning

Lecture 6 K-Nearest Neighbor Classifier

Navneet Goyal. Instance Based Learning  Rote Classifier  K- nearest neighbors (K-NN)  Case Based Resoning (CBR)

Nearest Neighbor. Predicting Bankruptcy Nearest Neighbor Remember all your data When someone asks a question –Find the nearest old data point –Return.

K nearest neighbor and Rocchio algorithm

1 Chapter 10 Introduction to Machine Learning. 2 Chapter 10 Contents (1) l Training l Rote Learning l Concept Learning l Hypotheses l General to Specific.

CS292 Computational Vision and Language Pattern Recognition and Classification.

Learning from Observations Chapter 18 Section 1 – 4.

CS 590M Fall 2001: Security Issues in Data Mining Lecture 3: Classification.

Carla P. Gomes CS4700 CS 4700: Foundations of Artificial Intelligence Carla P. Gomes Module: Nearest Neighbor Models (Reading: Chapter.

Classification Dr Eamonn Keogh Computer Science & Engineering Department University of California - Riverside Riverside,CA Who.

Instance Based Learning. Nearest Neighbor Remember all your data When someone asks a question –Find the nearest old data point –Return the answer associated.

These slides are based on Tom Mitchell’s book “Machine Learning” Lazy learning vs. eager learning Processing is delayed until a new instance must be classified.

1 Nearest Neighbor Learning Greg Grudic (Notes borrowed from Thomas G. Dietterich and Tom Mitchell) Intro AI.

Aprendizagem baseada em instâncias (K vizinhos mais próximos)

KNN, LVQ, SOM. Instance Based Learning K-Nearest Neighbor Algorithm (LVQ) Learning Vector Quantization (SOM) Self Organizing Maps.

Steep learning curves Reading: DH&S, Ch 4.6, 4.5.

INSTANCE-BASE LEARNING

Instance-Based Learners So far, the learning methods that we have seen all create a model based on a given representation and a training set. Once the.

CS Instance Based Learning1 Instance Based Learning.

Nearest-Neighbor Classifiers Sec minutes of math... Definition: a metric function is a function that obeys the following properties: Identity:

Quiz Th. March Propositional Logic Learning.

Module 04: Algorithms Topic 07: Instance-Based Learning

Supervised Learning and k Nearest Neighbors Business Intelligence for Managers.

APPLICATIONS OF DATA MINING IN INFORMATION RETRIEVAL.

K Nearest Neighborhood (KNNs)

K Nearest Neighbors Saed Sayad 1www.ismartsoft.com.

 2003, G.Tecuci, Learning Agents Laboratory 1 Learning Agents Laboratory Computer Science Department George Mason University Prof. Gheorghe Tecuci 9 Instance-Based.

Visual Information Systems Recognition and Classification.

NEAREST NEIGHBORS ALGORITHM Lecturer: Yishay Mansour Presentation: Adi Haviv and Guy Lev 1.

Data Mining Practical Machine Learning Tools and Techniques Chapter 4: Algorithms: The Basic Methods Section 4.7: Instance-Based Learning Rodney Nielsen.

Project by: Cirill Aizenberg, Dima Altshuler Supervisor: Erez Berkovich.

1 Chapter 10 Introduction to Machine Learning. 2 Chapter 10 Contents (1) l Training l Rote Learning l Concept Learning l Hypotheses l General to Specific.

KNN & Naïve Bayes Hongning Wang Today’s lecture Instance-based classifiers – k nearest neighbors – Non-parametric learning algorithm Model-based.

Fast Query-Optimized Kernel Machine Classification Via Incremental Approximate Nearest Support Vectors by Dennis DeCoste and Dominic Mazzoni International.

Outline K-Nearest Neighbor algorithm Fuzzy Set theory Classifier Accuracy Measures.

Lazy Learners K-Nearest Neighbor algorithm Fuzzy Set theory Classifier Accuracy Measures.

Machine Learning ICS 178 Instructor: Max Welling Supervised Learning.

An Improved Algorithm for Decision-Tree-Based SVM Sindhu Kuchipudi INSTRUCTOR Dr.DONGCHUL KIM.

CS Machine Learning Instance Based Learning (Adapted from various sources)

K-Nearest Neighbor Learning.

Eick: kNN kNN: A Non-parametric Classification and Prediction Technique Goals of this set of transparencies: 1.Introduce kNN---a popular non-parameric.

Debrup Chakraborty Non Parametric Methods Pattern Recognition and Machine Learning.

Kansas State University Department of Computing and Information Sciences CIS 890: Special Topics in Intelligent Systems Wednesday, November 15, 2000 Cecil.

SLIQ (SUPERVISED LEARNING IN QUEST) STUDENT: NIKOLA TERZIĆ PROFESOR: VELJKO MILUTINOVIĆ.

Instance-Based Learning Evgueni Smirnov. Overview Instance-Based Learning Comparison of Eager and Instance-Based Learning Instance Distances for Instance-Based.

1 Text Categorization  Assigning documents to a fixed set of categories  Applications:  Web pages  Recommending pages  Yahoo-like classification hierarchies.

KNN & Naïve Bayes Hongning Wang

General-Purpose Learning Machine

Hierarchical Clustering: Time and Space requirements

Ananya Das Christman CS311 Fall 2016

Instance Based Learning (Adapted from various sources)

Machine Learning Week 1.

K Nearest Neighbor Classification

Nearest-Neighbor Classifiers

Prepared by: Mahmoud Rafeek Al-Farra

یادگیری بر پایه نمونه Instance Based Learning Instructor : Saeed Shiry

Instance Based Learning

Chap 8. Instance Based Learning

Lecture 7: Simple Classifier (KNN)

Machine Learning: UNIT-4 CHAPTER-1

Nearest Neighbors CSC 576: Data Mining.

Presentation transcript:

K nearest neighbors algorithm Parallelization on Cuda PROF. VELJKO MILUTINOVIĆ MAŠA KNEŽEVIĆ 3037/2015

K nearest neighbors algorithm Classification algorithm. Supervised learning. Instance-based learning – method for classifying objects based on closest training examples in the feature space. 2/12

Simple KNN Algorithm Whenever we have a new point to classify, we find its K nearest neighbors from the training data. The distance is calculated using the Euclidean Distance. 3/12

Simple KNN Algorithm Add each training example to the list of training_examples. Given a query instance x q to be classified, Let x 1, x 2, …, x k denote the k instances from training_examples that are nearest to x q. Return the class that represents the maximum of the k instances. 4/12

KNN Example 5/12 If K = 5, the query instance x q will be classified as negative since the majority of its neighbors are classified as negative.

KNN implementation 6/12 The user sets a value for K which is supposed to be odd. Two thirds of the data set is used for training, one third for testing. The training examples are put into a list.

KNN implementation 7/12 Testing the algorithm consists of comparing one test sample to all of the training examples. The Euclidean distance is found for all the training examples. The list is then sorted, the first K are taken into consideration when searching for the class of the test example.

Parallelization on GPU 8/12

KNN on GPU 9/12 Calculation of the Euclidean distance between the query instance x q that needs to be classified and a training example. Sorting algorithm to sort the training set in order to easily find the first K nearest neighbors.

Example Task 10/12 Determine if a patient has diabetes using the attributes given in the data set, such as: triceps skin fold thickness, body mass index, diabetes pedigree function, age … Public data set – Pima Indian Diabetes Data Set ( indians-diabetes/) – scaled appropriately for the testing purposes. indians-diabetes/

Results 11/12

Example Task 12/12 Original data set: Bigger data set: