TINONS1 Nonlinear SP and Pattern recognition

Slides:

Advertisements

Similar presentations

1 Image Classification MSc Image Processing Assignment March 2003.

Advertisements

1 Welcome to the Kernel-Class My name: Max (Welling) Book: There will be class-notes/slides. Homework: reading material, some exercises, some MATLAB implementations.

An Introduction of Support Vector Machine

Machine learning continued Image source:

An Overview of Machine Learning

Data Visualization STAT 890, STAT 442, CM 462

As applied to face recognition.  Detection vs. Recognition.

Supervised and Unsupervised learning and application to Neuroscience Cours CA6b-4.

Chapter 1: Introduction to Pattern Recognition

x – independent variable (input)

OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Introduction to Neural Networks Simon Durrant Quantitative Methods December 15th.

Review Rong Jin. Comparison of Different Classification Models  The goal of all classifiers Predicating class label y for an input x Estimate p(y|x)

Part I: Classification and Bayesian Learning

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Introduction to machine learning

Radial-Basis Function Networks

Crash Course on Machine Learning

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

JM - 1 Introduction to Bioinformatics: Lecture VIII Classification and Supervised Learning Jarek Meller Jarek Meller Division.

CEN 592 PATTERN RECOGNITION Spring Term CEN 592 PATTERN RECOGNITION Spring Term DEPARTMENT of INFORMATION TECHNOLOGIES Assoc. Prof.

Introduction to Pattern Recognition

嵌入式視覺 Pattern Recognition for Embedded Vision Template matching Statistical / Structural Pattern Recognition Neural networks.

Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University

Pattern Recognition Vidya Manian Dept. of Electrical and Computer Engineering University of Puerto Rico INEL 5046, Spring 2007

Introduction to Pattern Recognition Chapter 1 (Duda et al.) CS479/679 Pattern Recognition Dr. George Bebis 1.

: Chapter 1: Introduction 1 Montri Karnjanadecha ac.th/~montri Principles of Pattern Recognition.

Outline Separating Hyperplanes – Separable Case

This week: overview on pattern recognition (related to machine learning)

ECSE 6610 Pattern Recognition Professor Qiang Ji Spring, 2011.

Data Mining Joyeeta Dutta-Moscato July 10, Wherever we have large amounts of data, we have the need for building systems capable of learning information.

Chapter 4 CONCEPTS OF LEARNING, CLASSIFICATION AND REGRESSION Cios / Pedrycz / Swiniarski / Kurgan.

Pattern Recognition & Machine Learning Debrup Chakraborty

Speech Recognition Pattern Classification. 22 September 2015Veton Këpuska2 Pattern Classification  Introduction  Parametric classifiers  Semi-parametric.

Classification. An Example (from Pattern Classification by Duda & Hart & Stork – Second Edition, 2001)

Perception Introduction Pattern Recognition Image Formation

General Tensor Discriminant Analysis and Gabor Features for Gait Recognition by D. Tao, X. Li, and J. Maybank, TPAMI 2007 Presented by Iulian Pruteanu.

COMMON EVALUATION FINAL PROJECT Vira Oleksyuk ECE 8110: Introduction to machine Learning and Pattern Recognition.

Compiled By: Raj G Tiwari.  A pattern is an object, process or event that can be given a name.  A pattern class (or category) is a set of patterns sharing.

Kernel Methods A B M Shawkat Ali 1 2 Data Mining ¤ DM or KDD (Knowledge Discovery in Databases) Extracting previously unknown, valid, and actionable.

Overview of Supervised Learning Overview of Supervised Learning2 Outline Linear Regression and Nearest Neighbors method Statistical Decision.

Jun-Won Suh Intelligent Electronic Systems Human and Systems Engineering Department of Electrical and Computer Engineering Speaker Verification System.

Classification Heejune Ahn SeoulTech Last updated May. 03.

Ch 4. Linear Models for Classification (1/2) Pattern Recognition and Machine Learning, C. M. Bishop, Summarized and revised by Hee-Woong Lim.

Pattern Recognition April 19, 2007 Suggested Reading: Horn Chapter 14.

Elements of Pattern Recognition CNS/EE Lecture 5 M. Weber P. Perona.

MACHINE LEARNING 7. Dimensionality Reduction. Dimensionality of input Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Learning Kernel Classifiers 1. Introduction Summarized by In-Hee Lee.

Extending linear models by transformation (section 3.4 in text) (lectures 3&4 on amlbook.com)

Pattern recognition – basic concepts. Sample input attribute, attribute, feature, input variable, independent variable (atribut, rys, příznak, vstupní.

Linear Models Tony Dodd. 21 January 2008Mathematics for Data Modelling: Linear Models Overview Linear models. Parameter estimation. Linear in the parameters.

Digital Signal Processing Rahil Mahdian LSV Lab, Saarland University, Germany.

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 1: INTRODUCTION.

Machine Learning Supervised Learning Classification and Regression K-Nearest Neighbor Classification Fisher’s Criteria & Linear Discriminant Analysis Perceptron:

Big data classification using neural network

Data Mining, Machine Learning, Data Analysis, etc. scikit-learn

Machine Learning for Computer Security

ECE 417 Lecture 1: Multimedia Signal Processing

Introduction Machine Learning 14/02/2017.

IMAGE PROCESSING RECOGNITION AND CLASSIFICATION

The Elements of Statistical Learning

Pattern Recognition Sergios Theodoridis Konstantinos Koutroumbas

Machine Learning Ali Ghodsi Department of Statistics

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Introduction to Pattern Recognition

Data Mining, Machine Learning, Data Analysis, etc. scikit-learn

Data Mining, Machine Learning, Data Analysis, etc. scikit-learn

Introduction to Radial Basis Function Networks

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Presentation transcript:

TINONS1 Nonlinear SP and Pattern recognition Wednesday 12.15-14.00, Friday 10.15-12.00 + Friday 8.15-10.00 (without teacher) Literature – Duda, Hart and Stork, Pattern Classification, 2nd edition, 2001 Cases – One extensive case, results in mandatory report Exam – 2/3 oral exam (20 min.) 1/3 report

Examples Speech recognition / Speaker recognition Face/image recognition Signal detection – e.g. in radar images Text modelling – e.g. document similarity Google …

Example : digit recognition - Feature vector/matrix x = [0 0 0 … ; 0 0 120 0 .. ; 0 0 255 ..], that is, often high-dimensional –> problems Classification problem Requires a training and (preferably) test set Generalizability is important Distinguishes between supervised and unsupervised learning (and semi-supervised..)

Typical pattern recognition system Feature vector Probability estimates Preprocessing/Feature extraction Classification/regression Post-processing Signal Decision

Developing a system Creating a training/test dataset Finding good features Choosing the model for classification (or regression) Train/learn the parameters of the model from training set Test/evaluate on the test set Apply suitable postprocessing (e.g. loss function)

Case - Music Genre recognition Input signal – e.g. sound file Features could be rhythm, vocal, etc.. Or simply frequency content in different bands System could output classes such as jazz, pop, rock.. Task is classification, ie. to find function f f : RM → {1, 2, …, number of classes} (R – real numbers, M – dimensions)

Case – Face recognition Input – eg. RGB image Features – eg. Area of face, width, height, color, or simply image matrix as column vector Output : Identity (”Frank vs. Bill”), gender (”boy vs. girl”), obesity score?

Case – Gesture recognition Input – eg. accelerometry/gyro data sequence (eg. from wiimote/android) Features – perhaps just raw data sequence in single movement Output – recognize gesture (supervised/unsupervised) Control system based on gestures

Case – EEG/EKG signals Input – EEG/EKG signals Output EEG: Degree of mental activity (regression problem) EKG: Detection of irregular heart beat pattern

Other.. Speaker recognition / verification Bird sound classification Video tracking (eg. Kinect) Document classification / clustering Kickstarting the MSc. project ...

Introduction – linear regression -> MATLAB DEMO.. intro.m simple case - y : R → R, y(x,w) = w x Learn the parameters w from training set Model selection -> choosing a linear model Should really test on another set..

Linear regression Training – minimizing a cost (or error) function eg. E(w) = ∑n (y(w,xn) - tn)2 tn is ”target” ie. known output for xn E(w*) is a measure that tells how well the model corresponds to the observed data.. w* is here the found minima that minimizes E(w)

Sum-of-squares error

Polynomial regression - 0th order

1th order

3th order

9th order

Over-fitting

Introduction - Linear classifier -> MATLAB DEMO.. intro.m Topics discriminant functions, 1-of-K coding, decision boundary, classification error/accuracy, confusion matrix, outliers .. Limitations linear decision boundary