Pattern Recognition Vidya Manian Dept. of Electrical and Computer Engineering University of Puerto Rico INEL 5046, Spring 2007

Slides:



Advertisements
Similar presentations
Applications of one-class classification
Advertisements

Machine Learning Neural Networks
Chapter 1: Introduction to Pattern Recognition
Simple Neural Nets For Pattern Classification
CS292 Computational Vision and Language Pattern Recognition and Classification.
RBF Neural Networks x x1 Examples inside circles 1 and 2 are of class +, examples outside both circles are of class – What NN does.
OUTLINE Course description, What is pattern recognition, Cost of error, Decision boundaries, The desgin cycle.
Connectionist models. Connectionist Models Motivated by Brain rather than Mind –A large number of very simple processing elements –A large number of weighted.
Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.
Pattern Recognition 9/23/2008 Instructor: Wen-Hung Liao, Ph.D.
Chapter 2: Pattern Recognition
Machine Learning Motivation for machine learning How to set up a problem How to design a learner Introduce one class of learners (ANN) –Perceptrons –Feed-forward.
Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.
Neural Networks. Background - Neural Networks can be : Biological - Biological models Artificial - Artificial models - Desire to produce artificial systems.
1. Introduction to Pattern Recognition and Machine Learning. Prof. A.L. Yuille. Dept. Statistics. UCLA. Stat 231. Fall 2004.
Radial-Basis Function Networks
METU Informatics Institute Min720 Pattern Classification with Bio-Medical Applications Lecture Notes by Neşe Yalabık Spring 2011.
Introduction to Pattern Recognition
嵌入式視覺 Pattern Recognition for Embedded Vision Template matching Statistical / Structural Pattern Recognition Neural networks.
Introduction Mohammad Beigi Department of Biomedical Engineering Isfahan University
Introduction to Pattern Recognition Chapter 1 (Duda et al.) CS479/679 Pattern Recognition Dr. George Bebis 1.
: Chapter 1: Introduction 1 Montri Karnjanadecha ac.th/~montri Principles of Pattern Recognition.
MSE 2400 EaLiCaRA Spring 2015 Dr. Tom Way
Artificial Neural Nets and AI Connectionism Sub symbolic reasoning.
Teuvo Kohonen Dr. Eng., Emeritus Professor of the Academy of Finland; Academician Since the 1960s, Professor Kohonen has introduced several new concepts.
DIGITAL IMAGE PROCESSING Dr J. Shanbehzadeh M. Hosseinajad ( J.Shanbehzadeh M. Hosseinajad)
Explorations in Neural Networks Tianhui Cai Period 3.
ARTIFICIAL NEURAL NETWORKS. Introduction to Neural Networks.
Self organizing maps 1 iCSC2014, Juan López González, University of Oviedo Self organizing maps A visualization technique with data dimension reduction.
Computational Intelligence II Lecturer: Professor Pekka Toivanen Exercises: Nina Rogelj
Machine Learning Dr. Shazzad Hosain Department of EECS North South Universtiy
Compiled By: Raj G Tiwari.  A pattern is an object, process or event that can be given a name.  A pattern class (or category) is a set of patterns sharing.
1 Machine Learning The Perceptron. 2 Heuristic Search Knowledge Based Systems (KBS) Genetic Algorithms (GAs)
NEURAL NETWORKS FOR DATA MINING
LINEAR CLASSIFICATION. Biological inspirations  Some numbers…  The human brain contains about 10 billion nerve cells ( neurons )  Each neuron is connected.
Lecture 10: 8/6/1435 Machine Learning Lecturer/ Kawther Abas 363CS – Artificial Intelligence.
So Far……  Clustering basics, necessity for clustering, Usage in various fields : engineering and industrial fields  Properties : hierarchical, flat,
1 Pattern Recognition Pattern recognition is: 1. A research area in which patterns in data are found, recognized, discovered, …whatever. 2. A catchall.
Visual Information Systems Recognition and Classification.
METU Informatics Institute Min720 Pattern Classification with Bio-Medical Applications Part 8: Neural Networks.
School of Engineering and Computer Science Victoria University of Wellington Copyright: Peter Andreae, VUW Image Recognition COMP # 18.
Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning.
Neural Networks Presented by M. Abbasi Course lecturer: Dr.Tohidkhah.
Neural Networks Teacher: Elena Marchiori R4.47 Assistant: Kees Jong S2.22
November 21, 2013Computer Vision Lecture 14: Object Recognition II 1 Statistical Pattern Recognition The formal description consists of relevant numerical.
Face Detection Using Neural Network By Kamaljeet Verma ( ) Akshay Ukey ( )
Each neuron has a threshold value Each neuron has weighted inputs from other neurons The input signals form a weighted sum If the activation level exceeds.
Artificial Intelligence CIS 342 The College of Saint Rose David Goldschmidt, Ph.D.
Neural Networks. Background - Neural Networks can be : Biological - Biological models Artificial - Artificial models - Desire to produce artificial systems.
Pattern Recognition NTUEE 高奕豪 2005/4/14. Outline Introduction Definition, Examples, Related Fields, System, and Design Approaches Bayesian, Hidden Markov.
Pattern Recognition. What is Pattern Recognition? Pattern recognition is a sub-topic of machine learning. PR is the science that concerns the description.
Neural networks (2) Reminder Avoiding overfitting Deep neural network Brief summary of supervised learning methods.
Machine Learning Artificial Neural Networks MPλ ∀ Stergiou Theodoros 1.
Chapter 13 Artificial Intelligence. Artificial Intelligence – Figure 13.1 The Turing Test.
Pattern Recognition Lecture 20: Neural Networks 3 Dr. Richard Spillman Pacific Lutheran University.
INTRODUCTION TO NEURAL NETWORKS 2 A new sort of computer What are (everyday) computer systems good at... and not so good at? Good at..Not so good at..
Artificial Intelligence
Neural Network Architecture Session 2
Learning in Neural Networks
IMAGE PROCESSING RECOGNITION AND CLASSIFICATION
Pattern Recognition Sergios Theodoridis Konstantinos Koutroumbas
What is an ANN ? The inventor of the first neuro computer, Dr. Robert defines a neural network as,A human brain like system consisting of a large number.
Introduction to Pattern Recognition
Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.
Introduction to Pattern Recognition
Creating Data Representations
Neural Networks.
Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.
Outline Announcement Neural networks Perceptrons - continued
Presentation transcript:

Pattern Recognition Vidya Manian Dept. of Electrical and Computer Engineering University of Puerto Rico INEL 5046, Spring 2007

Human Perception Humans have developed highly sophisticated skills for sensing their environment and taking actions according to what they observe, e.g., –Recognizing a face –Understanding spoken words –Reading handwriting –Distinguishing fresh food from its smell We would like to give similar capabilities to machines

What is a Pattern? “A pattern is the opposite of a chaos; it is an entity vaguely defined, that could be given a name.” (Watanabe)

What is Pattern Recognition? A pattern is an entity, vaguely defined, that could be given a name, e.g., > Fingerprint image>speech signal > handwritten word>DNA sequence >human face>… Pattern recognition is the study of how machines can –Observe the environment –Learn to distinguish patterns of interest –Make sound and reasonable decisions about the categories of the patterns

Recognition Identification of a pattern as a member of a category we already know, or we are familiar with –Classification (known categories) –Clustering (creation of new categories) Category “A” Category “B” Classification Clustering

Pattern Recognition Given an input pattern, make a decision about the “category” or “class” of the pattern Pattern recognition is a very broad subject with many applications In this course we will study a variety of techniques to solve P.R. problems and discuss their relative strengths and weaknesses

Pattern Class A collection of “ similar ” (not necessarily identical) objects A class is defined by class samples (paradigms, exemplars, prototypes) Inter-class variability Intra-class variability

Pattern Class Model Different descriptions, which are typically mathematical in form for each class/population Given a pattern, choose the best-fitting model for it and then assign it to class associated with the model

Intra-class and Inter-class Variability The letter “T” in different typefaces Same face under different expression, pose….

Identical twins Characters that look similar Inter-class Similarity

Pattern Recognition Having been shown a few positive examples (and perhaps a few negative examples) of a pattern class, the system “learns” to tell whether or not a new object belongs in this class (Watanabe) COGNITION= Formation of new classes RECOGNITION= known classes

Pattern Recognition Applications Speech recognition Detection and diagnosis of disease Remote sensing (terrain classification, tanks detection) Character recognition Identification and counting of cells Fingerprint identification Web search Inspection (PC boards, IC masks, textiles)

Fish Classification Preprocessing will involve image enhancement, separating touching/occluding fishes and finding the boundary of the fish

Length Feature Training (design or learning) Samples

Lightness Feature Overlap in the histograms is small compared to length feature

Two-dimensional Feature Space (Representation) Two features together are better than individual features Cost of misclassification?

Complex Decision Boundary Issue of generalization

Boundary With Good Generalization Simplify the decision boundary!

Models for Pattern Recognition Template matching Statistical (geometric) Syntactic (structural) Artificial neural network (biologically motivated?) Hybrid approach

Statistical Pattern Recognition Preprocessing Feature extraction Classification Learning Feature selection Recognition Training pattern Patterns + Class labels Preprocessing

Each pattern is represented as a point in the d-dimensional feature space Features are domain-specific and be invariant to translation, rotation and scale Good representation  small intraclass variation, large interclass separation, simple decision rule No redundant features, too many features and less samples- curse of dimensionality (Huges phenomena) Pattern Representation using features x1x1 x2x2 x1x1 x2x2

Artificial Neural Networks Massive parallelism is essential for complex pattern recognition tasks (e.g., speech and image recognition) –Human take only a few hundred ms for most cognitive tasks; suggests parallel computation Biological networks attempt to achieve good performance via dense interconnection of simple computational elements (neurons) –Number of neurons  – –Number of interconnections/neuron  10 3 – 10 4 –Total number of interconnections  10 14

Artificial Neural Networks Nodes in neural networks are nonlinear, typically analog where is internal threshold or offset x1x1 x2x2 xdxd Y (output) w1w1 wdwd

Feed-forward nets with one or more layers (hidden) between the input and output nodes A three-layer net can generate arbitrary complex decision regions These nets can be trained by back-propagation training algorithm Multilayer Perceptron d inputs First hidden layer NH 1 input units Second hidden layer NH 2 input units c outputs

Statistical Pattern Recognition Patterns represented in a feature space Statistical model for pattern generation in feature space Given training patterns from each class, goal is to partition the feature space.

Image Analysis and Segmentation (classification) using texture features Aerial photograph of Anasco,PR Classified using Logical operators

Classification of color images using texture features Texture mosaic of 3 colored tiles and canvas texture. Classified image

Classification in Remote Sensing

Sensor

Classification of Landsat image of San Juan area, PR using Gabor texture features Landsat image 7 bands (R, G, B, IR and Thermal) Classified image using R,G,B