Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Statistical Decision Making.

Slides:

Advertisements

Similar presentations

ECE 8443 – Pattern Recognition LECTURE 05: MAXIMUM LIKELIHOOD ESTIMATION Objectives: Discrete Features Maximum Likelihood Resources: D.H.S: Chapter 3 (Part.

Advertisements

Data Mining Anomaly Detection Lecture Notes for Chapter 10 Introduction to Data Mining by Minqi Zhou © Tan,Steinbach, Kumar Introduction to Data Mining.

LECTURE 11: BAYESIAN PARAMETER ESTIMATION

What is Statistical Modeling

Assuming normally distributed data! Naïve Bayes Classifier.

Lecture 20 Object recognition I

Parameter Estimation: Maximum Likelihood Estimation Chapter 3 (Duda et al.) – Sections CS479/679 Pattern Recognition Dr. George Bebis.

Anomaly Detection. Anomaly/Outlier Detection  What are anomalies/outliers? The set of data points that are considerably different than the remainder.

1 lBayesian Estimation (BE) l Bayesian Parameter Estimation: Gaussian Case l Bayesian Parameter Estimation: General Estimation l Problems of Dimensionality.

Pattern Recognition. Introduction. Definitions.. Recognition process. Recognition process relates input signal to the stored concepts about the object.

Thanks to Nir Friedman, HU

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Introduction to machine learning

Jeff Howbert Introduction to Machine Learning Winter Classification Bayesian Classifiers.

Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C.

ECSE 6610 Pattern Recognition Professor Qiang Ji Spring, 2011.

1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 13 Oct 14, 2005 Nanjing University of Science & Technology.

ECE 8443 – Pattern Recognition LECTURE 06: MAXIMUM LIKELIHOOD AND BAYESIAN ESTIMATION Objectives: Bias in ML Estimates Bayesian Estimation Example Resources:

Bayesian Networks. Male brain wiring Female brain wiring.

Digital Camera and Computer Vision Laboratory Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan, R.O.C.

Speech Recognition Pattern Classification. 22 September 2015Veton Këpuska2 Pattern Classification  Introduction  Parametric classifiers  Semi-parametric.

CHAPTER 4: Parametric Methods. Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2 Parametric Estimation Given.

Image Classification 영상분류

ECE 8443 – Pattern Recognition LECTURE 07: MAXIMUM LIKELIHOOD AND BAYESIAN ESTIMATION Objectives: Class-Conditional Density The Multivariate Case General.

Computational Intelligence: Methods and Applications Lecture 12 Bayesian decisions: foundation of learning Włodzisław Duch Dept. of Informatics, UMK Google:

Classification Heejune Ahn SeoulTech Last updated May. 03.

1 A Bayesian statistical method for particle identification in shower counters IX International Workshop on Advanced Computing and Analysis Techniques.

Sample variance and sample error We learned recently how to determine the sample variance using the sample mean. How do we translate this to an unbiased.

Bayesian Classification. Bayesian Classification: Why? A statistical classifier: performs probabilistic prediction, i.e., predicts class membership probabilities.

1 E. Fatemizadeh Statistical Pattern Recognition.

MACHINE LEARNING 8. Clustering. Motivation Based on E ALPAYDIN 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2  Classification problem:

: Chapter 3: Maximum-Likelihood and Baysian Parameter Estimation 1 Montri Karnjanadecha ac.th/~montri.

Bayesian Classification Using P-tree  Classification –Classification is a process of predicting an – unknown attribute-value in a relation –Given a relation,

Chapter 3: Maximum-Likelihood Parameter Estimation l Introduction l Maximum-Likelihood Estimation l Multivariate Case: unknown , known  l Univariate.

1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 8 Sept 23, 2005 Nanjing University of Science & Technology.

Bayesian Classification

Bayesian Decision Theory Basic Concepts Discriminant Functions The Normal Density ROC Curves.

1 Unsupervised Learning and Clustering Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of.

Chapter 20 Classification and Estimation Classification – Feature selection Good feature have four characteristics: –Discrimination. Features.

METU Informatics Institute Min720 Pattern Classification with Bio-Medical Applications Part 9: Review.

Chapter 6. Classification and Prediction Classification by decision tree induction Bayesian classification Rule-based classification Classification by.

Basic Technical Concepts in Machine Learning Introduction Supervised learning Problems in supervised learning Bayesian decision theory.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Classification COMP Seminar BCB 713 Module Spring 2011.

BAYESIAN LEARNING. 2 Bayesian Classifiers Bayesian classifiers are statistical classifiers, and are based on Bayes theorem They can calculate the probability.

Computational Intelligence: Methods and Applications Lecture 26 Density estimation, Expectation Maximization. Włodzisław Duch Dept. of Informatics, UMK.

Ch 1. Introduction Pattern Recognition and Machine Learning, C. M. Bishop, Updated by J.-H. Eom (2 nd round revision) Summarized by K.-I.

Artificial Intelligence and Authorship: When Computers Learn to Read Kristin Betancourt COSC 480.

Bayesian Classification 1. 2 Bayesian Classification: Why? A statistical classifier: performs probabilistic prediction, i.e., predicts class membership.

Basic Technical Concepts in Machine Learning

CS479/679 Pattern Recognition Dr. George Bebis

Probability theory retro

Special Topics In Scientific Computing

Chapter 3: Maximum-Likelihood and Bayesian Parameter Estimation (part 2)

Outline Parameter estimation – continued Non-parametric methods.

REMOTE SENSING Multispectral Image Classification

REMOTE SENSING Multispectral Image Classification

POINT ESTIMATOR OF PARAMETERS

Statistical NLP: Lecture 4

Parametric Estimation

LECTURE 07: BAYESIAN ESTIMATION

CS 594: Empirical Methods in HCC Introduction to Bayesian Analysis

Hairong Qi, Gonzalez Family Professor

Chapter 3: Maximum-Likelihood and Bayesian Parameter Estimation (part 2)

Data Mining Anomaly Detection

Biological Background

Bayesian Decision Theory

Pattern Recognition ->Machine Learning- >Data Analytics Supervised Learning Unsupervised Learning Semi-supervised Learning Reinforcement Learning.

ECE – Pattern Recognition Lecture 4 – Parametric Estimation

Hairong Qi, Gonzalez Family Professor

Presentation transcript:

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Statistical Decision Making Supervised Learning: Using a training set to design classifier – Using a separate test set for accuracy Unsupervised Learning: clustering Parametric decision making: probability density function is known for each class, not the parameters (mean, variance) – must be estimated. Pattern Recognition1

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recogntion2

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion3

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion4

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion5

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion6

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Bayesian decision making refers to choosing the most likely class, given the value of the feature(s) P(x/C) is the conditional probability of obtaining feature x given that the sample is from class C P(C/x) = P(C) P(x/C) P(x) Example: What is the probability that a person has a cold (C) given that he or she has a fever (f) P(C) =0.01, P(f)=0.02, P(f/C)=0.04 P(C/f) = P(C) P(f/C) = (0.01)(0.4) = 0.2 P(f) 0.02 Pattern Recgntion7

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion8

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion9

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion10

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion11

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion12

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion13

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion14

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recogntion15

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Likelihood Ratio between class C i and C i R = P(C i /x) = P(C i ) P(x/C i ) P(C j /x) P(C j ) P(x/C j ) Likelihood Ratio between class A and B R = P(A /x) = P(A) P(x/A) P(B/x) P(B) P(x/B) If R>1 – select class A If R<1 – select class B Pattern Recgntion16

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Example: Detecting the HIV virus using the ELISA test H – patient has HIV virus H’ – patient does not have HIV virus Pos – patient tests positive Neg – patient tests negative Let: P(H)=0.15  P(H’)=0.85 P(Pos/H) = 0.95 and P(Pos/H’)=0.02 Pattern Recgntion17

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Bayes’ Theorem P(H/Pos) = P(H) P(Pos/H), P(H) P(Pos/H)+P(H’) P(Pos/H’) = (0.15)(0.95) = 0,893 (0.15)(0.95) + (0.85)(0.02) P(H/Pos)>0.5 Likelihood Ratio R = P(H) P(Pos/H) = (0.15)(0.95) = P(H’) P(Pos/H’) (0.85)(0.02) R>1 Pattern Recgntion18

Technological Educational Institute Of Crete Department Of Applied Informatics and Multimedia Intelligent Systems Laboratory Pattern Recgntion19