Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning

THE NAÏVE BAYES CLASSIFIER In the naïve Bayes classification scheme, the required estimate of the pdf at a point x=[x(1),...,x(l)]T ∈ R l is given as That is, the components of the feature vector x are assumed to be statistically independent.

Machine Learning Example. Generate a set X1 that consists of N1 = 50 5-dimensional data vectors that stem from two classes, ω1 and ω2. The classes are modeled by Gaussian distributions with means m1 = [0,0,0,0,0]T and m2 = [1,1,1,1,1]T and respective covariance matrices

Machine Learning Step 1. Classify the points of the test set X2 using the naive Bayes classifier, where for a given x, p(x|ωi ) is estimated as

Machine Learning Step 2. Compute the ML estimates of m1, m2, S1, and S2 using X1. Employ the ML estimates in the Bayesian classifier

Machine Learning Step 3. Compare the results obtained in steps 1 and 2. The resulting classification errors—naive_error and Bayes_ML_error—are and , respectively. In other words, the naive classification scheme outperforms the standard ML-based scheme.

Machine Learning The techniques that are built around the optimal Bayesian classifier rely on the estimation of the pdf functions describing the data distribution in each class.

Machine Learning

The focus is on the direct design of a discriminant function/decision surface that separates the classes in some optimal sense according to an adopted criterion.

Machine Learning Machine learning involves adaptive mechanisms that enable computers to learn from experience, learn by example and learn by analogy. Learning capabilities can improve the performance of an intelligent system over time. The most popular approaches to machine learning are artificial neural networks and genetic algorithms. This lecture is dedicated to neural networks.

Cell structures –Cell body –Dendrites –Axon –Synaptic terminals Machine Learning

Networks of processing units (neurons) with connections (synapses) between them Large number of neurons: Large connectitivity: 10 5 Parallel processing Distributed computation/memory Robust to noise, failures Machine Learning

Understanding the Brain Levels of analysis (Marr, 1982) 1.Computational theory 2.Representation and algorithm 3.Hardware implementation Reverse engineering: From hardware to theory Parallel processing: SIMD vs MIMD Machine Learning

Real Neural Learning Synapses change size and strength with experience. Hebbian learning: When two connected neurons are firing at the same time, the strength of the synapse between them increases. “Neurons that fire together, wire together.” Machine Learning

Neural Network Learning Learning approach based on modeling adaptation in biological neural systems. Perceptron: Initial algorithm for learning simple neural networks (single layer) developed in the 1950’s. Backpropagation: More complex algorithm for learning multi-layer neural networks developed in the 1980’s. Machine Learning

Perceptron Learning Algorithm First neural network learning model in the 1960’s Simple and limited (single layer models) Basic concepts are similar for multi-layer models so this is a good learning tool Still used in many current applications Machine Learning

Perceptron Node – Threshold Logic Unit x1x1 xnxn x2x2 w1w1 w2w2 wnwn z Machine Learning

x1x1 xnxn x2x2 w1w1 w2w2 wnwn z Learn weights such that an objective function is maximized. What objective function should we use? What learning algorithm should we use? Machine Learning

Perceptron Learning Algorithm x1x1 x2x2 z x1x1 x2x2 t Machine Learning

First Training Instance.8.3 z net =.8*.4 +.3*-.2 =.26 =1 x1x1 x2x2 t Machine Learning

Second Training Instance.4.1 z x1x1 x2x2 t net =.4*.4 +.1*-.2 =.14 =1 Machine Learning

Thank you!