Mehdi Ghayoumi Kent State University Computer Science Department Summer 2015 Exposition on Cyber Infrastructure and Big Data

Machine Learning

Science is a systematic enterprise that builds and organizes Knowledge in the form of testable explanations and predictions about nature and universe. universe

Why “Learn” ? Machine learning is programming computers to optimize a performance criterion using example data or past experience. There is no need to “learn” to calculate payroll Learning is used when: –Human expertise does not exist (navigating on Mars), –Humans are unable to explain their expertise (speech recognition) –Solution changes in time (routing on a computer network) –Solution needs to be adapted to particular cases (user biometrics)

What is Machine Learning? Optimize a performance criterion using example data or past experience. Role of Statistics: Inference from a sample Role of Computer science: Efficient algorithms to –Solve the optimization problem –Representing and evaluating the model for inference

Apply a prediction function to a feature representation of the image to get the desired output: f( ) = “apple” f( ) = “tomato” f( ) = “cow”

y = f(x)

Prediction Training Labels Training Images Training Image Features Learned model

Unsupervised “Weakly” supervised Fully supervised

SVM Neural networks Naïve Bayes Logistic regression Decision Trees K-nearest neighbor RBMs Etc.

Definition of Learning A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

Design a Learning System We shall use handwritten Character recognition as an example to illustrate the design issues and approaches

Step 0: –Lets treat the learning system as a black box Learning System Z

Step 1: Collect Training Examples (Experience). –Without examples, our system will not learn

Step 2: Representing Experience –Choose a representation scheme for the experience/examples The sensor input represented by an n-d vector, called the feature vector, X = (x 1, x 2, x 3, …, x n ) (1,1,0,1,1,1,1,1,1,1,0,0,0,0,1,1,1, 1,1,0, …., 1) (1,1,1,1,1,1,1,1,1,1,0,0,1,1,1,1,1, 1,1,0, …., 1)

Choose a representation scheme for the experience/examples The sensor input represented by an n-d vector, called the feature vector, X = (x1, x2, x3, …, xn) To represent the experience, we need to know what X is. So we need a corresponding vector D, which will record our knowledge (experience) about X The experience E is a pair of vectors E = (X, D)

–Assuming our system is to recognise 10 digits only, then D can be a 10-d binary vector; each correspond to one of the digits if X is digit 5, then d5=1; all others =0 if X is digit 9, then d9=1; all others =0 X = (1,1,0,1,1,1,1,1,1,1,0,0,0,0,1,1,1, 1,1,0, …., 1); D= (0,0,0,0,0,1,0,0,0,0) X= (1,1,1,1,1,1,1,1,1,1,0,0,1,1,1,1,1, 1,1,0, …., 1); D= (0,0,0,0,0,0,0,0,1,0) D = (d0, d1, d2, d3, d4, d5, d6, d7, d8, d9)

Step 3: Choose a Representation for the Black Box –We need to choose a function F to approximate the block box. For a given X, the value of F will give the classification of X. There are considerable flexibilities in choosing F Learning System F F(X) X

–F will be a function of some adjustable parameters, or weights, W = (w1, w2, w3, …w N ), which the learning algorithm can modify or learn Learning System F(W) F(W,X) X

Step 4: Learning/Adjusting the Weights We need a learning algorithm to adjust the weights such that the experience/prior knowledge from the training data can be learned into the system: E=(X,D) F(W,X) = D

Learning System F(W) F(W,X) X D E=(X,D) Error = D-F(W,X) Adjust W

Step 5: Use/Test the System –Once learning is completed, all parameters are fixed. An unknown input X is presented to the system, the system computes its answer according to F(W,X) Learning System F(W) F(W,X) X Answer

SVM Neural networks Naïve Bayes Logistic regression Decision Trees K-nearest neighbor RBMs Etc.

Machine Learning

Bayes Rule Thomas Bayes (c – 7 April 1761) was an English statistician, philosopher and Presbyterian minister, known for having formulated a specific case of the theorem that bears his name: Bayes' theorem. Bayes never published what would eventually become his most famous accomplishment; his notes were edited and published after his death by Richard Price. Machine Learning

Maximum Likelihood (ML) Machine Learning

The Euclidean Distance Classifier

Cell structures –Cell body –Dendrites –Axon –Synaptic terminals Machine Learning

Entropy (disorder, impurity) of a set of examples, S, relative to a binary classification is: where p 1 is the fraction of positive examples in S and p 0 is the fraction of negatives. Machine Learning

An SVM is an abstract learning machine which will learn from a training data set and attempt to generalize and make correct predictions on novel data.

Machine Learning Clustering: Partition unlabeled examples into disjoint subsets of clusters, such that: – Examples within a cluster are similar – Examples in different clusters are different

Machine Learning

Original data Machine Learning

(1) centering & whitening process Machine Learning

(2) FastICA algorithm Machine Learning

(2) FastICA algorithm Machine Learning

Thank you!