Machine Learning Motivation for machine learning How to set up a problem How to design a learner Introduce one class of learners (ANN) –Perceptrons –Feed-forward.

Slides:



Advertisements
Similar presentations
Artificial Neural Networks
Advertisements

A Brief Overview of Neural Networks By Rohit Dua, Samuel A. Mulder, Steve E. Watkins, and Donald C. Wunsch.
Slides from: Doug Gray, David Poole
1 Machine Learning: Lecture 4 Artificial Neural Networks (Based on Chapter 4 of Mitchell T.., Machine Learning, 1997)
Tuomas Sandholm Carnegie Mellon University Computer Science Department
Back-propagation Chih-yun Lin 5/16/2015. Agenda Perceptron vs. back-propagation network Network structure Learning rule Why a hidden layer? An example:
Kostas Kontogiannis E&CE
Machine Learning: Connectionist McCulloch-Pitts Neuron Perceptrons Multilayer Networks Support Vector Machines Feedback Networks Hopfield Networks.
Classification Neural Networks 1
Perceptron.
Machine Learning Neural Networks
Overview over different methods – Supervised Learning
Lecture 14 – Neural Networks
INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.
Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.
Artificial Intelligence Statistical learning methods Chapter 20, AIMA (only ANNs & SVMs)
20.5 Nerual Networks Thanks: Professors Frank Hoffmann and Jiawei Han, and Russell and Norvig.
Chapter 6: Multilayer Neural Networks
Artificial Neural Networks
Data Mining with Neural Networks (HK: Chapter 7.5)
Artificial Neural Networks
LOGO Classification III Lecturer: Dr. Bo Yuan
Artificial Neural Network
Machine learning Image source:
Machine learning Image source:
Artificial Neural Networks
Computer Science and Engineering
1 Artificial Neural Networks Sanun Srisuk EECP0720 Expert Systems – Artificial Neural Networks.
Chapter 9 Neural Network.
Neural Networks AI – Week 23 Sub-symbolic AI Multi-Layer Neural Networks Lee McCluskey, room 3/10
CS464 Introduction to Machine Learning1 Artificial N eural N etworks Artificial neural networks (ANNs) provide a general, practical method for learning.
Machine Learning Chapter 4. Artificial Neural Networks
Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.
ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 16: NEURAL NETWORKS Objectives: Feedforward.
Artificial Neural Networks. The Brain How do brains work? How do human brains differ from that of other animals? Can we base models of artificial intelligence.
Artificial Neural Networks An Introduction. What is a Neural Network? A human Brain A porpoise brain The brain in a living creature A computer program.
A note about gradient descent: Consider the function f(x)=(x-x 0 ) 2 Its derivative is: By gradient descent. x0x0 + -
Neural Networks and Machine Learning Applications CSC 563 Prof. Mohamed Batouche Computer Science Department CCIS – King Saud University Riyadh, Saudi.
Linear Discrimination Reading: Chapter 2 of textbook.
Non-Bayes classifiers. Linear discriminants, neural networks.
Linear Classification with Perceptrons
Neural Networks and Backpropagation Sebastian Thrun , Fall 2000.
Neural Network Basics Anns are analytical systems that address problems whose solutions have not been explicitly formulated Structure in which multiple.
Back-Propagation Algorithm AN INTRODUCTION TO LEARNING INTERNAL REPRESENTATIONS BY ERROR PROPAGATION Presented by: Kunal Parmar UHID:
CS621 : Artificial Intelligence
Neural Networks Presented by M. Abbasi Course lecturer: Dr.Tohidkhah.
Artificial Neural Network
EEE502 Pattern Recognition
Learning Neural Networks (NN) Christina Conati UBC
Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons 2. Multi-Layered.
Artificial Neural Networks (ANN). Artificial Neural Networks First proposed in 1940s as an attempt to simulate the human brain’s cognitive learning processes.
1 Perceptron as one Type of Linear Discriminants IntroductionIntroduction Design of Primitive UnitsDesign of Primitive Units PerceptronsPerceptrons.
Chapter 6 Neural Network.
Bab 5 Classification: Alternative Techniques Part 4 Artificial Neural Networks Based Classifer.
Data Mining: Concepts and Techniques1 Prediction Prediction vs. classification Classification predicts categorical class label Prediction predicts continuous-valued.
Learning: Neural Networks Artificial Intelligence CMSC February 3, 2005.
Pattern Recognition Lecture 20: Neural Networks 3 Dr. Richard Spillman Pacific Lutheran University.
Learning with Neural Networks Artificial Intelligence CMSC February 19, 2002.
CSE343/543 Machine Learning Mayank Vatsa Lecture slides are prepared using several teaching resources and no authorship is claimed for any slides.
Learning with Perceptrons and Neural Networks
Learning in Neural Networks
Artificial Neural Networks
Machine Learning Today: Reading: Maria Florina Balcan
Classification Neural Networks 1
Lecture Notes for Chapter 4 Artificial Neural Networks
Seminar on Machine Learning Rada Mihalcea
David Kauchak CS158 – Spring 2019
CS621: Artificial Intelligence Lecture 18: Feedforward network contd
Outline Announcement Neural networks Perceptrons - continued
Presentation transcript:

Machine Learning Motivation for machine learning How to set up a problem How to design a learner Introduce one class of learners (ANN) –Perceptrons –Feed-forward network Back-prop –Other types of networks

Components of a “Well-Posed” Learning Problem Task: the domain of the problem Experience: information about the domain Performance measure: a metric to judge how well the trained system can solve the problem Learner: a computer program whose performance on the task improves (according to the metric) with more experience

Example: Classification Task: Predict whether the user might like a movie or not Experience: database of movies the user has seen and the user’s ratings for them Performance Measure: percent of times the system correctly predicts the user’s preference

Example: Speech Recognition Task: take dictations from the user Experience: a collection of recordings of acoustic utterances with their transcriptions Performance Measure: percent of words correctly identified

Example: Function Modeling Task: approximate an unknown function f(x) Experience: a set of data points: {x i, f(x i )} Performance Measure: average error rate between f(x), the target function, and h(x), the function the system learned, over m test points e.g.

Designing a Learner Training experience –Kind of feedback? –A representative sample? –Learner has control? Target function –Specify expected behavior Function representation –Specify form and parameters Learning algorithm

Artificial Neural Networks Inspired by neurobiology A network is made up of massively interconnect “neurons” Good for some learning problems –Noisy training examples (contain errors) –Target function input can be best described by a vector (e.g., robot sensor data) –Target function is continuous (differentiable)

Perceptron w0w0 w1w1 wnwn 1 x1x1 xnxn O={-1,+1}  O = g(In) = g(xw) = +1 : In >  -1: otherwise n weighted inputs: In = w 0 +x 1 w 1 + x 2 w 2 + … + x n w n = x w An activation function, g(In) …

Training a Perceptron Quantify error –compare output with correct answer Update weights to minimize error  is a constant, the learning rate

How Powerful Are Perceptrons? A perceptron can represent simple Boolean functions –AND, OR, NOT A network of perceptron can represent any Boolean function A perceptron cannot represent XOR –Why?

Linearly Separable Refer to pictures from R&N Fig. 19.9

Gradient Descent Guarantees convergence Approximates non-linearly separable functions Search through the weight space Define error as a continuous function of the weights

Multilayer Network x1x1 x2x2 x n … Input units Hidden units Output units uiui … ujuj w ij OjOj … w ni

Training a Multilayer Network Need to update weights to minimize error, but… –How to assign portions of “blame” to each weights fairly? –In a multilayer network, a weight may (eventually) contribute to multiple outputs –Need to back-propagate the error

Back-Propagation Between a hidden unit and an output unit: Between an input unit and a hidden unit:

Artificial Neural Network Summary Expressiveness: Can approximate any function of a set of attributes Computational efficiency: May take a long time to train to convergence Generalization: generalizes well Sensitivity to noise: very tolerant Transparency: can be used like a black box Prior knowledge: difficult to incorporate