Intelligent Environments1 Computer Science and Engineering University of Texas at Arlington.

Slides:



Advertisements
Similar presentations
Slides from: Doug Gray, David Poole
Advertisements

Learning in Neural and Belief Networks - Feed Forward Neural Network 2001 년 3 월 28 일 안순길.
1 Machine Learning: Lecture 4 Artificial Neural Networks (Based on Chapter 4 of Mitchell T.., Machine Learning, 1997)
Ch. Eick: More on Machine Learning & Neural Networks Different Forms of Learning: –Learning agent receives feedback with respect to its actions (e.g. using.
Data Mining Classification: Alternative Techniques
CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.
Smart Home Technologies
Supervised Learning Recap
Tuomas Sandholm Carnegie Mellon University Computer Science Department
Kostas Kontogiannis E&CE
Lecture 13 – Perceptrons Machine Learning March 16, 2010.
Artificial Neural Networks
Machine Learning: Connectionist McCulloch-Pitts Neuron Perceptrons Multilayer Networks Support Vector Machines Feedback Networks Hopfield Networks.
Machine Learning Neural Networks
Instance Based Learning
Lecture 14 – Neural Networks
Neural NetworksNN 11 Neural Networks Teacher: Elena Marchiori R4.47 Assistant: Kees Jong S2.22
CES 514 – Data Mining Lecture 8 classification (contd…)
20.5 Nerual Networks Thanks: Professors Frank Hoffmann and Jiawei Han, and Russell and Norvig.
Artificial Neural Networks Artificial Neural Networks are (among other things) another technique for supervised learning k-Nearest Neighbor Decision Tree.
Data Mining with Decision Trees Lutz Hamel Dept. of Computer Science and Statistics University of Rhode Island.
Machine Learning Neural Networks.
Machine Learning Motivation for machine learning How to set up a problem How to design a learner Introduce one class of learners (ANN) –Perceptrons –Feed-forward.
Artificial Neural Networks
Artificial Neural Networks
CS Instance Based Learning1 Instance Based Learning.
Neural Networks. Background - Neural Networks can be : Biological - Biological models Artificial - Artificial models - Desire to produce artificial systems.
Machine learning Image source:
Machine learning Image source:
Artificial Intelligence Lecture No. 28 Dr. Asad Ali Safi ​ Assistant Professor, Department of Computer Science, COMSATS Institute of Information Technology.
Artificial Neural Networks
Midterm Review Rao Vemuri 16 Oct Posing a Machine Learning Problem Experience Table – Each row is an instance – Each column is an attribute/feature.
Machine Learning1 Machine Learning: Summary Greg Grudic CSCI-4830.
Presentation on Neural Networks.. Basics Of Neural Networks Neural networks refers to a connectionist model that simulates the biophysical information.
Computer Science and Engineering
Using Neural Networks in Database Mining Tino Jimenez CS157B MW 9-10:15 February 19, 2009.
Neural NetworksNN 11 Neural netwoks thanks to: Basics of neural network theory and practice for supervised and unsupervised.
IE 585 Introduction to Neural Networks. 2 Modeling Continuum Unarticulated Wisdom Articulated Qualitative Models Theoretic (First Principles) Models Empirical.
Chapter 9 Neural Network.
Machine Learning Chapter 4. Artificial Neural Networks
11 CSE 4705 Artificial Intelligence Jinbo Bi Department of Computer Science & Engineering
Machine Learning Dr. Shazzad Hosain Department of EECS North South Universtiy
NEURAL NETWORKS FOR DATA MINING
LINEAR CLASSIFICATION. Biological inspirations  Some numbers…  The human brain contains about 10 billion nerve cells ( neurons )  Each neuron is connected.
1 Pattern Classification X. 2 Content General Method K Nearest Neighbors Decision Trees Nerual Networks.
Non-Bayes classifiers. Linear discriminants, neural networks.
Smart Home Technologies Data Mining and Prediction.
Back-Propagation Algorithm AN INTRODUCTION TO LEARNING INTERNAL REPRESENTATIONS BY ERROR PROPAGATION Presented by: Kunal Parmar UHID:
Neural Networks Demystified by Louise Francis Francis Analytics and Actuarial Data Mining, Inc.
Introduction to Neural Networks Introduction to Neural Networks Applied to OCR and Speech Recognition An actual neuron A crude model of a neuron Computational.
Neural Networks Presented by M. Abbasi Course lecturer: Dr.Tohidkhah.
Neural Networks Teacher: Elena Marchiori R4.47 Assistant: Kees Jong S2.22
Chapter 8: Adaptive Networks
Neural Networks. Background - Neural Networks can be : Biological - Biological models Artificial - Artificial models - Desire to produce artificial systems.
SUPERVISED AND UNSUPERVISED LEARNING Presentation by Ege Saygıner CENG 784.
Data Mining: Concepts and Techniques1 Prediction Prediction vs. classification Classification predicts categorical class label Prediction predicts continuous-valued.
Combining Models Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya.
Learning in Neural Networks
Data Mining Lecture 11.
Machine Learning Today: Reading: Maria Florina Balcan
Neural Networks Advantages Criticism
XOR problem Input 2 Input 1
Artificial Intelligence Chapter 3 Neural Networks
Instance Based Learning
Neural Networks Geoff Hulten.
Artificial Intelligence Chapter 3 Neural Networks
A task of induction to find patterns
CS621: Artificial Intelligence Lecture 22-23: Sigmoid neuron, Backpropagation (Lecture 20 and 21 taken by Anup on Graphical Models) Pushpak Bhattacharyya.
A task of induction to find patterns
Artificial Intelligence Chapter 3 Neural Networks
Presentation transcript:

Intelligent Environments1 Computer Science and Engineering University of Texas at Arlington

Intelligent Environments2 Prediction for Intelligent Environments Motivation Techniques Issues

Intelligent Environments3 Motivation An intelligent environment acquires and applies knowledge about you and your surroundings in order to improve your experience. “acquires”  prediction “applies”  decision making

Intelligent Environments4 What to Predict Inhabitant behavior Location Task Action Environment behavior Modeling devices Interactions

Intelligent Environments5 Example Where will Bob go next? Location t+1 = f(…) Independent variables Location t, Location t-1, … Time, date, day of the week Sensor data Context Bob’s task

Intelligent Environments6 Example (cont.) TimeDateDayLocation t Location t /25MondayBedroomBathroom /25MondayBathroomKitchen /25MondayKitchenGarage /25MondayGarageKitchen /25MondayKitchenBedroom /25MondayBedroomLiving room /25MondayLiving roomBathroom /25MondayBathroomBedroom /26TuesdayBedroomBathroom

Intelligent Environments7 Example Learned pattern If Day = Monday…Friday & Time > 0600 & Time < 0700 & Location t = Bedroom Then Location t+1 = Bathroom

Intelligent Environments8 Prediction Techniques Regression Neural network Nearest neighbor Bayesian classifier Decision tree induction Others

Intelligent Environments9 Linear Regression xy

Intelligent Environments10 Multiple Regression n independent variables Find b i System of n equations and n unknowns

Intelligent Environments11 Regression Pros Fast, analytical solution Confidence intervals y = a ± b with C% confidence Piecewise linear and nonlinear regression Cons Must choose model beforehand Linear, quadratic, … Numeric variables

Intelligent Environments12 Neural Networks

Intelligent Environments13 Neural Networks synapses per neuron Synapses propagate electrochemical signals Number, placement and strength of connections changes over time (learning?) Massively parallel

Intelligent Environments14 Computer vs. Human Brain ComputerHuman Brain Computational units1 CPU, 10 8 gates10 11 neurons Storage units10 10 bits RAM, bits disk neurons, synapses Cycle time10 -9 sec10 -3 sec Bandwidth10 9 bits/sec10 14 bits/sec Neuron updates / sec

Intelligent Environments15 Computer vs. Human Brain “The Age of Spiritual Machines,” Kurzweil.

Intelligent Environments16 Artificial Neuron

Intelligent Environments17 Artificial Neuron Activation functions

Intelligent Environments18 Perceptron

Intelligent Environments19 Perceptron Learning

Intelligent Environments20 Perceptron Learns only linearly-separable functions

Intelligent Environments21 Sigmoid Unit

Intelligent Environments22 Multilayer Network of Sigmoid Units

Intelligent Environments23 Error Back-Propagation Errors at output layer propagated back to hidden layers Error proportional to link weights and activation Gradient descent in weight space

Intelligent Environments24 NN for Face Recognition 90% accurate learning head pose for 20 different people.

Intelligent Environments25 Neural Networks Pros General purpose learner Fast prediction Cons Best for numeric inputs Slow training Local optima

Intelligent Environments26 Nearest Neighbor Just store training data (x i,f(x i )) Given query x q, estimate using nearest neighbor x k : f(x q ) = f(x k ) k nearest neighbor Given query x q, estimate using majority (mean) of k nearest neighbors

Intelligent Environments27 Nearest Neighbor

Intelligent Environments28 Nearest Neighbor Pros Fast training Complex target functions No loss of information Cons Slow at query time Easily fooled by irrelevant attributes

Intelligent Environments29 Bayes Classifier Recall Bob example D = training data h = sample rule

Intelligent Environments30 Naive Bayes Classifier Naive Bayes assumption Naive Bayes classifier y represents Bob’s location

Intelligent Environments31 Bayes Classifier Pros Optimal Discrete or numeric attribute values Naive Bayes easy to compute Cons Bayes classifier computationally intractable Naive Bayes assumption usually violated

Intelligent Environments32 Decision Tree Induction Day Time > 0600 Location t Time < 0700 Bathroom M…F yes Bedroom … no Sat Sun

Intelligent Environments33 Decision Tree Induction Algorithm (main loop) 1. A = best attribute for next node 2. Assign A as attribute for node 3. For each value of A, create descendant node 4. Sort training examples to descendants 5. If training examples perfectly classified, then Stop, else iterate over descendants

Intelligent Environments34 Decision Tree Induction Best attribute Based on information-theoretic concept of entropy Choose attribute reducing entropy (~uncertainty) from parent to descendant nodes A1A2 Bathroom (0) Kitchen (50) Bathroom (50) Kitchen (0) Bathroom (25) Kitchen (25) Bathroom (25) Kitchen (25) ??BK v2v2 v1v1 v1v1 v2v2

Intelligent Environments35 Decision Tree Induction Pros Understandable rules Fast learning and prediction Cons Replication problem Limited rule representation

Intelligent Environments36 Other Prediction Methods Hidden Markov models Radial basis functions Support vector machines Genetic algorithms Relational learning

Intelligent Environments37 Prediction Issues Representation of data and patterns Relevance of data Sensor fusion Amount of data

Intelligent Environments38 Prediction Issues Evaluation Accuracy False positives vs. false negatives Concept drift Time-series prediction Distributed learning