Intelligent Environments1 Computer Science and Engineering University of Texas at Arlington
Intelligent Environments2 Prediction for Intelligent Environments Motivation Techniques Issues
Intelligent Environments3 Motivation An intelligent environment acquires and applies knowledge about you and your surroundings in order to improve your experience. “acquires” prediction “applies” decision making
Intelligent Environments4 What to Predict Inhabitant behavior Location Task Action Environment behavior Modeling devices Interactions
Intelligent Environments5 Example Where will Bob go next? Location t+1 = f(…) Independent variables Location t, Location t-1, … Time, date, day of the week Sensor data Context Bob’s task
Intelligent Environments6 Example (cont.) TimeDateDayLocation t Location t /25MondayBedroomBathroom /25MondayBathroomKitchen /25MondayKitchenGarage /25MondayGarageKitchen /25MondayKitchenBedroom /25MondayBedroomLiving room /25MondayLiving roomBathroom /25MondayBathroomBedroom /26TuesdayBedroomBathroom
Intelligent Environments7 Example Learned pattern If Day = Monday…Friday & Time > 0600 & Time < 0700 & Location t = Bedroom Then Location t+1 = Bathroom
Intelligent Environments8 Prediction Techniques Regression Neural network Nearest neighbor Bayesian classifier Decision tree induction Others
Intelligent Environments9 Linear Regression xy
Intelligent Environments10 Multiple Regression n independent variables Find b i System of n equations and n unknowns
Intelligent Environments11 Regression Pros Fast, analytical solution Confidence intervals y = a ± b with C% confidence Piecewise linear and nonlinear regression Cons Must choose model beforehand Linear, quadratic, … Numeric variables
Intelligent Environments12 Neural Networks
Intelligent Environments13 Neural Networks synapses per neuron Synapses propagate electrochemical signals Number, placement and strength of connections changes over time (learning?) Massively parallel
Intelligent Environments14 Computer vs. Human Brain ComputerHuman Brain Computational units1 CPU, 10 8 gates10 11 neurons Storage units10 10 bits RAM, bits disk neurons, synapses Cycle time10 -9 sec10 -3 sec Bandwidth10 9 bits/sec10 14 bits/sec Neuron updates / sec
Intelligent Environments15 Computer vs. Human Brain “The Age of Spiritual Machines,” Kurzweil.
Intelligent Environments16 Artificial Neuron
Intelligent Environments17 Artificial Neuron Activation functions
Intelligent Environments18 Perceptron
Intelligent Environments19 Perceptron Learning
Intelligent Environments20 Perceptron Learns only linearly-separable functions
Intelligent Environments21 Sigmoid Unit
Intelligent Environments22 Multilayer Network of Sigmoid Units
Intelligent Environments23 Error Back-Propagation Errors at output layer propagated back to hidden layers Error proportional to link weights and activation Gradient descent in weight space
Intelligent Environments24 NN for Face Recognition 90% accurate learning head pose for 20 different people.
Intelligent Environments25 Neural Networks Pros General purpose learner Fast prediction Cons Best for numeric inputs Slow training Local optima
Intelligent Environments26 Nearest Neighbor Just store training data (x i,f(x i )) Given query x q, estimate using nearest neighbor x k : f(x q ) = f(x k ) k nearest neighbor Given query x q, estimate using majority (mean) of k nearest neighbors
Intelligent Environments27 Nearest Neighbor
Intelligent Environments28 Nearest Neighbor Pros Fast training Complex target functions No loss of information Cons Slow at query time Easily fooled by irrelevant attributes
Intelligent Environments29 Bayes Classifier Recall Bob example D = training data h = sample rule
Intelligent Environments30 Naive Bayes Classifier Naive Bayes assumption Naive Bayes classifier y represents Bob’s location
Intelligent Environments31 Bayes Classifier Pros Optimal Discrete or numeric attribute values Naive Bayes easy to compute Cons Bayes classifier computationally intractable Naive Bayes assumption usually violated
Intelligent Environments32 Decision Tree Induction Day Time > 0600 Location t Time < 0700 Bathroom M…F yes Bedroom … no Sat Sun
Intelligent Environments33 Decision Tree Induction Algorithm (main loop) 1. A = best attribute for next node 2. Assign A as attribute for node 3. For each value of A, create descendant node 4. Sort training examples to descendants 5. If training examples perfectly classified, then Stop, else iterate over descendants
Intelligent Environments34 Decision Tree Induction Best attribute Based on information-theoretic concept of entropy Choose attribute reducing entropy (~uncertainty) from parent to descendant nodes A1A2 Bathroom (0) Kitchen (50) Bathroom (50) Kitchen (0) Bathroom (25) Kitchen (25) Bathroom (25) Kitchen (25) ??BK v2v2 v1v1 v1v1 v2v2
Intelligent Environments35 Decision Tree Induction Pros Understandable rules Fast learning and prediction Cons Replication problem Limited rule representation
Intelligent Environments36 Other Prediction Methods Hidden Markov models Radial basis functions Support vector machines Genetic algorithms Relational learning
Intelligent Environments37 Prediction Issues Representation of data and patterns Relevance of data Sensor fusion Amount of data
Intelligent Environments38 Prediction Issues Evaluation Accuracy False positives vs. false negatives Concept drift Time-series prediction Distributed learning