Download presentation
Presentation is loading. Please wait.
Published byKeanu Atlee Modified over 10 years ago
2
ARTIFICIAL INTELLIGENCE: THE MAIN IDEAS Nils J. Nilsson OLLI COURSE SCI 102 Tuesdays, 11:00 a.m. – 12:30 p.m. Winter Quarter, 2013 Higher Education Center, Medford Room 226 nilsson@cs.stanford.edu http://ai.stanford.edu/~nilsson/ Course Web Page: www.sci102.com/ For Information about parking near the HEC, go to: http://www.ci.medford.or.us/page.asp?navid=2117 There are links on that page to parking rules and maps
3
AI in the News ?
4
Perception Action Selection Memory PART ONE (Continued) REACTIVE AGENTS
5
Summary: Neural Networks Have Many Applications
6
But Some Are Not Very User- Friendly Fair Isaac Experience
7
Models of the Cortex Using Deep, Hierarchical Neural Networks All connections are bi-directional
9
The Neocortex
10
Two Pioneers in Using Networks to Model the Cortex Geoffrey Hinton Jeff Hawkins Hierarchical Temporal Memory
11
More About Jeff Hawkinss Ideas http://www.numenta.com/htm- overview/education/HTM_CorticalLearningAlgorithms.pdf
12
Dileep Georges Hierarchical Temporal Memory (HTM) Model A Convolutional Network George is a founder of startup, Vicarious http://vicarious.com/team.html
13
A Mini-Column of the Neo-Cortex From: HIERARCHICAL TEMPORAL MEMORY http://www.numenta.com/htm-overview/education/HTM_CorticalLearningAlgorithms.pdf
14
Figure 10. Columnar organization of the microcircuit. George, Dileep and Hawkins, Jeff: (2009) Towards a Mathematical Theory of Cortical Micro-circuits. PLoS Comput Biol 5(10): e1000532. doi:10.1371/journal.pcbi.1000532 http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1000532
15
Figure 9. A laminar biological instantiation of the Bayesian belief propagation equations used in the HTM nodes. George D, Hawkins J (2009) Towards a Mathematical Theory of Cortical Micro-circuits. PLoS Comput Biol 5(10): e1000532. doi:10.1371/journal.pcbi.1000532 http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1000532
16
Ray Kurzweils New Book
17
Unsupervised Learning
18
Letting Networks Adapt to Their Inputs All connections are bi-directional Massive number of inputs Weight Values Become Those For Extracting Features of Inputs Honglak Lee,et al., Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations, Proceedings of the 26th Annual International Conference on Machine Learning, 2009
19
Hubel & Wiesels Detector Neurons Short bar of light projected onto a cats retina Response of a single neuron in the cats visual cortex (as detected by a micro-electrode in the anaesthetized cat) David Hubel, Torsten Wiesel
20
Use of Deep Networks With Unsupervised Learning First Layer Learns Building-Block Features Common to Many Images All connections are bi-directional
21
Second Layer Learns Features Common Just to Cars, Faces, Motorbikes and Airplanes cars, faces, motorbikes, airplanes
22
Third Layer Learns How to Combine the Features of the Second Layer Into a Representation of the Input cars, faces, motorbikes, airplanes
23
Output Layer Can be Used to Make a Decision CAR
24
The Net Can Make Predictions About Unseen Parts of the Input
25
Building High-level Features Using Large Scale Unsupervised Learning Quoc V. Lee,et al. (Google and Stanford) 1,000 Google Computers, 1,000,000,000 Connections
26
10 million 200x200 pixel images downloaded from the Internet (stills from YouTube) Unsupervised learning for three days Large Scale Unsupervised Learning (Continued) a face neuron Recognizes 22,000 object categories a cat neuron
27
81.7% accuracy in detecting faces out of 13,026 faces in a test set One Result http://research.google.com/archive/unsupervised_icml2012.html For more information about these experiments at Google/Stanford, see:
28
Using Models (i.e., Memory) Can Make Agents Even More Intelligent Perception Action Selection Model of World (e.g., a map)
29
Types of Models Maps Memory of Previous States List of State-Action Pairs
30
Models can be pre-installed or learned
31
where am I? where is everything else? Learning and Using Maps Neato Robot Vacuum
32
Neato Robotics Mapping System
33
NEATO ROBOTICS XV11
34
S-R Rules Using State of the Agent Perception Action Selection determines thestateof the world Library of States and Actions (Memory) IF state 1, THEN action a IF state 2, THEN action b...
35
Ways to Represent States Lists of numbers, such as (1,7,3,4,6) Arrays, such as Statements, such as Color(Walls, LightBlue) Shape(Rectangular)...
36
Library of States & Actions (1,7,3,4,6) a (1,6,2,8,7) b (4,5,1,8,5) c... (7,4,8,9,2) k (1,5,2,8,6) Input (present state) Closest Match
37
Example: Face Recognition Using a large database containing many, many images of faces, a small set of building-block faces is computed: The average of all faces: http://cognitrn.psych.indiana.edu/nsfgrant/FaceMachine/faceMachine.html
38
Familiar Uses of Building Blocks A Musical Tone Consists of Harmonics
39
Library of Known Faces (Represented as composites of the building-block faces) (0,0,1,0,0,-2,-2,0,-1,-2,-2,-1,2,-1,0) (2,2,-2,0,0,1,2,2,-1,2,2,-1,,0,2,0) (-3,2,1,1,-2,1,-2,3,0,0,0,-4,-3,2,-2)(4,1,3,-1,4,0,4,4,1,4,4,-4,4,-4,-4) Plus Thousands More SamJoe Sue Mike
40
Library of Known Faces Query Face Represented as a composite of the building-block faces (present state) Sue is the Closest Match Face Recognition Sam Joe Sue Mike (-2,2,1,1,-2,1,-2,3,1,0,0,-4,-3,2,-2) (0,0,1,0,0,-2,-2,0,-1,-2,-2,-1,2,-1,0) (2,2,-2,0,0,1,2,2,-1,,2,2,-1,,0,2,0) (-3,2,1,1,-2,1,-2,3,0,0,0,-4,-3,2,-2) (4,1,3,-1,4,0,4,4,1,4,4,-4,4,-4,-4)
41
Another Kind of Model A table of states and actions and values State and ActionValue State 1, Action b State 1, Action c 13 7 State 2, Action g State 2, Action h State 2, Action j 394394 State 3, Action m State 3, Action n 2626
42
Why have values for multiple actions instead of just noting the best action? Because the values in the table can be changed (learned) depending on experience! REINFORCEMENT LEARNING (Another Point of Contact with Brains)
43
Pioneers in the Use of Reinforcement Learning in AI Andy BartoRich SuttonChris Watkins
44
An Example: Learning a Maze
45
But the Mouse Doesnt Have a Map of the Maze (Like We Do) Instead it remembers the states it visits and assigns their actions random initial values State and ActionValue State 1, up3 State 2, left State 2, down State 2, right 256256 <add more when encountered)
46
It Can Change the Values in the Table The First Step (state 1, up) gets initial random value 3
47
state 2, has 3 actions, each with initial random values There is only one action possible (up), and the mouse ends up in state 2
48
Now the mouse updates the value of (state 1, up) in its table 5 value propagates backward (possibly with some loss)
49
Sooner or later, the mouse stumbles into the goal and gets a reward
50
The reward value is propagated backward value propagates backward (with some loss) 99
51
And So On... With a Lot of Exploration, the Mouse Learns the Maze
52
Reward Centers Alter Dopamine Concentrations Reinforcement Learning in Animals
53
The Brains Reward Centers Associate Values With States A Neural Substrate of Prediction and Reward, Wolfram Schultz, Peter Dayan, P. Read Montague, Science, 275 :1593-1599, 14 March 1997. When The Actual Value of a State is Better Than The Predicted Value, Dopamine is Released; When Worse Than Expected Dopamine is Inhibited.
54
Learning to Flip Pancakes By Sylvain Calinon, Advanced Robotics Department Italian Institute of Technology
55
Learning to Fly a Model Helicopter
56
TD-Gammon Temporal Difference Learning and TD-Gammon, By Gerald Tesauro www.research.ibm.com/massive/tdl.html backgammon boards resulting from a move from current board predicted probabilities of winning (the values of the input boards) train neural network so that value of current board is closer to value of the best next board
57
Summary of TD-Gammon Results
58
Bottom Line: Reactive Agents Can Be Quite Effective! See: Daniel Kahneman, Thinking, Fast and Slow Perception Action Selection Model of World
59
An Interesting Novel About Neural Networks Galatea 2.2, Richard Powers
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.