1. Departments of Physics and Informatics
The Age of Robotics
Kevin H. Knuth, Ph.D.
Departments of Physics and Informatics
University at Albany

2. “If every tool, when ordered, or even of its own accord, could do the work that befits it... If the weavers' shuttles were to weave of themselves, then there would be no need either of apprentices for the master workers or of slaves for the lords.”
Aristotle, “De Republica Atheniensium”, ~322 BC

3. Leonardo da Vinci The Father of Robotics
2 May 2008
Kevin H Knuth

4. Leonardo’s Robot
Based on da Vinci’s sketches from 1495 The robot is a knight in German-Italian Medieval Armor
2 May 2008
Kevin H Knuth

5. (Image: Institute and Museum of the History of Science, Florence)
Leonardo’s Automata A small wooden car, powered by springs could be programmed by inserting various cogs and gears.
(Image: Institute and Museum of the History of Science, Florence)
2 May 2008
Kevin H Knuth

6. The Dream of Robotics has long been with us…
Maria from Fritz Lange’s Metropolis (1927)
2 May 2008
Kevin H Knuth

7. The Dream of Robotics has long been with us…
The Tin Man Frank L. Baum’s The Wizard of Oz
(1900 Book, 1939 Film)
Robby the Robot Forbidden Planet
1956
2 May 2008
Kevin H Knuth

8. And has changed with us…
Bender Futurama
(1999-Present)
R2D2 and C3PO Star Wars
(1977)
2 May 2008
Kevin H Knuth

9. Reality is always Stranger than Fiction…
Opportunity currently on Mars
Cassini in orbit around Saturn
2 May 2008
Kevin H Knuth

10. And creeps up on us Unsuspectingly…
2 May 2008
Kevin H Knuth

11. And creeps up on us Unsuspectingly…
2 May 2008
Kevin H Knuth

12. How do we make them INTELLIGENT?
2 May 2008
Kevin H Knuth

13. How Do We Function?
2 May 2008
Kevin H Knuth

14. The Brain: The Living State of Matter
Neurons
Connections per Neuron
Maximum Firing Rate: 1 ms
1kHz massively parallel computer
Information Processed on order of 100s ms
MUST use Prior Information
The Virtual Hospital, Ch 5, Williams, Gluhbegovic, and Jew
2 May 2008
Kevin H Knuth

15. A Powerful Computer
Hree is an ecxlelnet eaxmlpe of how yuor wnodreful mnid can raed tihs txet eevn touhgh its all jmbuled.
2 May 2008
Kevin H Knuth

16. Sounds and Prior Information
Listen to these sounds…
Sounds from Haskins Laboratories, Rubin, Remez, Pardo
2 May 2008
Kevin H Knuth

17. Sounds and Prior Information
Now listen to this one…
Sounds from Haskins Laboratories, Rubin, Remez, Pardo
2 May 2008
Kevin H Knuth

18. Sounds and Prior Information
And now go back to this one…
Sounds from Haskins Laboratories, Rubin, Remez, Pardo
2 May 2008
Kevin H Knuth

19. Sounds and Prior Information
What about the rest?
Sounds from Haskins Laboratories, Rubin, Remez, Pardo
2 May 2008
Kevin H Knuth

20. Prior Information is Key
Only 10% of the inputs into primary visual cortex come from the retina via the lateral geniculate nucleus. The rest come from higher visual and frontal areas.
Perception can also be modified by attention.
Thus the brain can actively focus on relevant information.
Human Brain: basal view (front at top) The Virtual Hospital, Ch 5, Williams, Gluhbegovic, and Jew
2 May 2008
Kevin H Knuth

21. The Brain Models its Environment
The frontal regions of the brain create models of the world based on prior experience. These models affect perception and attention.
In addition, the brain models itself.
Experiments in multi-sensory processing has shown that the information processing is consistent with Bayes Theorem
The Virtual Hospital, Ch 5, Williams, Gluhbegovic, and Jew
2 May 2008
Kevin H Knuth

22. Thinking Machines
Your frontal lobes carry a model of yourself that is continually updated from data received from a dense sensor network. This implements both ‘Instrument Health Monitoring’ and ‘Calibration’
You learn from new data by updating your model of the world.
You actively seek new data by asking relevant questions.
2 May 2008
Kevin H Knuth

23. Body and Brain form a Symbiotic Unit
2 May 2008
Kevin H Knuth

24. Relevant Information
2 May 2008
Kevin H Knuth

25. Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, (Originally published in Russian 1962)
2 May 2008
Kevin H Knuth

26. Three minute recording
Free Examination
Three minute recording
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, (Originally published in Russian 1962)
2 May 2008
Kevin H Knuth

27. Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, (Originally published in Russian 1962)
2 May 2008
Kevin H Knuth

28. Estimate Ages of the People
Three minute recording
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, (Originally published in Russian 1962)
2 May 2008
Kevin H Knuth

29. Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, (Originally published in Russian 1962)
2 May 2008
Kevin H Knuth

30. Remember their Clothes
Three minute recording
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, (Originally published in Russian 1962)
2 May 2008
Kevin H Knuth

31. Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, (Originally published in Russian 1962)
2 May 2008
Kevin H Knuth

32. Estimate Material Circumstances
Three minute recording
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, (Originally published in Russian 1962)
2 May 2008
Kevin H Knuth

33. Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, (Originally published in Russian 1962)
2 May 2008
Kevin H Knuth

34. How Long has the Visitor been away?
Three minute recording
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, (Originally published in Russian 1962)
2 May 2008
Kevin H Knuth

35. Relevance and Perception
A. L. Yarbus, Eye Movements and Vision, Plenum, New York, (Originally published in Russian 1962)
2 May 2008
Kevin H Knuth

36. Do We Analyze Everything?
2 May 2008
Kevin H Knuth

37. http://viscog.beckman.uiuc.edu/grafs/demos/15.html 2 May 2008
Kevin H Knuth

38. Inattentional Blindness
2 May 2008
Kevin H Knuth

39. AUTOMATED INQUIRY
2 May 2008
Kevin H Knuth

40. Spirit and Opportunity: Remote Science
More and more are our instruments required to perform science operations further from the intervention of humans.
Dust devils whip across Gusev Crater on Mars
2 May 2008
Kevin H Knuth

41. The Expansive Floor of Gusev Crater
2 May 2008
Kevin H Knuth

42. Rock Outcrop (Methuselah)
2 May 2008
Kevin H Knuth

43. Underwater Robotic Explorers
At the Monterey Bay Aquarium Research Institute (MBARI), researchers are employing robotic submarines to explore the Deep Pacific Ocean.
2 May 2008
Kevin H Knuth

44. Intelligent Autonomous Instruments
Require:
Stability Control
Instrument Health Monitoring
Automated Calibration
Accurate Onboard Data Analysis
Adequate Data Coverage
Ability to Actively Seek Data
2 May 2008
Kevin H Knuth

45. Novel Instrument Design
To accomplish these goals, these novel instruments must
Monitor their own state (health and calibration)
Infer their state from self-sensing
Be equipped with dense sensor networks
Infer calibration parameters
Learn from data
Make inferences from data
Perform hypothesis testing
Ask new questions
Actively seek new data
Select optimal experiments
2 May 2008
Kevin H Knuth

46. The Basic Components
2 May 2008
Kevin H Knuth

47. The LEGO Mindstorms NXT System1
The NXT Brick is the brain of the system. 2
Touch Sensor 3
Microphone 4
Light Sensor
Ultrasonic Rangefinder 5
$250!
Servo Motors 6
2 May 2008
Kevin H Knuth

48. Lego teams with HiTecnic
NEW!
Accelerometer
Prototype Board
Color Sensor
Digital Compass
Sensor and Motor
Multiplexers
2 May 2008
Kevin H Knuth

49. Robotic Scientists This robot is equipped with a light sensor.
It is to locate and characterize a white circle on a black playing field with as few measurements as possible.
LANDMINE DETECTION!
2 May 2008
Kevin H Knuth

50. The Robot’s “Thoughts”
Past Measurement DARK
Past Measurement LIGHT
Next Measurement
Mean Circle
Set of Hypothesized Circles
Area within Robot’s Reach colored according to ENTROPY
Robot Center
2 May 2008
Kevin H Knuth

51. George Eliot (Mary Anne Evans),
'Am I already in the shadow of the Coming Race? and will the creatures who are to transcend and finally supersede us be steely organisms, giving out the effluvia of the laboratory, and performing with infallible exactness more than everything that we have performed with a slovenly approximativeness and self-defeating inaccuracy?'
George Eliot (Mary Anne Evans),
The Impressions of Theophrastus Such, 1879.
2 May 2008
Kevin H Knuth

52. Special Thanks to:
John Skilling Ariel Caticha Janos Aczél Keith Earle Philip Erner Deniz Gencaga Philip Goyal Steve Gull Jeffrey Jewell Carlos Rodriguez
And also to:
Emily Knuth Rockne Knuth Ann Knuth Joshua Knuth Roland Derouen Bernadette Derouen
For their patience with the robotic invasion that has been thrust upon them.