1 Andrew Ng, Associate Professor of Computer Science Robots and Brains

2 Who wants a robot to clean your house? [Photo Credit: iRobot]

3 Stanford AI Robot [Credit: Ken Salisbury]

4 Stanford AI Robot [Credit: Ken Salisbury]

5

6 What’s missing? Control Perception The software

7 Stanford autonomous helicopter

8 Computer GPS Accelerometers Compass

9 Stanford autonomous helicopter GPS Accelerometers Compass

10

11 Mathematical specification of helicopter [Courtesy of David Shim]

12 [Courtesy of David Shim]

13 Machine learning

14 Machine learning

15 Machine learning to fly helicopter

16 What’s missing? The software Control Perception

17 “Robot, please find my coffee mug”

18 “Robot, please find my coffee mug” Mug

19 Why is computer vision hard? But the camera sees this:

20 Computer programs (features) for vision SIFT Spin image HoG Textons Shape context GIST

21 Why is speech recognition hard? What a microphone records: “Robot, please find my coffee mug.”

22 Computer programs (features) for audio ZCR Spectrogram MFCC Rolloff Flux

23 The idea: Most of perception in the brain may be one simple program.

24 Auditory cortex learns to see Auditory Cortex The “one program” hypothesis [Roe et al., 1992]

25 Somatosensory cortex learns to see The “one program” hypothesis Somatosensory Cortex [Metin & Frost, 1989]

26 Neurons in the brain

27 Neural Network (Sparse Learning) x1x1 x2x2 x3x3 Output Layer L 1 Layer L 2 Layer L 4 Layer L 3 x4x4

28 How does the brain process images? Neuron #1 of visual cortex (model) Neuron #2 of visual cortex (model) Visual cortex looks for lines/edges.

29 Comparing to Biology Visual cortexLearning algorithm

30 Comparing to Biology Auditory cortex Learning algorithm

31 Comparing to Biology Visual Learning algorithm Brain Sound

32 Computer vision results (NORB benchmark) Neural Network: accuracy Classical computer vision (SVM): accuracy

33 Correctly found mugMistake

34 Correctly found mugMistake

35 Correctly found mugMistake

36 Correctly found mugMistake

37 Hope of progress in Artificial Intelligence

39 Option 1 BLACK

40 Machine learning Option 2

41 Comparing to Biology Learning algorithm Visual cortex [PICTURE]

42 Comparing to Biology Learning algorithm Auditory cortex

43 Missed Mugs True positivesFalse positives

44 Missed Mugs True positivesFalse positives

45 Missed Mugs True positivesFalse positives Results using non-embodied vision

46 Missed Mugs True positivesFalse positives

47 Missed Mugs True positivesFalse positives Results using non-embodied vision

48 Missed Mugs True positivesFalse positives Classifications using embodied agent

49 Missed Mugs True positivesFalse positives

50 Missed Mugs True positivesFalse positives Results using non-embodied vision

51 Missed Mugs True positivesFalse positives

52 Missed Mugs True positivesFalse positives

53 Missed Mugs True positivesFalse positives

54 Missed Mugs True positivesFalse positives