1 Andrew Ng, Associate Professor of Computer Science Robots and Brains

2 Who wants a robot to clean your house? [Photo Credit: iRobot]

3 Stanford STAIR Robot [Credit: Ken Salisbury]

4

5 What’s missing? Control Perception The software

6 Stanford autonomous helicopter

7 Computer GPS Accelerometers Compass

8

9 Computer program to fly helicopter [Courtesy of David Shim]

10 Option 1 BLACK

11 Machine learning Option 2

12 Machine learning

13 Machine learning to fly helicopter

14 What’s missing? The software Control Perception

15 “Robot, please find my coffee mug”

16 “Robot, please find my coffee mug” Mug

17 Why is computer vision hard? But the camera sees this:

18 Computer programs (features) for vision SIFT Spin image HoG Textons Shape context GIST

19 Why is speech recognition hard? What a microphone records: “Robot, please find my coffee mug.”

20 Computer programs (features) for audio ZCR Spectrogram MFCC Rolloff Flux

21 The idea: Most of perception in the brain may be one simple program

22 Auditory cortex learns to see Auditory Cortex The “one program” hypothesis [Roe et al., 1992]

23 Somatosensory cortex learns to see The “one program” hypothesis Somatosensory Cortex [Roe et al., 1992]

24 Neurons in the brain

25 Neural Network x1x1 x2x2 x3x3 Output Layer L 1 Layer L 2 Layer L 4 Layer L 3 x4x4

26 How does the brain process images? Neuron #1 of visual cortex (model) Neuron #2 of visual cortex (model) Primary visual cortex looks for “edges.”

27 Comparing to Biology Learning algorithm Visual cortex [PICTURE]

28 Comparing to Biology Learning algorithm Auditory cortex [PICTURE]

29 Computer vision results (NORB benchmark) Neural Network: accuracy Classical computer vision (SVM): accuracy

30 Missed Mugs True positivesFalse positives

31 Missed Mugs True positivesFalse positives

32 Missed Mugs True positivesFalse positives

33 Missed Mugs True positivesFalse positives

34 Missed Mugs True positivesFalse positives Results using non-embodied vision

35 Missed Mugs True positivesFalse positives

36 Missed Mugs True positivesFalse positives Results using non-embodied vision

37 Missed Mugs True positivesFalse positives Classifications using embodied agent

38 Missed Mugs True positivesFalse positives

39 Missed Mugs True positivesFalse positives Results using non-embodied vision

40 Missed Mugs True positivesFalse positives

41 Missed Mugs True positivesFalse positives

42 Hope of progress in Artificial Intelligence