2. Modeling the Brain’s Operating System Dana H. Ballard Computer Science Dept. University of Austin Texas, NY, USA International Symposium “Vision by Brains and Machines” November 13th-17th Montevideo, Uruguay

3. Embodied Cognition Maurice Merleau-Ponty 1906-1961 World Body Brain

4. Timescales 10 -2 10 -1 10 1 10 2 10 0 Round-trip through Cortical Memory Shortest Recognition time Modal fixation time Attention Switching Time Sentence generation Speed Chess minimum search Activity time 10 3 Memory encoding sec Continuous Discrete

6. Marr Brooks =

7. Behaviors compete for body’s motor resources Behaviors obtain sensory information Behaviors are scheduled from a pool Three Levels of a Human “Operating System” Behavior 1 2 3

8. Task: Make a PBJ sandwich Computational Abstraction Hierarchy Component: Remove Jelly Jar Lid Routine: Locate Lid

9. Multi-tasking As revealed by gaze sharing in human data Shinoda and Hayhoe, Vision Research 2001

10. Roelfsema et al PNAS 2003 Visual Routines

11. Introducing “Walter” Pickup cans Stay on sidewalk Avoid obstacles

12. Control of visuo-motor routines “active” “inactive” + only ~4 can run simultaneously + 100-200ms update per behavior

13. Walter’s Visual Routines image Can locationsSidewalk location1-d obstacle locs

14. You are here state action Reinforcement Learning Primer : Before Learning

15.          policy value Reinforcement Learning Primer : After Learning

16. Microbehavior for Litter Cleanup 2b. Value of Policy Q ,d d  2a. Policy 1. Visual Routine Heading from Walter’s perspective

17. Learned Microbehaviors LitterSidewalkObstacles

18. Microbehaviors and the body’s resources “active” “inactive” Walking direction uses weighted average of Q values. Gaze direction must use a single best Q value.

19. The best Q given a sample state The expected Q given the state uncertainty Which Microbehavior should get the gaze vector? -

20. obs can side obs can side

21. Performance Comparison

22. Walter crosses the street Pickup cans Stay on sidewalk Avoid obstacles

23. Running Behaviors: Eye Movement Trace

24. Three trials

25. Eyetracker in V8 helmet

26. A curved path in real space Produces the perception of a straight path in visual space Human Ss walk Walter’s route in Virtual Reality. Their 6 dof head position and 2 dof gaze positions are continuously tracked. Three subjects were used. The resultant video and eye track signal are scored frame-by- frame. Methods

27. A human walks Walter’s route

28. Obstacle Litter Sidewalk Corner Crosswalk Otherside Human data: individual fixations

29. Walter(3 trials) Human subjects(3) Walter and the humans have similar task priorities

30. Human data: Two samples with different contexts Obstacle avoidance Litter Sidewalk Crosswalk Otherside Near obstacles Approaching crosswalk Walter

31. On Crosswalk Approaching crosswalk Walter and the human Ss all exhibit context sensitivities. Human gaze locations are interpreted based on gaze location. The actual internal state is unknown. Waiting for light Walter Human Ss Scheduling Context

32. Rewards can be changed quickly LitterSidewalkObstacles

33. Walter Humans Changing the reward schedule “ignore the litter”“ignore the obstacles”

34. Saliency Map vs Gaze courtesy of program provided by Dr. Laurent Itti at the iLab, USC Match No match

35. Credit Assignment - MIT model

36. Credit Assignment - Our Model

37. The laboratory at Rochester Computer ScienceCognitive Science Dana BallardMary Hayhoe Brian Sullivan Jelena Jovancevic Constantin Rothkopf Alumni Chen Yu Pili Aivar Nathan Sprague Jochen Triesch Al Robinson Neil Mennie Weilie Yi Jason Droll Xue Gu Jonathan Shaw