Modeling the Brain’s Operating System Dana H. Ballard Computer Science Dept. University of Austin Texas, NY, USA International Symposium “Vision by Brains and Machines” November 13th-17th Montevideo, Uruguay

Embodied Cognition Maurice Merleau-Ponty World Body Brain

Timescales Round-trip through Cortical Memory Shortest Recognition time Modal fixation time Attention Switching Time Sentence generation Speed Chess minimum search Activity time 10 3 Memory encoding sec Continuous Discrete

Marr Brooks =

Behaviors compete for body’s motor resources Behaviors obtain sensory information Behaviors are scheduled from a pool Three Levels of a Human “Operating System” Behavior 1 2 3

Task: Make a PBJ sandwich Computational Abstraction Hierarchy Component: Remove Jelly Jar Lid Routine: Locate Lid

Multi-tasking As revealed by gaze sharing in human data Shinoda and Hayhoe, Vision Research 2001

Roelfsema et al PNAS 2003 Visual Routines

Introducing “Walter” Pickup cans Stay on sidewalk Avoid obstacles

Control of visuo-motor routines “active” “inactive” + only ~4 can run simultaneously ms update per behavior

Walter’s Visual Routines image Can locationsSidewalk location1-d obstacle locs

You are here state action Reinforcement Learning Primer : Before Learning

         policy value Reinforcement Learning Primer : After Learning

Microbehavior for Litter Cleanup 2b. Value of Policy Q ,d d  2a. Policy 1. Visual Routine Heading from Walter’s perspective

Learned Microbehaviors LitterSidewalkObstacles

Microbehaviors and the body’s resources “active” “inactive” Walking direction uses weighted average of Q values. Gaze direction must use a single best Q value.

The best Q given a sample state The expected Q given the state uncertainty Which Microbehavior should get the gaze vector? -

obs can side obs can side

Performance Comparison

Walter crosses the street Pickup cans Stay on sidewalk Avoid obstacles

Running Behaviors: Eye Movement Trace

Three trials

Eyetracker in V8 helmet

A curved path in real space Produces the perception of a straight path in visual space Human Ss walk Walter’s route in Virtual Reality. Their 6 dof head position and 2 dof gaze positions are continuously tracked. Three subjects were used. The resultant video and eye track signal are scored frame-by- frame. Methods

A human walks Walter’s route

Obstacle Litter Sidewalk Corner Crosswalk Otherside Human data: individual fixations

Walter(3 trials) Human subjects(3) Walter and the humans have similar task priorities

Human data: Two samples with different contexts Obstacle avoidance Litter Sidewalk Crosswalk Otherside Near obstacles Approaching crosswalk Walter

On Crosswalk Approaching crosswalk Walter and the human Ss all exhibit context sensitivities. Human gaze locations are interpreted based on gaze location. The actual internal state is unknown. Waiting for light Walter Human Ss Scheduling Context

Rewards can be changed quickly LitterSidewalkObstacles

Walter Humans Changing the reward schedule “ignore the litter”“ignore the obstacles”

Saliency Map vs Gaze courtesy of program provided by Dr. Laurent Itti at the iLab, USC Match No match

Credit Assignment - MIT model

Credit Assignment - Our Model

The laboratory at Rochester Computer ScienceCognitive Science Dana BallardMary Hayhoe Brian Sullivan Jelena Jovancevic Constantin Rothkopf Alumni Chen Yu Pili Aivar Nathan Sprague Jochen Triesch Al Robinson Neil Mennie Weilie Yi Jason Droll Xue Gu Jonathan Shaw