Matching brain and body dynamics Daniel Wolpert: – "Why don't plants have brains?" – "Plants don't have to move!" Early phases of embodied artificial intelligence: – Navigation and orientation – Wheeled robots: Mismatch in complexity levels of sensory, neural and motor systems (unsuccessful, and inconsistent with the “ecological balance principle”)

Matching brain and body dynamics Research inspired by rats – Place cells (O’Keefe and Dostrovsky, 1971) – Rat-mimicking robots Problem 1: complex sensory-motor skills not accounted for Problem 2: rat cognition not limited to navigation

An important goal of developmental robotics is to construct a robot which develops into an intelligent adult human-like robot Development of sensory and motor systems – Biological organisms changes shape Growth cannot yet be modeled well on robots – Robot experiment by Max Lungarella From low-precision to high-precision motor system From low-resolution to high-resolution sensory system Using previously aquired skills to learn new ones Time – Many time scales involved (millisecs -> years) – Today, robot learn a skill very quickly Other aspects of development

Motivation – Why would a robot want to develop/improve over time? – Autotelic principle Good for more complicated tasks A single motivating principle Inspired by the concept of “flow”: Experience/feeling when performing an expert activity Maintain constant flow of information through system Example: learn to reach -> learn to grasp -> learn to drink Other aspects of development

Learning in embodied systems Exploration guided by embodiment facilitates learning Example: Baby learning to categorize objects – Hypothetically, random signal are sent to arm muscles – Movements constrained by anatomy and muscle-tendon properties – Objects grasped due to (hypothetical) grasp reflex – At some point, object is brought to mouth, leading to strong sensory input (sensory-motor correlation) – Eventually, categorization possible with less interaction

Social interaction Important aspect of development Imitation – Powerful strategy for humans – Requires sophisticated perceptual abilities – Mirror neurons (Gallese et al, 1996) – Shared/joint attention – Scaffolding Natural language – Essential for human development – Can emerge from interaction among robot agents

Summary Human-like robots – Has potential, but are not very human-like – Haptic system, mouth region, tongue, speech system, advanced muscle-tendon system Design principles for developmental systems – Time-scale integration principle – Development as an incremental process principle – Discovery principle – Social interaction principle – Motivated complexity principle

Place cells, grid cells, head-direction cells, border cells, and more.. A manifold of spatial maps in the brain Derdikman & Moser, TICS, 2010

Place cells Fire specifically when animal is at certain position Allow read-out of position Located in the hippocampus Strongly influenced by distal sensory cues Active both in light and dark – i.e. not dependent on vision only

Place cells Can also encode – Olfactory information – Time intervals Remapping can occur when environment is changed

Grid cells Multiple firing locations, forming a hexagonal grid of the entire space Located in the parahippocampal cortex (input/output to hippocampus) Grid space, orientation and phase Modulated by velocity

KTH research contributing to grid cell research

Head-direction cells Cells that only fire when head is pointing in certain direction Located in the parahippocampal cortex

Border cells Cells that only fire when animal is close to a geometric boundary Located in the parahippocampal cortex Discovered in 2008

Multi-compartment environment Rapid transition between spatial maps

Development of spatial maps Critical spatial representation components are already present when rat explores open environment outside the nest for the first time (Langston et al, & Wills et al, Science, 2010) Head-direction cells: – present 2 days after eye-opening Place cells: – present 2 days after eye-opening Grid cells: – present 4-8 days after eye-opening In line with Immanuel Kant’s theory that the concept of space does not require experience

Questions remaining How does place cells relate to grid cells? Do humans have neurons with similar functions? – Place cells (Ekstrom et al, 2003) – Grid cells (Doeller et al, 2010) – Path cells (Jacobs et al, 2010) Clockwise/counterclockwise paths Neurosurgical patients in VR-game Entorhinal cortex

Mouse place cells in virtual reality experiment Virtual reality environment created with Quake2 video game engine youtube

Mouse place cells in virtual reality experiment