Scenario Specification and Problem Finding Luca Tummolini ISTC-CNR MindRACES, First Review Meeting, Lund, 11/01/2006

MindRACES, First Review Meeting, Lund, 11/01/2006 Objective From the Annex 1: “identify 6 scenarios corresponding to the situation in which cognition requires anticipation and implement 3 of them” Scenarios are used for: Understanding the role of anticipation in cognitive systems Identify the challenges Facilitate future integration MindRACES, First Review Meeting, Lund, 11/01/2006

MindRACES, First Review Meeting, Lund, 11/01/2006 The scenarios Finding and Looking for IDSIA, IST, ISTC-CNR, NBU, UW-COGSCI Anticipation in a dynamic world LUCS, OFAI, UW-COGSCI Guards and Thieves ISTC-CNR, LUCS, NBU, OFAI The scenarios highlight the role of anticipation in: Perception and categorization Selective attention Deliberation Action monitoring and control Skill learning and routinization Coordination Emotions MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 1: Finding and Looking for Partners involved: IDSIA, IST, ISTC-CNR, NBU, UW-COGSCI Two different assumptions: the agent has a model of the environment the agent learns the model of the environment The agent: a pan-and-tilt camera mounted on a fixed base a simulated fovea (higher resolution at the centre) on a mobile robot a robotic arm MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 1: Finding and Looking for Task 1: Finding a specific object Find and reach a specific object in the environment (e.g. a green circle). This task involves: Selective attention Anticipation based on memory Goal-based motor-primitives Detection of regularities in the spatial relations among objects 1. Aggiungere le tre immagini di IDSIA MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 1: Finding and Looking for Task 2: Finding members of a class of objects Find an object transferring a system of relations from one domain to another This task involves: Anticipation by analogy Detection of relations among objects Relation extraction and encoding Categorization of observed objects MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 1: Finding and Looking for Task 3: Looking for an object Look for an object that is hidden in one of the rooms in a believable way The target location can be probabilistically biased towards certain locations This task involves: Prediction by analogy making Decision making about the rooms to visit Building models of the world structure and of the regularities between objects Emotional responses to expected and unexpected events 1. Inserire immagine di boicho MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 2: Anticipation in a Dynamic World Partners involved: LUCS, OFAI, UW-COGSCI This scenario involves prediction of objects with an intrinsic dynamics The first three tasks involve looking at two games with moving objects and the task is to predict the movement of one or several targets The last task deals with the prediction of a rolling ball from a developmental perspective MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 2: Anticipation in a Dynamic World Task 1: The fish catching game Goal: anticipate location and velocity of some targets in the scene The movement of the targets is very regular Prediction from different point of views This task involves: Selective attention Target tracking based on anticipatory model Tracking in a fixed environment Quick relearning of the model when the viewing angle changes 1. Aggiungere immagini del trenino di LUCS, cambiare titolo??? MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 2: Anticipation in a Dynamic World Task 2: Marble run game Goal: anticipate location and velocity of some targets in the scene Paths can be re-arranged Combination of the continuous dynamics of the ball with a compositional structure Generalization between different configurations of the set-up This task involves Selective attention Target tracking based on anticipatory model Tracking of partially occluded and moving targets MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 2: Anticipation in a Dynamic World Task 3: Learning previous two games at the same time Context sensitive learning Priming effects biasing the predictions Detection of relevant context This task involves: Generalization of target model from one viewing condition to another and between the different games Context dependent selection of target, predictive models and tracking strategy Simultaneous modelling, recognition, prediction and estimation of viewer’s position MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 2: Anticipation in a Dynamic World Task 4: Basic how-to knowledge Initial basic interactions with the environment driven by basic instincts and motivations The robot learns through reinforcement which interactions are allowed by the objects in the environment This task involves: The acquisition of object generalisations and concepts Learning of basic dynamical properties of objects (e.g. predict where a ball will reappear behind an obstacle) MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 2: Anticipation in a Dynamic World Task 5: Generalisation If the ball moves slowly, the robot learns to look for the ball on the right side of the wall If the ball moves fast, the robot learns to look for the ball on the other end of the wall If the ball produces a noise, the robot goes to look for it behind the wall This task involves: The development of more sophisticated concepts, such as object persistence Flexible how-to knowledge MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 3: Guards and Thieves Partners involved: ISTC-CNR, LUCS, NBU, OFAI Agents can have two different roles (guards or thieves) Thieves try to steal valuable resources Guards protect the valuables restricting access to them (e.g. blocking the entrance) In some variants of the task, only one of the two roles will be played by an anticipatory cognitive system In the more complex variants, both roles will be played by anticipatory agents 1. Aggiungere le immagini dello scenario di Lucs che presenta MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 3: Guards and Thieves Task 1: Conflict in accessing the valuables - simple Two agents: one thief and one guard The valuables are hidden in at least two different places The session ends either when the thief has collected or found all the valuables or when the guard has blocked the thief. This task involves: Recognition of the adversary among the moving objects Prediction of the adversary behaviour (avoiding/intercepting) Integrating different levels of action control (e.g. routinary, reasoning), based on different kinds of expectations (e.g. implicit, explicit) MindRACES, First Review Meeting, Lund, 11/01/2006

Scenario 3: Guards and Thieves Task 2: Conflict in the access to valuables - complex A social task involving several agents – several thieves and a guard The session ends either when all the valuables have been collected or found or when the guard has blocked all the thieves The thieves should be able to distinguish between guards (danger) and fellows (cooperation) This task involves: Help and critical help by anticipating other’s needs, actions or capabilities, e.g. by removing obstacles or doing part of other’s work Delegating by trusting; e.g. an agent can explicitly “ask another one for help” MindRACES, First Review Meeting, Lund, 11/01/2006

Thank you for the attention MindRACES, First Review Meeting, Lund, 11/01/2006