6 th Framework Programme - Priority 2 “Information Society Technologies” FP6/2003/IST/2 Joint Action Science and Technology JAST - FP Joint-Action Science and Technology Ruud Meulenbroek Nijmegen Institute for Cognition and Information Radboud University Nijmegen, NL JAST - IPM

Overview 1.Goals 2.Means 3.Co-ordinator 4.Organization 5.Activities 6.Website 7.Delegation Joint-Action Science and Technology

1. Goals Cognitive Systems –JAST will extend cognitive science beyond the domain of studying individual cognitive systems Ultimate goal of JAST –To build jointly-acting autonomous systems that communicate and work intelligently on non-trivial mutual tasks Primary goal –Understanding the cognitive architecture of collaborating cognitive systems

2a. Means: Consortium Partners 1.Radboud Universiteit Nijmegen - Nijmegen Institute for Cognition and Information - FC Donders Centre for Cognitive Neuroimaging The Netherlands – NICI + FCDC (Bekkering, Meulenbroek, Hagoort, Toni) 2.Max Planck Institute for Biological Cybernetics, Tübingen Germany – MPIT (Buelthoff, Thornton, Chatziastros) 3.Institute of Communication and Computer Systems, Athens Greece – ICCS (Uzunoglu et al.) 4.The University of Edinburgh United Kingdom – UEDIN (Gurman-Bard, Carletta, Oberlander) 5.Max Planck Institute for Psycholinguistics Nijmegen The Netherlands – MPIN (Levinson, de Ruiter) 6.Technische Universität München Germany – TUM (Knoll et al.) 7.Universidade do Minho Portugal – UMP (Bicho, Erlhagen)

2b. Means: Multidisciplinary team Partners Disciplines 1.NICI Neurocognitive scientists 2.FCDCid. 3.MPIT Cognitive scientists 4.ICCS Neuroscientists 5.UEDINPsycholinguists 6.MPIN id. 7.TUM Robotocists 8.UMP id.

2c. Means: Common Scenario Arbitrary initial object layout Resulting objects after completion of multimodal dialogue sequence Joint-Action Science and Technology

2d. JAST Construction Task Joint-Action Science and Technology (video)

2e: State-of-the-art starting conditions: Demonstrator-Hardware –Two directly cooperating arms. A third arm may be used for holding aggregates –Assembly without any fixtures or specialised tools –A rich set of action primitives (motor, sensor, sensorimotor) has been implemented for positioning, grasping, assembling, disassembling, goal-directed motion under collision avoidance, etc. –Robust recognition and manipulation of parts in all positions –3 Robots, 10 colour cameras (partly articulated)

2f. Beyond state-of-the-art goals System to be developed: real-time, adaptive planning dialogue-supported perception, reasoning, action cross-modal associations model of action-dynamics rule- and self-organization based learning monitoring and repairing errors by self/other multiple degrees-of-freedom in perception-action

3a. Co-ordinator

3b. Manager

4. Work-Packages

5. Activities RTD activities in WP –See excerpt JAST ( Two-day Opening Conference –Athens, Greece –February, 2005 –Dates: to be announced Summer School in 2006

6. JAST Website

7. Delegation Members of JAST attending the CogSys Kick-Off Meeting