ICDL 2004 IST RobotCub RobotCub IST

ICDL 2004 IST RobotCub Main Keywords Cognition (manipulation) Human Development Embodiment Community Building

ICDL 2004 IST RobotCub Two Goals or a two-fold Goal? Advance our understanding of several key issues in cognition Create a physical platform for embodied cognitive research

ICDL 2004 IST RobotCub Main Figures RobotCub is a 5 years projects Presently the consortium is composed of 16 Partners, 11 from Europe, 3 from Japan and 2 from the USA. Total Funding is 8.5 M€ and total effort is 1,651 person-months (138 person-years) Coordinator is the LIRA-Lab of the University of Genova: Giulio Sandini, Giorgio Metta and David Vernon

ICDL 2004 IST RobotCub Current Consortium

ICDL 2004 IST RobotCub An open system! Robot-Cub engineering goal is to build a humanoid platform whose design is open to the scientific community and can be duplicated and improved by the community of its users. We are looking for other projects willing to join RobotCub with the same open attitude. Clearly stated in our Consortium Agreement

ICDL 2004 IST RobotCub Seeking International Collaboration The European Commission is funding part of the internationalization activities and the project will support the research of groups interested in using the robot platform (about 2 Meuro have been set aside for this purpose). …reference person’s name from each of the ongoing projects interested to contribute…

ICDL 2004 IST RobotCub Main Objectives (60 months) 1...a professionally documented, reproducible open platform shaped like a child-humanoid 2...the understanding through real-world implementation of exploratory and manipulation-based cognitive skills 3...the study and implementation of the initial period of human cognitive development in an embodied artificial system 4...building of an international scientific community

ICDL 2004 IST RobotCub …which rest on several props Unity of cognition: body and mind-ware co-design: –Requirements: hands, dexterity, etc. Exploiting holistic design: –Complex bodies requires global optimization Cost effectiveness: –An open system can lead to a more efficient design/exploitation process

ICDL 2004 IST RobotCub Science-wise Manipulation as a link between action and perception Manipulation from grasping to gesture production with communicative intents From tool use to the creation of tool-ware

ICDL 2004 IST RobotCub General layout (60 months) TimeActivities 0 Start of activities on specific aspects of cognition (several parallel activities), start the design of the CUB, start internationalization activities, start open source support 18 months First evaluation of cognitive investigation Mechanical design of the CUB 3 years Robot-cub and mindware integration Launch of competitive call for proposals 3 yearsNew investigation starts using the open platform 5 years Integration of the mindware developed during the five years

ICDL 2004 IST RobotCub Activity allocation Building the robotic platform Doing a lot of science Building a communit y

ICDL 2004 IST RobotCub ACT 1: Mind (Cognitive Neuroscience) RC 1.3 Interaction and Affordance RC 1.3 Interaction and Affordance RC 1.4 Interaction and Imitation RC 1.4 Interaction and Imitation RC 1.2 Bimanual Coordination RC 1.2 Bimanual Coordination RC 1.5 Interaction and Communication RC 1.5 Interaction and Communication RC 1.1 Eye-Head-Hand Coordination RC 1.1 Eye-Head-Hand Coordination ACT 2: Body (Cognitive Robotics) RC 2.2 Arm-Hand and Touch RC 2.2 Arm-Hand and Touch RC 2.3 Spine-Legs and Walk RC 2.3 Spine-Legs and Walk RC 2.4 Electronics and Control RC 2.4 Electronics and Control RC 2.5 Enabling Technologies RC 2.5 Enabling Technologies RC 2.1 Head-eye and Vision RC 2.1 Head-eye and Vision ACT 3: Open System (Dissemination) RC 3.2 Electronic Drawings RC 3.2 Electronic Drawings RC 3.3 Firmware and Software RC 3.3 Firmware and Software RC 3.4 CUB Manufacture RC 3.4 CUB Manufacture RC 3.5 Internationalization RC 3.5 Internationalization RC 3.1 Mechanical Drawings RC 3.1 Mechanical Drawings RC 3.6 Maintenance and Update RC 3.6 Maintenance and Update Learning and Development Community Building Activities

ICDL 2004 IST RobotCub Child-like, how much? Approx 934mm 243mm 369mm 439mm Avg. 14Kg lb

ICDL 2004 IST RobotCub Temporal Dependencies (60 Months) ACT 3: Open System (Dissemination) ACT 2: Body (Cognitive Robotics) ACT 1: Mind (Cognitive Neuroscience) Experiments with existing platforms Experiments with new components Define hardware architecture Design new components Realize new components Realize whole system Experiment with whole system Specifications Design Documents Executive Documents Maintenance and Update Final Demonstration Robot-Cub Platform Revised Executive Documents Define manipulation experiments Scientific Dissemination Activities time 18 months 60 months

ICDL 2004 IST RobotCub The six objectives of month 18 SO-1 A model of human infants’ cognitive development based on recent and well documented experimental results. The model shall include, in an experimentally reproducible way, a description of how the robotic artifact develops over time showing the formation of manipulation skills of varying levels. Psychophysical and behavioral experiments will be planned and carried out to answer specific questions on the implementation and to acquire relevant missing information about the developmental process. Responsibility: Claes von Hofsten (UNIUP)

ICDL 2004 IST RobotCub SO-2 The complete design of all CUB components and a suitable integration plan. This includes the definition of the functional and technical specifications of the CUB mechanics, electronics and software architecture. At this stage of the project, the partners responsible for the CUB design and for testing the CUB individual components will have: a) completed the design stage, b) run a mechanical compatibility test and c) written a preliminary plan for integration.. Responsibility: Giorgio Metta (UGDIST)

ICDL 2004 IST RobotCub SO-3 The initial results of the implementation of cognitive abilities in an artificial system. This objective will be demonstrated through extensive testing of the robots’ cognitive abilities in realistic situations, implemented in several of the existing robotic platforms. In addition to basic manipulatory and visual skills, the robots will be equipped with a number of basic social skills, enabling natural interactions between robots and humans. Responsibility: David Vernon (UGDIST)

ICDL 2004 IST RobotCub SO-4 Results of the testing of new technologies to be used in the CUB platform. Particularly important for the scope and goal of Robot-cub is to monitor and to test continuously new technologies for sensors and actuators as well as the electronic (HW/SW) components of the Open System Platform. Responsibility: Paolo Dario (SSSA)

ICDL 2004 IST RobotCub SO-5 The Community Building activities will be carried out all along the project’s duration. Responsibility: Giulio Sandini (UGDIST)

ICDL 2004 IST RobotCub SO-6 Define and refine the IPR and licencing rules of robotcub Responsibility: David Vernon (UGDIST)

ICDL 2004 IST RobotCub Contacts Next meeting is November in Genova Everybody is welcome!

ICDL 2004 IST RobotCub How do we get there through a roadmap?

ICDL 2004 IST RobotCub Synthesize our knowledge of human cognitive development Neuroscience leads  Organize state of the art understanding of cognition Prepare, think, plan, new experiments to fill the gaps Derive suggestions for robotic experiments, CUB requirements, materials, skills, control, etc.

ICDL 2004 IST RobotCub Since we’re focusing on manipulation DevelopmentLearning what Perception/Action exploitation Component of Social Interaction Goal of system Immediate No reaching yet Head-eye coordinationGazing, smooth pursuitShared attentionLook around Pre-ReachingApproach an object Controlling arm and hand movements in space PointingTouch Delay between action onset and consequen ces Power grasping Eye-hand coordination based on object position and object motion Anticipatory closing of the hand Reaching for object held by other person “I got it!” Grasp (become “owner”) Differentiated grasping Adjustments to object shape and size Eye-arm-hand coordination based on object s’ shape Take and give Grasp appropriately (geometric) Object manipulation Objects’ affordances Eye-arm-hand Coordination based on actions to be executed on objects Play games Handle objects appropriately (use) Long delays Imitate acts on objects Associate what is seen with what the system can do What I do looks like what I see Play gamesAction’s Interpretation Act to Communicate Associate what is seen (perceived) with “meaning” What I do generates some reactions CommunicationAction’s meaning

ICDL 2004 IST RobotCub Start the robot design investigation Leveraging on our past experience different aspects of the CUB design are handled in parallel Specific targeted investigation on, for instance, materials or new sensing, actuation technology might be launched [budget has been allocated for this purpose]

ICDL 2004 IST RobotCub In parallel start mindware investigation Robotic experiments are initiated relying on existing setups Aims: –clarify specific aspects of interests along the developmental timeline –Uncover the developmental mechanisms useful in building adaptive machines –Clarify the link between action, perception, and cognition

ICDL 2004 IST RobotCub Opening the system Starting the organization of the open system repository Establishing adequate level of documentation [both SW and HW] Establishing standards (e.g. for sharing upgrades) Attracting the community to the CUB

ICDL 2004 IST RobotCub Start building a community Dissemination: advertise the project worldwide Training: getting young scientists involved with cognitive robotics Lobbying: getting other funding agencies (third country participants) involved with financing cognitive robotics actions

ICDL 2004 IST month 18 Coalesce the activities into:  Design of the CUB and integration plan  Design of parts of the mindware architecture and stricter integration plan  Specific results from technology search [to go into the CUB], from neuroscience [to fill the “gaps”], and from the internationalization and training activities [international community building]  …and more importantly a demonstration!

ICDL 2004 IST RobotCub The 18 months  Robot  Play-pal  Sitting around a table, playing with toys, tools, objects…  Learn how to:  Properly grasp a toy/tool  Associate tools to specific actions  Adjust the action wrt the environmental conditions  Combine primitive grasp/manipulative actions  Interact appropriately with the play-pal in learning a new game or task

ICDL 2004 IST RobotCub month 18  Initial specifications of the CUB (open system)  Definition of the cognitive architecture and initial implementation of manipulative behaviors  Creation of the core components of the international community and plans for an international project