1. Overview of Year 1 Progress Angelo Cangelosi & ITALK team
italkproject.org

2. Integration of partners Talks overview
Project summary
Vision and methodology
Objectives
Main results
Methodological
Scientific
Integration of partners
Collaboration space map
Talks overview
ITALK Year 1 Review Düsseldorf, 30 June 2009

3. Integration of partners Talks overview
Project summary
Vision and methodology
Objectives
Main results
Methodological
Scientific
Integration of partners
Collaboration space map
Talks overview
ITALK Year 1 Review Düsseldorf, 30 June 2009

4. ITALK Strategic Goal Project centred on two main S/T hypotheses:
The parallel development of action and social interaction permits the bootstrapping of language capabilities, which on their part enhance cognitive development (e.g. Arbib 2002, Corballis 2002, Glenberg 1997, Tomasello 2003; Cangelosi & Riga 2006)
Developmental robotic approaches based on the integration of action, social interaction and language have fundamental technological implications for designing communication in robots (e.g. Cangelosi et al. 2005; Metta et al. 2006; Weng et al. 2001; Roy 2005; Nehaniv et al. 2007)

5. Integration & Cognitive Bootstrapping
ITALK Research Themes
iCub
Development
& Extension
ITALK Research
Emergence of Language
Action
Learning
(Manipulation)
Conceptual
Learning
Integration & Cognitive Bootstrapping
Social
Learning
& Interaction

6. Integration & Cognitive Bootstrapping
ITALK Research Themes
iCub
Development
& Extension
WP1:
Robot & Action
(IIT)
ITALK Research
WP4: Language
(BIEL, USD)
Emergence of Language
Action
Learning
(Manipulation)
Conceptual
Learning
WP5: Integrate
(PLYM, ISAB, all)
Integration & Cognitive Bootstrapping
WP2:Concept.
(CNR)
Social
Learning
& Interaction
WP3: Social
(UH)

7. Objectives
To provide new theoretical insights, models and scientific explanations of the integration of action, social and linguistic skills
To develop an interdisciplinary set of methods
To develop innovative and cognitively plausible engineering principles, techniques and approaches
To demonstrate the effectiveness of the above scientific and technical advances through the use of robotic experiments
To provide new theoretical insights, models and scientific explanations of the integration of action, social and linguistic skills and in particular on the hypothesis that action, social and linguistic knowledge co-develop and further bootstrap cognitive development
To develop an interdisciplinary set of methods for analyzing the interaction of language, action and cognition in humans and artificial cognitive agents using robot learning experiments, computer simulations, cognitive linguistic analysis, and experimental investigations from developmental linguistics, the neuroscience of language and action, and human-robot interaction experiments;
To develop innovative and cognitively plausible engineering principles, techniques and approaches for the design of communicative and linguistic capabilities in cognitive robots able to interact with their physical and social world and to manipulate entities, artefacts and other agents including humans;
To demonstrate the effectiveness of the above scientific and technical advances through the use of robotic experiments on the acquisition of object manipulation, social skills and linguistic capabilities in simulated and physical cognitive robots. In particular, robotic agents will be able to:
acquire complex object manipulation capabilities through social interaction;
develop an ability to create and use embodied concepts;
develop social skills that allow flexible interaction with other agents or people;
develop linguistic abilities to communicate about their interaction with the world.

8. Objective I – year 1
To provide new theoretical insights, models and scientific explanations…
to develop roadmap for future research the identifies the main challenges and theoretical, methodological, and experimental tools in the study of language development in robots (WP5)
to consider the role of preliminary robotic experiments for action and language integration research (WP1-5)

9. Objective II – year 1 To develop interdisciplinary set of methods…
to develop and test the iCub simulator for robotic simulation experiments (WP1)
to develop constructivist grammar classifications of linguistic constructions from robot directed speech as input to developmental robotics experiments (WP3)
to test the use of machine learning techniques (e.g. neural networks, artificial vision routines, speech recognition systems) for developmental robotics experiments on action, categorisation, social and language learning (WP1-5)
to use experimental HRI and neuroscience methods for robot learning experiments (WP3-5)

10. Objective III – year 1
To develop innovative and cognitively plausible engineering principles, techniques and approaches…
to design and develop an enhanced iCub robotic platform for language learning and object manipulation experiments (WP1)
to use standardisation methods for action and language learning experiments (WP1, WP5)

11. Objective IV – year 1
To demonstrate the effectiveness … through the use of robotic experiments
to plan and carry out preliminary robotic experiments on the development of object manipulation capabilities (WP1)
to plan and carry out preliminary robotic experiments of how an artificial embodied agent can develop embodied concepts (WP2)
to plan and carry out preliminary robotic experiments on socially constructed linguistic phenomena (WP3, WP4, WP5)

12. Integration of partners Talks overview
Project summary
Vision and methodology
Objectives
Main results
Methodological
Scientific
Integration of partners
Collaboration space map
Talks overview
ITALK Year 1 Review Düsseldorf, 30 June 2009

13. Main Methodological Results
Objectives II (methods) and III (eng. principles)
Extended iCub platforms for linguistic experiments
Four new iCubs
Open Source iCub Simulator

14. Main Methodological Results (1)
Extended iCub platforms for linguistic experiments
Microphones
CFW custom board for video and audio signals acquisition
ESMERALDA
Sphinx

15. Main Methodological Results (2)
2. Four new iCubs

16. Main Methodological Results (3)
3. Open Source iCub Simulator
Software development (IIT, PLYM, CNR)
Single and multi-robot scenario
Preliminary experiments
Used outside consortium (e.g. projects CHRIS, IMclever)

17. Main Methodological Results (3)
3. Open Source iCub Simulator
- “realistic” model of physical robot

18. Main Methodological Results (3)
3. Open Source iCub Simulator
- multi-engine (ODE, Newton)
- single and multi-agent

19. Main Scientific Results
Objectives I (theory) and IV (experiments)
Roadmap for future research
ROSSUM architecture scaling-up
Analyses of tutoring behaviour
Cognitive linguistic analyses
Simulation experiments (Y1)
Planning experiments (Y2+)

20. Main Scientific Results (1)
Roadmap for future research
International workshop
Public roadmap document
Submitted to:

21. Main Scientific Results (2)
2. ROSSUM architecture scaling-up
Experiment with Kaspar II

22. Main Scientific Results (3)
3. Analyses of tutoring behaviour
Tutoring Spotter
Acoustic Packaging methodology

23. Main Scientific Results (3)
3. Analyses of tutoring behaviour
Tutoring Spotter

24. Main Scientific Results (3)
3. Analyses of tutoring behaviour
With automatic motion recognition

25. Main Scientific Results (4)
4. Cognitive linguistic analyses
development of psycholinguistically motivated grammar learning scenarios
based on empirical analyses of child- directed and robot-directed speech.
criteria for the creation of suitable input for embodied grammar learning experiments were devised on the basis of the empirical analyses

26. Main Scientific Results (5)
5. Simulation experiments (Y1)
Cognitive architecture for understanding simple instruction
Hybrid HRI-simulator testing
Neural oscillator model for selective attention
Active categorical perception
Pilot experiments with physical robot

27. Main Scientific Results (5)
5. Simulation experiments (Y1)
Understanding simple instructions

28. Main Scientific Results (5)
5. Simulation experiments (Y1)
Hybrid HRI-simulator testing

29. Main Scientific Results (5)
5. Simulation experiments (Y1)
Neural oscillator model for selective attention (Borisyuk et al. in press)

30. Main Scientific Results (5)
5. Simulation experiments (Y1)
Active categorical perception (Massera et al. 2009)

31. Main Scientific Results (6)
6. Planning experiments (Y2+)
Action and Language compositionality with Tani’s networks (T1.4, 4.1, 4.3)
Biases in category learning (T2.2)
HRI Experiments on gesture, negation and grammar (T3.2, 3.3)
Acoustic Packaging analyses (T4.2)
Neuroscience and gaze in HRI (T5.3) …

32. Integration of partners Talks overview
Project summary
Vision and methodology
Objectives
Main results
Methodological
Scientific
Integration of partners
Collaboration space map
Talks overview
ITALK Year 1 Review Düsseldorf, 30 June 2009

33. Collaboration Space Map
Tasks 5.1
RoadMap

34. Collaboration Space Map
Tasks 1.1:
iCub development,
Training, ASR

35. Collaboration Space Map
Task 1.2:
iCub simulator

36. Collaboration Space Map
Tasks 1.4, 4.1, 4.3:
Action, language,
compositionality

37. Collaboration Space Map
Tasks 3.1
Cognitive Linguistic
analysis

38. Collaboration Space Map
Tasks 3.2:
HRI Experiments

39. Collaboration Space Map
ALL TASKS

40. Integration of partners Talks overview
Project summary
Vision and methodology
Objectives
Main results
Methodological
Scientific
Integration of partners
Collaboration space map
Talks overview
ITALK Year 1 Review Düsseldorf, 30 June 2009

41. Time Plan

42. Next presentations
ITALK Year 1 Review Düsseldorf, 30 June 2009