iCub Interactive Tutoring Demo Lars Schillingmann, Katrin Lohan, Sebastian Wrede

Research Questions How does robot feedback elicit tutoring behavior? Compare tutoring behavior towards robot with saliency-based vs acoustic-packaging based behavior Hyp. 1: Signalling detected AP ends to user will increase contingency in interaction Temporal structure of tutor‘s eye gaze bouts to robot should be similar as in infant directed interactions Hyp. 2: APs are related to action structure and therefore signal understanding motionese behavior of tutor should be affected of robot‘s timing of feedback ITALK Year 1 Review Düsseldorf, 30 June 2009

Scenario iCub Tutor Sensors Microphone array Camera Actuators Head Eyes Microphone array Camera (right eye) ITALK Year 1 Review Düsseldorf, 30 June 2009

Scenario Tutor task Explain objects to iCub by action and speech Robot feedback Saliency based behavior After still head phase: Start of acoustic packaging behavior After a certain phase with no acoustic package Start saliency based behavior ITALK Year 1 Review Düsseldorf, 30 June 2009

Scenario Saliency based behavior Look at most salient point by moving head and eyes Saliency modules Colors Skin-color Motion Acoustic packaging behavior Behavior begin: Surprise lock eye gaze to pre-defined tabel-top position AP start: neutral AP end: smile ITALK Year 1 Review Düsseldorf, 30 June 2009

System Architecture Subsumption-like coupling of two separate subsystems Acoustic Packaging iCub Attention Acoustic packaging overrides behavior of attention system Analysis of voice activity and motion history Overrides motion control commands generated by attention selection Provides robot feedback through iCub’s face expressions Tutoring feedback triggered through multi-modal analysis of State of robots attention Human-robot interaction AP-Phase Neutral Attentive Smiling

Software Architecture Challenges Effective re-use of components Technological coupling of sub-systems across different frameworks Functional coupling, in particular coordination of sub-systems Approach Integration of Yarp into new event-based data-flow engine (part of XCF Toolkit) Development of arbitration component

Current State of Integration Up-front staff (and knowledge) exchange Robot working on-site iCub face assembled successfully at BIEL robot Yarp integrated in BIEL software infrastructure Gmake autobuild system (Ubuntu Linux) iCub modules Acoustic packaging Yarp and XCF middleware

Brief Outlook on Integration Summary Highly integrated system for interactive tutoring Real robot system for experimental analysis and exploration of tutoring situations Successful deployment of iCub hardware platform Demonstration of software re-use and cross-framework integrability Next steps Yarp interface for Esmeralda (VVV09) Evaluation of advanced feedback strategies Head pose / gaze Gestures

Thanks to... Katrin Lohan (BIEL) Lars Schillingmann (BIEL) Vadim Tikhanoff (IIT)‏ Marco Maggiall (IIT) Stefan Krüger (BIEL) Sebastian Wrede (BIEL) ITALK Year 1 Review Düsseldorf, 30 June 2009

Demo live from Bielefeld... Lars Schillingmann Sebastian Wrede Katrin Lohan iCub Video Conferencing: Michael Johannfunke ITALK Year 1 Review Düsseldorf, 30 June 2009

Thank you! ITALK Year 1 Review Düsseldorf, 30 June 2009