ICT work programme ICT 22 Multimodal and natural computer interaction Aleksandra Wesolowska (Unit G.3 - Data Value Chain) Juan Pelegrin (Unit G.4 - Inclusion, skills & youth) Philippe Gelin (Unit G.2 – Creativity)

Multimodal & Natural human-computer interaction ICT 22 – 2014 – Call 1 Rationale systems and devices are becoming more and more powerful the human-computer interface is lagging behind bottleneck for efficient and seamless use Objective achieve transparency and invisibility of technology effortless, effective human-machine dialogue easy use of complex and powerful systems, easy access to information Multidisciplinary research: communication technologies, language/speech processing, cognitive & behavioural analysis, creative industries… Various uses: search, information retrieval, elderly, people with special needs, designers/artists... 2

Multimodal & Natural human-computer interactionICT 22 – R&I action (a) Several research projects (budget: 7,5 MEUR) on advanced human machine interaction interactive, intuitive, multimodal systems: natural ways of delivering answers to user queries human-like social agents: communicative, conversational, affective and social capabilities autonomous: able to learn and react proactively to new situations conversational/spoken interaction: able to cope with spontaneous dialogue, in multiple languages knowledge-based agents: able to find, retrieve, combine/integrate information from many sources multiple delivery platforms for developed technologies research based on and/or producing freely available & reusable resources 3

Multimodal & Natural ICT 22 – R&I action (a) human-computer interaction Expected impact 4 ICT 22 – R&I action (a) Improve multilingual speech processing and bridge the gap between recognition and synthesis, exploiting metadata and other contextual data Increase the automatic inferences capacities on rich context thanks to improved language understanding, sensed environments/objects, use of social media and agent’s experience

Multimodal & Natural ICT 22 – R&I action (b) human-computer interaction Expected impact R&I projects on Multimodal and Natural interaction (Budget: 16 MEUR) Develop multi modal & adaptive interfaces (inc. BNCI) to assist people with disabilities Research: User interaction with intelligent systems user modelling How to detect behaviours Interoperability standards Interaction and cooperation between machine and human intelligence Advance human machine interaction to enable disabled/ elderly to participate fully in society 5

Multimodal & Natural ICT 22 – Innovation action (c) human-computer interaction Several Innovation projects on multimodal interfaces (7,5 MEUR) What is exected more efficient and natural ways of interacting with computers improve users' experience. How Create non-intrusive interaction with humans where real and virtual content are blended. Starting From one or multiple smart devices and sensors (scene analysis, voice recognition, human position, gestures and body language detection,…) Thanks to Built with a user centric approach Result: cost effective; address clear market needs be validated ( in domains such as those of the creative industries fields.) 6

Multimodal & Natural ICT 22 – Innovation action (c) human-computer interaction Expected impact 7 ICT 22 – R&I action (a) Enable better uses of ICT technologies within the creative industries by providing directly usable solutions addressing their specific needs. Provide a large spill over of the knowledge acquired to a maximum of European industries. Improve the competitive position of the European industries through the provision of cost effective, innovative and high-value products and services.

8 Thank you! ICT 22 action (a) ICT 22 action (b) ICT 22 action (c)