Bruxelles, October 3-4, 2000 10036 Interface Multimodal Analysis/Synthesis System for Human Interaction to Virtual and Augmented Environments IST Concertation.

Slides:

Advertisements

Similar presentations

The Access Grid Ivan R. Judson 5/25/2004.

Advertisements

WEB- BASED TRAINING Chapter 4 Virginija Limanauskiene, KTU, Lithuania.

Immersive Media Vision Create seamless immersive environment for distributed, interactive real and virtual events Reproduce audio, video and other senses.

Information Society Technologies Third Call for Proposals Norbert Brinkhoff-Button DG Information Society European Commission Key action III: Multmedia.

SpeciesLink The Brazilian experience on setting up a network Renato De Giovanni Centro de Referência em Informação Ambiental, CrIA.

WSF 2010 towards Dakar … and beyond. Goals of This Meeting  Open a discussion on the issues and challenges on communication in the WSF process  Exchange.

Audio-based Emotion Recognition for Advanced Information Retrieval in Judicial Domain ICT4JUSTICE 2008 – Thessaloniki,October 24 G. Arosio, E. Fersini,

Expressive Tangible Acoustic Interfaces Antonio Camurri, Corrado Canepa, and Gualtiero Volpe InfoMus Lab, DIST-University of Genova, Viale Causa 13, Genova,

Matthias Wimmer, Ursula Zucker and Bernd Radig Chair for Image Understanding Computer Science Technische Universität München { wimmerm, zucker, radig

ENTERFACE’08 Multimodal Communication with Robots and Virtual Agents.

Designing Facial Animation For Speaking Persian Language Hadi Rahimzadeh June 2005.

Facial expression as an input annotation modality for affective speech-to-speech translation Éva Székely, Zeeshan Ahmed, Ingmar Steiner, Julie Carson-Berndsen.

3D Face Modeling Michaël De Smet.

Mining Minds Presenter12-July-2014KHU Information Curation Layer Low Level Context-awareness.

Live Conferencing Tim Neumann Learning Technologies Unit Institute of Education.

 INTRODUCTION  STEPS OF GESTURE RECOGNITION  TRACKING TECHNOLOGIES  SPEECH WITH GESTURE  APPLICATIONS.

Data warehouse example

Experiences with Structured Recording and Replay in Interactive Remote Instruction Kurt Maly et al. Old Dominion University, Norfolk, Virginia, USA.

Artificial Intelligence & Information Analysis Group (AIIA) Centre of Research and Technology Hellas INFORMATICS & TELEMATICS INSTITUTE.

Communicating with Avatar Bodies Francesca Barrientos Computer Science UC Berkeley 8 July 1999 HCC Research Retreat.

EKMAN’S FACIAL EXPRESSIONS STUDY A Demonstration.

Recognizing Emotions in Facial Expressions

Sunee Holland University of South Australia School of Computer and Information Science Supervisor: Dr G Stewart Von Itzstein.

Network of Excellence in Content Based Semantic Scene Analysis and Information Retrieval Coordinating Partner: Centre for Research and Technology Hellas.

Database Construction for Speech to Lip-readable Animation Conversion Gyorgy Takacs, Attila Tihanyi, Tamas Bardi, Gergo Feldhoffer, Balint Srancsik Peter.

Using ICT to Support Students who are Deaf. 2 Professional Development and Support: Why? Isolation Unique and common problems Affirmation Pace of change.

GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004.

1 Darmstadt, October 02, 2007 Amalia Ortiz Asociación VICOMTech Mikeletegi Pasealekua Donostia - San Sebastián (Gipuzkoa)

Tools for Middle School Students to Create Vignettes Naomi Caldwell Jeff Naisbitt Matt Miller.

Chapter 7. BEAT: the Behavior Expression Animation Toolkit

Three Topics Facial Animation 2D Animated Mesh MPEG-4 Audio.

EStream – Best Practice in the Use of Streaming Media © A. Knierzinger, C. Weigner Increasing the use of Streaming technology in school education in Europe.

2 CEVI (Interactive Rendering Center) is a research center linked to the Language and Information Systems department in the Universitat Jaume I. Research.

SPEECH CONTENT Spanish Expressive Voices: Corpus for Emotion Research in Spanish R. Barra-Chicote 1, J. M. Montero 1, J. Macias-Guarasa 2, S. Lufti 1,

Distributed Objects and Middleware. Sockets and Ports Source: G. Coulouris et al., Distributed Systems: Concepts and Design.

VoiceUNL : a proposal to represent speech control mechanisms within the Universal Networking Digital Language Mutsuko Tomokiyo (GETA-CLIPS-IMAG) & Gérard.

Presented By:- Sudipta Dhara Roll Table of Content Table of Content 1.Introduction 2.How it evolved 3.Need of Middleware 4.Middleware Basic 5.Categories.

Multimodality, universals, natural interaction… and some other stories… Kostas Karpouzis & Stefanos Kollias ICCS/NTUA HUMAINE WP4.

1 Galatea: Open-Source Software for Developing Anthropomorphic Spoken Dialog Agents S. Kawamoto, et al. October 27, 2004.

Online Learning Florence Martin Associate Professor in Instructional Technology

Tracking Functionality Changes in IRI: A Distance Education Software System C. Michael Overstreet, Kurt Maly, Ayman Abdel-Hamid, Ye Wang Old Dominion University.

Chris Hewitt, Wild Mouse Male, Age 42, Happy ARC31 2.

Jennifer Lee Final Automated Detection of Human Emotion.

THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW The EyesWeb XMI multimodal platform Overview 5 March 2015.

ENTERFACE’08 Multimodal Communication with Robots and Virtual Agents mid-term presentation.

Ekman’s Facial Expressions Study A Demonstration.

Facial Expressions and Emotions Mental Health. Total Participants Adults (30+ years old)328 Adults (30+ years old) Adolescents (13-19 years old)118 Adolescents.

Interpreting Ambiguous Emotional Expressions Speech Analysis and Interpretation Laboratory ACII 2009.

Building Educational Virtual Environments page 1 ICALT 2002 Building Educational Virtual Environments C. Bouras, and T. Tsiatsos Computer Engineering and.

NCP meeting Jan 27-28, 2003, Brussels Colette Maloney Interfaces, Knowledge and Content technologies, Applications & Information Market DG INFSO Multimodal.

Under Guidance of Mr. A. S. Jalal Associate Professor Dept. of Computer Engineering and Applications GLA University, Mathura Presented by Dev Drume Agrawal.

What is it? Details Look at the whole.  Form of communication ◦ Communicating without words ◦ Non-verbal ◦ Using facial expressions ◦ Gestures  Can.

Nataliya Nadtoka James Edge, Philip Jackson, Adrian Hilton CVSSP Centre for Vision, Speech & Signal Processing UNIVERSITY OF SURREY.

ICT Support for Students in Higher Education The Open University of Israel Department of Education and Psychology Tali Heiman: Dorit.

Automated Detection of Human Emotion

Common object request broker

Tim Neumann Learning Technologies Unit Institute of Education

Using Body Language The Better Speaker Series 279.

Inventory of Distributed Computing Concepts and Web services

CONNECTIVE APP Connect, Communicate , Encourage, Educate, and aware disable people.

What is blue eyes ? aims on creating computational machines that have perceptual and sensory ability like those of human beings. interactive computer.

Towards lifelike Computer Interfaces that learn

Communicating with Avatar Bodies

The Second Elearning Workshop

Multimodal Caricatural Mirror

Project #2 Multimodal Caricatural Mirror Intermediate report

Presented by: Mónica Domínguez

Founded Meuro/year funding 12 FP6 and 20 FP5 IST projects

e-Inclusion Information Day January, Brussels

Automated Detection of Human Emotion

Presentation transcript:

Bruxelles, October 3-4, Interface Multimodal Analysis/Synthesis System for Human Interaction to Virtual and Augmented Environments IST Concertation Meeting Bruxelles - October 3-4, 2000

Bruxelles, October 3-4, 2000 The Consortium DIST – University of Genoa IC Lernout & Hauspie BP Linköpings Universitet SP Universitat Politecnica de Catalunya EP Ecole Polytechnique Fédérale de Lausanne CH P University of Geneva CHP Informatics and Telematics Institute ELP Tecnologia Automazione Uomo scrl. IP ELAN Informatique FP University of Maribor SIP Curtin University of Technology AUP Umea University SP Centre National de la Recherche Scientifique FP W Interactive SA FP

Bruxelles, October 3-4, 2000 A “man in the computer” Natural Speech Natural Expressions Emotional Speech Synthesis Emotional Face&Body Animation ?

Bruxelles, October 3-4, 2000 Man-to-machine action coherent analysis of audio-video channels high level interpretation and data fusion speech emotion understanding facial expression classification

Bruxelles, October 3-4, 2000 Machine-to-man feedback human-like audio-video feedback simulating a "person in the machine” MPEG-4 face and body animation text-to-speech with lip synchronization speech and facial expressions emotional synthesis

Bruxelles, October 3-4, 2000 WORK DONE SO FAR Specifications of the common sw platform Bimodal multi-language, multi-speaker corpus recording Development of preliminary tools Implementation of the intermediate common sw platform Finalization of the CfP for ICAV3D Participation to IBC2000 Preparation toward IST2000 Preliminary market study

Bruxelles, October 3-4, 2000 The INTERFACE platform Network Platform (large set of distributed independent tools) Integrated Platform (a reduced set of strongly integrated tools) Demonstration Platform (personalization of the Integrated Platform on specific application dependent context)

Bruxelles, October 3-4, 2000 Tools under development Low/high level video analysis Low/high level speech analysis High level facial animation Natural dialog manager 3D human and cartoon characters Phoneme markers to FAP conversion Emotional speech synthesis High level body animation

Bruxelles, October 3-4, 2000 The Network CSP C IDL C++ IDL Java IDL Client C IDL C++ IDL Java IDL Server ORB - Object Request Broker

Bruxelles, October 3-4, 2000 Mapping for the 14 MPEG-4 visemes p, b, mf, v T, D t, d k, g tS, dZ, Ss, z n, l r A eI O U

Bruxelles, October 3-4, 2000 Facial emotion modeling and synthesis Happiness Sadness Anger Fear Disgust Surprise

Bruxelles, October 3-4, 2000 Bimodal emotional analysis : training Classifier (GMM) Video Features Audio Analysis Audio Frames Video Analysis Video Frames Features Selection Frames #

Bruxelles, October 3-4, 2000 Bimodal emotion synthesis: runtime Audio Analysis Classifier (GMM) Audio Features Audio Frames Predefined Movements or Expressions “Expression Blender”

Bruxelles, October 3-4, 2000 Feature tracking tool: Track2FAP

Bruxelles, October 3-4, 2000 Applications Virtual speaker for web news announcing Virtual operator for web call centers Virtual teacher for remote teaching Virtual salesman in e-commerce Virtual friend for web chatting Virtual guide for Internet navigation