Activity presentation www-clips.imag.fr Director: Jean Caelen (DR CNRS) Vice-Director : Catherine Berrut (Prof. UJF)

Institutions UMR 5524

Research Topics Man-Machine Interaction(s) Collaborative interactions with machines Speech, Image, Language Processing E-Learning Perceptive Rooms and Augmented Reality Multimedia Document Indexing Man Machine Interaction Engineering Speech Recognition and Natural Language Dialogue Collaborative and Multimodal Interfaces Information Retrieval Machine Translation Language Processing

Structure Direction Research ARCADE GEOD GETA IIHM MRIM Laboratoty council Direction council Technical staff Maintenance Projects Administration Secretariat People Management Financial Subsistance Communication Valorisation MultiCom People : 72 (staff + PhD students) UJF, CNRS, INPG, UPMF, US

Research Teams ARCADE (J.P. Peyrin, prof. UJF) e-Learning (Ateliers de Réalisation et de Conception d’Applications Destinées à l’Education) GEOD (J.F. Serignat, MCF INPG) Speech Group (Groupe d’Etude sur l’Oral et le Dialogue) GETA (C. Boitet, prof. UJF) Machine Translation Group (Groupe d’Etude sur la Traduction Automatique) IIHM (J. Coutaz, prof. UJF) Man Machine Interaction Engineering (Ingénierie de l’Interaction Homme-Machine) MRIM (M.F. Bruandet, prof. UJF) Multimedia Information Retrieval (Modélisation et Recherche d’Informations Multimédia)

People 1/1/2002 Regular Staff73 (a) Profs + Researchers29 (b) Engineers + Admin.11 (c) PhD + Post-docs26 (d) Under contracts 7 (e) Masters and others24 (g) Invited10 Total 107

Technological transfert MultiCom : Usability platform for the design and the evaluation of interactive systems Start-up creations 1998 : NOVADIS, ergonomics, cognitive engineering 1999 : ARLYS (MENRT, CARMI, Microsoft, Axe et Cible), e- learning, education software development 2000 : DMAILER, indexing and processing

MultiCom Technological Platform : To study human factors in situated tasks To design and evaluate innovative and interactive services (or systems) by, Real usage experimentation Simulation (Woz for instance) Partnership Linguists, Sociologists, Ergonomists

Relationship and contracts MultiCom 6 industrial : (Schneider, France-Télécom, Leti-CEA, STM), 2 european : SIRLAN IST 5th PCRD, ADAMOS, Finlandia 2 regional : SHIVA RRA 99 et Thématique 2000 Research 6 industrial : (Thalès, France-Télécom, EDF, Alcatel) in RNRT, RNTL, CTI CNET, DGA 2 international : C-STAR, UNL (17 countries) 5 european : NESPOLE, TACIT, TIPS, ARIADNE, GLOSS, CAMELEON, FAME 3 research networks : CNRS (french networks) 2 regional 5 local (C-STAR, RICOM, RESIDE-HIS, EPIC, e-Xpérience)

Multi-team projects C-STAR/NESPOLE! : Face to face speech translation (GEOD, GETA) RICOM : Multimodal interaction for Multimedia information retrieval (GEOD, IIHM, MRIM) Smart Rooms : disable people monitoring, home services… (GEOD, IIHM) FORMID : e-learning, virtual teaching (ARCADE, IIHM)

C-STAR : Consortium for Speech Translation Advanced Research (Japan, Corea, USA, Germany, Italy, France) NESPOLE! (IST project) : NEgotiating through SPOken Language in E-commerce

Projet C-STAR Speech translation Applications to : tourism, hot line, e-commerce

2 ways possible source Speech recognition Translation Speech synthesis target speaking listening texte IF IF coding Generation

Interchange Format Meaning representation Based on –Speech acts What to want or to do –Concepts What to say –Arguments Discourse predicates give-information +availability+room room-type=(single ; double)

The 22th july 1999 at Grenoble

RICOM : Information Retrieval: Interactive, Collaborative and Multimodale HALPIN : Man-Machine Dialogue CoVitesse : Information Representation

I found 1206 documents on the topic artificial intelligence. Do you want refine your research ? What have you on... e, artificial intelligence ? HALPIN Man-Machine Dialogue : speech recognition + dialogue management + information retrieval + speech synthesis

History of dialogue Input Outputs Documents Speech Buttons Speech Recognition

Collaborative surfing Trace of navigation Collected documents Session control Document annotation

RESIDE-HIS : Smart Home for disable people COMEDI : Informal Communication

RESIDE-HIS

Telemonitoring Réseau Téléphonique Public Modem H Home bus Physiological and biological sensors Microphones Monitoring on : Events Alarms Statistics

Plan HIS

CoMedi : Informal communication Feed-back, privacy control Awareness, Focus Acces control Feed-back control

Example of filters Poster Curtain EigenSpace

EigenSpace filter: Main Component Analysis Image baseEigen Space Image captured Image to send Eigen vectorArgMin(distance)

FORMID : FORmation Interactive à Distance Magic Board : Collaborative Space and augmented reality

FORMID : Toolkit for teaching by example I learn what I do. FORMID GeneSimu Oasis

Magic Board

Platform for design and evaluation of interactive systems Platform for design and evaluation of interactive systems

Development ObservationsTask analysis Design Software design Lifecyle

Objectives To anticipate the usage and to predict the usability To design and to evaluate interactive systems along the lifecycle Participatory design method And simulation techniques (as Moz) for the experimentation and the test using

A good starting point : enrich the first idea Taking into account all the criteria about (a) sociological, (b) ergonomical, (c) economical, (d) technological points of view Idea Functions Specifications Scenario Observation Diagnostic Validation Mockup Evaluation

Sociology… Practice analysis Usage meaning By questionnaire By interviews By recording practices Applying sociological criteria  Recommendations for the design

Ergonomics... Simulation, Woz, Scenarios… Test human perceptive Test human behavior (after capture) under definite and limited scenarios Task and activity analysis  Recommendations for the design

The first cycle ends… Functionalities definition Design constraints Criteria Evaluation definition Specifications  Then development can start along a second iterative cycle

The hardware platform User Devices for observation Wizard 1 Wizard 2 Control video Multimedia recording Interactive devices

Conclusion… CLIPS-IMAG is… A research lab on interactive systems Open on the technological transfert Focused on user centered design methods