Experiments on Building Language Resources for Multi-Modal Dialogue Systems Goals identification of a methodology for adapting linguistic resources for human-machine dialogue systems, without training corpora; Multilinguality supported, uniform linguistic coverage of speech interpretation modules case study: synchronising parser’s and SR interface for French for MIAMM project L.Romary*, A.Todirascu**, D.Langlois* *LORIA, Nancy, France **Université de Technologie de Troyes, France Speech Recognition’s Experiments system: ESPERE (small vocabularies) protocol: 88 sentences recorded, 4 speakers, 2 men and 2 females, no OOV, all sentences in language several methods for estimating P(w|v) 1.Frequencies in training corpus: WER = 3.8, SD = Uniform probabilities for bigrams in training corpus: WER = 4.0, SD = 0.4 True frequencies are not useful 3.All bigrams are possible (not null probabilities, back-off with ): WER >> 4.0 Constraint bigrams from grammar are necessary 4.Training corpus (A) is adaptation corpus for a general newspaper one (B): Linear combination  performance are less good than when A is used alone Parser’s Experiments iteration updating process after interaction with other modules adding new lexical entries local grammars generated by a meta-grammar preference for substitution Domain-specific syntactic components (elliptic phrases, navigation verbs, noun groups) mapping lexical entries to domain ontology transforming derivation trees into MMIL representations The Multimedia Information Access Using Multiple Modalities (MIAMM) Project ( ) A prototype of a human-machine dialogue interface regrouping various interaction modalities: speech, haptics, graphics; Case study: searching music into an existing database using all the modalities Multilinguality supported: English, French, German difficulties: multi-modal training corpora not available, Information flow: several models for the same language (one for speech, one for parsing : how to cover the same language), changing specifications during the project Adapting parser’s resources TAG widely accepted formalism for syntactic parsing XML standard for grammars (TAGML) and for semantic representations (MMIL) existing resources for English, French for free texts Speech recognizer’s language model Statistical language model 400 words vocabulary Huge number of possible sentences Training corpus: generated with context-free grammar. Two steps: Bigram model using classes (ex: P(DECADES|the) Uniform distribution of words into classes: P(90’s|DECADES) User Scenarios useful in the context of lacking real human-machine interactions; designed to obtain homogeneous linguistic coverage, for all the languages: several styles (or registers - familiar, elaborated); specific phrases (politeness phrases, time intervals - "from the sixties"); various syntactic components (passive constructions, relative clauses, questions and ellipses); dates or names developers worked independently building exhaustive user scenarios Context-free grammar covers the linguistic phenomena from the user scenario, for every language Technical vocabulary covers the linguistic phenomena from the user scenario, for every language Developping the language resources