1. A system for generating teaching initiatives in a computer-aided language learning dialogue Nanda Slabbers University of Twente Netherlands June 9, 2005 Nancy Alistair Knott University of Otago New Zealand

2. 2 Overview Introduction Existing work Goals –Formal goals –Substantive goal Te Kaitito –Authoring mode –Student mode Conclusion Questions

3. 3 Introduction A dialogue system has to create two different kinds of utterances: –Responses: not much content selection –Initiatives: content selection is very important It is common to analyse the task of natural language generation as a pipeline: Content Selection Sentence Planning Syntactic Realisation

4. 4 Existing Work Existing systems which focus on generating initiatives: –Used to accomplish a certain task. –The difference is that a language learning dialogue doesn’t have to look like a tutorial dialogue. Existing systems which focus on language learning: –Most initiatives come from the student. –In such systems it is hard to learn by adapting utterances made by the teacher.

5. 5 Goals of the Generated Initiatives Content selection is the key process in generating initiatives. The process of content selection is typically defined in relation to two goals: –Formal goal: creating a coherent text. –Substantive goal: achieving a certain effect on the hearer (in our case teaching the student a number of syntactic rules).

6. 6 Formal Goal - Possible Dialogue Acts Dialogue coherence cannot be defined at the level of dialogue acts. Therefore the system will only generate initiatives of the following dialogue acts: –Assertions –Questions: Genuine questions Teaching questions

7. 7 Formal Goal – Topic Continuity We prefer initiatives which maintain the current topic of the dialogue –Topic of utterance: set of individuals and predicates it introduces –Topic of dialogue: combination of topics of last utterances –Topic continuity: overlap between topic of dialogue and topic of possible initiative

8. 8 Formal Goal – Mix of Dialogue Acts First we want to decide which dialogue act will be taken: –Normally we decide this based on the topic continuity criterion. –If there are initiatives which score equally in the topic continuity criterion, we prefer those which belong to a dialogue act which has not been recently used.

9. 9 Formal Goal – Other Criteria After we have decided which dialogue act will be taken, we have to compare the possible initiatives belonging to that dialogue act. To achieve this we introduce three dialogue act specific criteria: –Assertions: strategy –Genuine questions: order of creation –Teaching questions: complexity

10. 10 Substantive Goal – Syntax Criterion Every utterance is parsed and syntactic representations are created. Using these representations a set of target syntactic rules can be created for each lesson. Each lesson teaches a different set of rules, beginning with simple rules and progressing to more complex rules. Using these rules initiatives which involve target rules can be preferred to other initiatives.

11. 11 Formal and Substantive Goals - Initiative Generation Algorithm Our algorithm for generating initiatives has four steps: –Identify a set of possible topics. –Generate a set of candidate initiatives. –Create a shortlist of initiatives by scoring and ranking them based on the criteria. –Assign a syntactic score and choose the initiative with the highest total score.

12. 12 Te Kaitito Te Kaitito is a bilingual dialogue system meant to teach the Maori language, but we will demonstrate the English version. The system supports two modes: –authoring mode: to author new lessons. –student mode: to practice the lesson and learn the target rules.

13. 13 Authoring Mode In authoring mode the user is assumed to be a teacher. The teacher can author a character by: –Telling facts about itself: “You are John” –Entering question generation rules: “How is a person?” At the end the facts and questions are stored in two separate knowledge bases. The set of target syntactic rules is created automatically.

14. 14 Authoring Mode – Example Dialogue A Your name is John TKOkay AYou are from Dunedin TKOkay AYou are fine TKOkay AHow is a person? TKOkay AWhat is a person’s name? TKOkay AWhere is a person from? TKOkay

15. 15 Student Mode – Start of the Lesson Start of the dialogue: –Empty common ground –Load knowledge bases and set of rules During the dialogue: –Dialogue consists of pairs of forward-looking and backward-looking dialogue acts. –At transition relevant points the initiative is passed to the student, but he or she can concede the initiative to the system.

16. 16 Student Mode – Example Dialogue S TKHow are you? SI am fine TKOkay SHow are you? TKI am fine S TKWhat is your name? SMy name is Ann TKOkay SWho are you TKI am John S TKWhere are you from? SI am from Dunedin TKOkay S TKI am also from Dunedin S TKLet’s see if you remember: Who am I? SYou are John TKCorrect, well done!

17. 17 Student mode – End of the Lesson The previous dialogue continues until the system has evidence that all target rules have been assimilated by the student: –Each target syntactic rule has an assimilation score that is updated after an utterance. –When the score reaches a certain boundary value the associated rule has been assimilated. –When all rules reach the boundary value, the lesson ends successfully. –When the student does not learn the rule, even when a correct application of the rule is shown, the lesson may end unsuccessfully.

18. 18 Conclusion Te Kaitito’s teaching dialogues are useful to practice conversational skills. The system only has a simple grammar, but since novice students also have a limited grammar, this is satisfactory. Adding initiatives is also beneficial: –When the student is lost, the dialogue can be continued by simply pressing the Enter key. –The initiatives provide models of well-formed sentences, so the student can modify these and try them out.

19. 19 Questions Are there any questions?