Equal-party Conversation System for Language Learning Chih-yu Chao (advisor: Stephanie Seneff) April 14 th, 2006 Dialogs on Dialogs Reading Group

April 14th, Overview Motivation Approach Implementation Future Work References

April 14th, Motivation (1 of 3) Computer-Assisted Language Learning –Reading & writing –Listening (few) –Speaking (fewer) –Conversation (none?!)

April 14th, Motivation (2 of 3) Human tutors vs. computers –Confidence, stress –Making mistakes –Patience –Accessibility –Feedback

April 14th, Motivation (3 of 3) The proposed system –Domain-specific conversations –Sentence formation –Speaking –Listening –Less stressful environment –Game-like interaction process Remember the new vocabulary/sentence patterns more efficiently

April 14th, Approach (1 of 5) Domain –Activity scheduling (cf. database-accessing) Part of the lesson plan –Not simply form-filling Activity proposal Confirmation of likes/dislikes Activity negotiation Schedule checking Multiple turns of these actions

April 14th, Approach (2 of 5) Taking the initiative –Forming different types of sentences by playing a different role in each dialogue

April 14th, Approach (3 of 5) Languages of interest –Mandarin Chinese  English The game –Persona A schedule from yesterday to day after tomorrow Preference of activities –The virtual buddy also has a persona represented internally –Respond and/or negotiate based on the given persona until an agreement is reached –New persona

April 14th, :setup ( {c like :min_count 3 :max_count 5 :choices ( "swimming" "basketball" "television" "movies" "football" "baseball" "tennis" "dancing" "video games" "shopping" ) :unique 1 :pattern ":choices" } {c dislike :min_count 3 :max_count 5 :choices ( "swimming" "basketball" "television" "movies" "football" "baseball" "tennis" "dancing" "video games" "shopping" ) :unique 1 :rule_out "like" :pattern ":choices" } {c schedule :min_count 3 :max_count 7 :choices ( "swimming" "basketball" "television" "movies" "football" "baseball" "tennis" "dancing" "shopping" "studying" "video games" ) :ordinates ( ":day" ":time" ) :pattern ":choices" :day ( "yesterday" "today" "tomorrow" "day after tomorrow") :time ( "morning" "afternoon" "evening" ) :rule_out "dislike" } ) :features {c features :day ( "yesterday" "today" "tomorrow" "day after tomorrow") :time ( {c morning :start_hour 9 :end_hour 12 } {c afternoon :start_hour 13 :end_hour 17 } {c evening :start_hour 19 :end_hour 22 } )

April 14th, A: "what are you doing day after tomorrow afternoon?" B: "I am going to study the day after tomorrow in the afternoon" A: "are you free day after tomorrow morning?" B: "yes I am going to do nothing the day after tomorrow in the morning" A: "would you like to play tennis with me the day after tomorrow in the morning" B: "yes I do like to play tennis" A: "okay let's get together the day after tomorrow in the morning to play tennis" B: "yes that would be great" ====================== New Persona ====================== A: "what are you doing tomorrow evening?" B: "I am going to do nothing tomorrow evening" A: "would you like to play football with me tomorrow evening" B: “to play football is not bad" A: "okay let's get together tomorrow evening to play football" B: “okay to play football is not bad"

April 14th, Approach (5 of 5) Other features –Get help by speaking in the native language 2 recognizers, threshold –Score –Keeps track of the difficulties Same material (vocabulary or sentence pattern) will be reviewed until the user succeeds

April 14th, Implementation (1 of 3) Interlingua-based approach –Natural language input  language- independent semantic frame –TINA: syntactic and semantic analyses –Genesis: semantic frames  well-formed surface strings in the target language Based on the language-dependent generation rules (Pluto)

April 14th, {c verify :auxil “link” :topic {q pronoun :name “you” :number “pl” :domain “LanguageLesson” :sro “are you free day after tomorrow afternoon” :ilang “english” :olang “pinyin” :num_oreds 2 :pred {p attribute :topic “free” } :pred {p temporal :topic {q rel_date :name “day after tomorrow” } :num_preds 1 :pred {p daypart :topic {q daytime :name “afternoon” } } } }

April 14th, Implementation (3 of 3) 2 simulated users –Dialogues automatically generated by the system –Without eliciting input from human subjects –Acquire a large amount of data with very little cost –Good for different types of bootstrapping Coverage of language generation rules Accuracy of speech recognition Quality of speech synthesis

April 14th, Future Work Interface –Phone & web –Web only (with audio GUI) Peer-to-peer interaction mode –(Delayed) interaction between 2 learners –Target language only User study Curriculum design Evaluation –Software –Pedagogy

April 14th, References [1] V. Zue, S. Seneff, J. R. Glass, J. Polifroni, C. Pao, T. J. Hazen, and I. L. Hetherington. Jupiter: A telephone-based conversational interface for weather information. IEEE Trans. on Speech and Audio Processing, 8(1): , [2] S. Seneff and J. Polifroni. Dialogue Management in the Mercury Flight Reservation System. In Proceedings of ANLP-NAACL 2000, Satellite Workshop, pp. 1--6, Seattle, WA. [3] S. Seneff. TINA: A Natural Language System for Spoken Language APplications. In Computational Linguistics, 18(1):61--86, [4] L. Baptist and S. Seneff. GENESIS-II: A Versatile System for Language Generation in Conversational System Applications. In Proceedings of ICSLP, pp , Beijing, China, October [5] B. Cowan. PLUTO: A Preprocessor For Multilingual Spoken Language Generation. Master's thesis, MIT, Cambridge, MA, February 2004.

Questions?