1. menu2dialog Staffan Larsson, Robin Cooper, Stina Ericsson Department of linguistics Göteborgs Universitet

2. Background Current projects –TRINDI: task-oriented dialogue –SIRIDUS: explore action-oriented dialogue (and more) –D’Homme: Talking to applications in the home Many applications and services have menu- based interfaces –mobile phone –video –computer –automatic cinema ticket booking via phone But menus have certain disadvantages Dialogue systems can provide more user- friendly interfaces

3. the basic idea of this paper: conversion menu 2 dialogue –menu-based interfaces can be automatically or semi-automatically converted into dialogue plans flexible dialogue –dialogue plans are used in a way that enables more flexible interaction than the original menu-based interface modularity, plug&play –we isolate general, domain independent dialogue knowledge from domain- dependent dialogue plans

4. Overview Menu vs. dialogue GoDiS: a dialogue system Menu components Dialogue plans Converting menus to dialogue plans Examples

5. Menus vs. dialogue: Drawbacks of menus User must follow predesigned menu structure this structure may be complicated and difficult to learn menus allow only one activity at a time

6. Menu vs. dialogue: Advantages of flexible dialogue provide information in any order provide information without pre- specifying the task not necessary to learn menu structure doing several things at once

7. GoDiS: a dialogue system Originally developed withing the european TRINDI project implemented using the information state approach, using the TrindiKit based on a theory about dialogue based on ”Questions Under Discussion” (Ginzburg) and dialogue plans presented at the previous workshop at IJCAI-99

8. TRINDIKIT dialogue theory (IS, rules, moves etc) domain knowledge (resources) domain-specific system Building a system using TrindiKit domain-independent DME software engineering (basic types, control flow)

9. input inter- pret Information State lexicon control updateselect gene- rate output device interface domain DEVICE

10. PRIVATE =PLAN = AGENDA = { findout(?return) } SHARED = findout(? x.month(x)) findout(? x.class(x)) respond(? x.price(x)) COM = dest(paris) transport(plane) task(get_price_info) QUD = LM = { ask(sys, x.origin(x)) } BEL = { } TMP = (same structure as SHARED) Sample GoDiS information state (travel agency domain)

11. integrateAnswer Before an answer can be integrated by the system, it must be matched to a question on QUD pre: eff: in( SHARED.LM, answer(usr, A)) fst( SHARED.QUD, Q) relevant_answer(Q, A) pop( SHARED.QUD ) reduce(Q, A, P) add( SHARED.COM, P) Sample update rule

12. Typical human-computer dialog in a travel agency S: Hello, how can I help you? U: I want price information please S: Where do you want to go? U: Paris S: How do you want to travel? U: A flight please S: When do you want to travel U: April S: what class did you have in mind? … S: The price is $123

13. Typical human-human dialogue S(alesman), C(ustomer) S: hi C: flights to Paris S: when do you want to travel? C: April, as cheap as possible...

14. Accommodation Lewis (1979): If someone says something at t which requires X to be in the conversational scoreboard, and X is not in the scoreboard at t, then (under certain conditions) X will become part of the scoreboard at t A: My cat is hungry –B accommodates “A has a cat” Has been applied to referents and propositions, as parts of the conversational scoreboard / information state

15. Question accommodation If questions are part of the information state, they too can be accommodated If the latest move was an answer, and there is an action in the plan to ask a matching question, put that question on QUD Requires that the number of possible matching questions is not too large (or can be narrowed down by asking clarification question)

16. Update rule for question accommodation QuAcc pre: eff: in( SHARED.LM, answer(usr, A)) in( PRIVATE.PLAN, findout(Q)) relevant_answer(Q, A) push( SHARED.QUD, Q)

17. Task accommodation In some cases, the system may not even know what task the user wants the system to perform If latest move was an answer, and there is currently no plan, –find (in the domain resource) a task and corresponding plan containing a matching question; –accommodate the task and load the appropriate plan Similar to plan recognition, but no dynamic plan reconstruction or assumption that the full plan is shared

18. Update rule for task accommodation taskAcc - An answer move matches a task if the plan contains a question matching the answer - More complex version generates clarification question if the number of plans > 1 pre: eff: in( SHARED.LM, Move) domain :: match_task(Move, Task, Plan) add( SHARED.BEL, task(Task)) set( PRIVATE.PLAN, Plan)

19. Flexible dialogue with GoDiS in the travel agency domain Question accommodation –S: when do you want to travel? –U: in April, as cheap as possible Task accommodation –S: hello –U: flights to Paris

20. Work on GoDiS dialogue types handled by GoDiS –information seeking –instructional (system instructs user) Currently being extended to handle –action-oriented (user instructs system) –multiple simultaneous plans –collaborative negotiation GoDiS has been adapted to several domains, including –travel agency (TRINDI) –mobile phone interface (course project) –interface to intellegent house (D’Homme) –cinema ticket booking (MA thesis) –internet search (MA thesis) –telephone receptionist (SIRIDUS)

21. Menu components multiple-choice list –the user selects one of several prespecified choices dialogue window –user types requested information tickbox –user ticks box (or not) system message –information from system; user clicks ”OK”

22. Sample menu from a Nokia 3210 mobile phone search phonebook –dialogue window to type in name Name: _______ –message showing the number Kalle 031 12 34 56 –tickbox: whether to call this person Call? –if a call is made, a message: Calling 031 12 34 56

23. Dialogue plans domain-specific partial specifications of how the system should act in a dialogue –specifies default behaviour –may be overridden by user initiative operational semantics of plan constructs –determined by information state update rules

24. Dialogue plan constructs action A exec(S) –S is a task/subplan sequence if P then C –P is a proposition and C a construct –If P is (believed to be) true, execute C case(,...,, C n )

25. Actions findout(Q): find the answer to Q raise(Q): raise Q respond(Q): provide answer to Q if there is one in the private beliefs inform(P): inform of P (Q is a question, P a proposition)

26. Questions y/n-question: ?P –” Do you want to call NN?” wh-question: ?x.P(x) –”Who do you want to search for?” alt-question: {?P 1, ?P 2,..., ?P n } –”Do you want to search, add to, erase from or change the phonebook?”

27. Converting menus to plans MenuPlan multi-choice list L 1, L 2,..., L n findout({?L 1,?L 2,...,?L n }); case(,..., ) where T 1 is the task corresponding to L 1 tick-box +/-Pfindout(?P) dialogue window p=___ findout(?x.p(x)) pop-up message M inform(M)

28. Converting menus to plans, cont’d in addition, we need –if-then statements to check conditions and pick out propositions –device actions, e.g. “call up number N” “add name N with number M to phonebook” … these are not currently assumed to be part of the menu specification

29. the menu again search phonebook –dialogue window to type in name Name: _______ –message showing the number Kalle 031 12 34 56 –tickbox: whether to call this person Call? –if a call is made, a message: Calling 031 12 34 56

30. Corresponding plan findout(?x.name(x)); lookup_database(?x.number(x)); if number(N) then respond(?x.number(x) ), findout( ?call ), if call and name(M) then inform(calling_name(M))))

31. Another plan: change ringtone findout(?x.name(x)); findout(?x.ringtone(x)); if name(N) and ringtone(R) then save-ringtone( N, R ), inform(ringtone-changed);

32. Example 1: standard menu navigation U:Go to the phonebook S: Phonebook. Do you want to search, add, or remove an entry, or assign a ringing tone? U: Assign ringing tone S: What name? U: John S: Which ringtone? U: Bossanova S: The ringtone for John is now Bossanova.

33. Example 2: question accommodation U: Assign ringtone S: What name? U: Bossanova for John accommodate ”which ringtone?” S: The ringing tone for John is now Bossanova.

34. Example 3: task accommodation U: Bossanova for John accommodate task: assign ringtone accommodate ”which ringtone?” accommodate ”what name?” S: The ringing tone for John is now Bossanova.

35. Example 4: task clarification U: 5 o’clock –matches two tasks; ask for clarification S: Do you want to set the clock or the alarm clock? U: alarm clock –accommodate ”what time?” S: the alarm clock is now set to 5

36. Example 5 (video): task and question accommodation U: record from 19:30 to 20:00 today –accommodate task: add program –accommodate ”what starting time?” –accommodate ”what end time?” –accommodate ”what day?” S: What channel? U: Seven S: OK, I will record channel 7 from 19:30 to 20:00 today.

37. Example 6: doing several things at once U: Search the phonebook for Jim S: The number is 031 997321 U: Assign ringtone “Attraction” name(jim) is already a shared proposition S: The ringtone for Jim is now Attraction. Do you want to make a call? U: No Note: proposition name(jim) is used by both plans

38. end of talk

39. Menus in VoiceXML Why is VoiceXML interesting? –becoming industry standard –supports plan constructs similar to GoDiS –multiple active grammars allow behaviour reminiscent of question and task accommodation VoiceXML takes form-filling dialogue management to its limit

40. GoDiS vs. VoiceXML VoiceXML: if input matches several fields, the first is chosen –GoDiS can ask clarification question VoiceXML: user initiated subdialogues cannot share information with main dialog, or other subdialogues –the default in GoDiS is that all information is shared between subdialogues unclear how to implement several simultaneously active plans

41. GoDiS vs. VoiceXML information state is global and keeps plan separate from accumulated propositions –VoiceXML based on forms, which can be seen as local information states VoiceXML mixes dialogue knowledge, domain knowledge, and language knowledge in a single specification –GoDiS keeps them separate, enabling easier reconfiguration and plug-and-play –when implement a new menu-driven system, there is no need to reimplement general dialogue strategies

42. GoDiS vs. VoiceXML Open question whether VoiceXML can extend to more complex dialogue, e.g. negotiation –but the information state approach is ideal for complex dialogues –GoDiS is currently being extended to handle collaborative negotiation Can GoDiS be combined with VoiceXML? –have GoDiS use VoiceXML specifications, with some additions