1. GoDiS AI-course, Chalmers April 22, 2002 Staffan Larsson

2. this lecture introduction to GoDiS –information state –moves, plans, actions –accommodation –optimistic grounding and acceptance information-oriented vs. action oriented dialogue command dialogue & menus instructional dialogue & text

3. originally: information-seeking dialogue –extended to action-oriented dialogue information state based Ginzburg’s notion of Questions Under Discussion (QUD) dialogue plans to drive dialogue simpler than general reasoning and planning more versatile than frame-filling and finite automata GoDiS features

4. GoDiS & TrindiKit TrindiKit QUD-based dialogue theory (IS, rules,...) domain & language resources generic GoDiS system domain-specific GoDiS system information state approach

5. input inter- pret TIS IS proper 5 module interface variables 3 resource interface variables DATABASE LEXICON DOMAIN data- base control updateselect gene- rate output lexicon domain knowledge DME

6. basic GoDIS infostate type PRIVATE : PLAN : stackset( Action ) AGENDA : stack( Action ) SHARED : BEL : set( Prop ) TMP : (same type as SHARED) COM : set( Prop ) QUD : stack( Question ) LU: SPEAKER: Speaker MOVES: assocSet( Move ) + module interface variables INPUT : String LATEST-MOVES: Set(Move) LATEST-SPEAKER: Speaker NEXT-MOVES: Set(Move) OUTPUT: String + resource interface variables LEXICON : Lexicon DOMAIN : Domain DATABASE : Database

7. PRIVATE =PLAN = AGENDA = { findout(?return) } SHARED = findout(? x.month(x)) findout(? x.class(x)) respond(? x.price(x)) COM = dest-city(paris) transport(plane) task(get_price_info) QUD = LU = SPEAKER = sys MOVES = {? x.depart-city(x)) - false } BEL = { } TMP = (same structure as SHARED) Sample GoDiS information state

8. Activity-based moves in GoDiS Moves are determined by the relation of the content to the domain –utterance U is an answer if the content A of U is a relevant answer to a question Q in the domain –moves are not necessarily speech acts! GoDiS basic dialogue moves –ask(Q), Q:question –answer(A), A:answer (proposition or fragment) –inform(P), P:proposition –request-repetition –greet, quit

9. integrateAnswer Before an answer can be integrated by the system, it must be matched to a question on QUD pre: eff: in($ SHARED.LM, answer(usr, A)) fst($ SHARED.QUD, Q) $ DOMAIN :relevant_answer(Q, A) pop( SHARED.QUD ) DOMAIN: reduce(Q, A, P) add( SHARED.COM, P) Sample update rule

10. Dialogue plans domain-specific partial specifications of how the system should act in a dialogue –specifies default behaviour –may be overridden by user initiative operational semantics of plan constructs –determined by information state update rules

11. Dialogue plan constructs action A exec(S) –S is a task/subplan sequence if P then C –P is a proposition and C a construct –If P is (believed to be) true, execute C case(,...,, C n )

12. Actions findout(Q): find the answer to Q raise(Q): raise Q respond(Q): provide answer to Q if there is one in the private beliefs inform(P): inform of P (Q is a question, P a proposition)

13. Questions y/n-question: ?P –”Do you want a return ticket?” –”Do you want to call NN?” wh-question: ?x.P(x) –“What city do you want to go to?” –”Who do you want to search for?” alt-question: {?P 1, ?P 2,..., ?P n } –”Do you want to search, add to, erase from or change the phonebook?”

14. Typical human-computer dialog S: Hello, how can I help you? U: I want price information please S: Where do you want to go? U: Paris S: How do you want to travel? U: A flight please S: When do you want to travel U: April S: what class did you have in mind? … S: The price is $123

15. Dialogue plans for information-seeking dialogue Find out how user wants to travel Find out where user wants to go to Find out where user wants to travel from Find out when user wants to travel … Lookup database Tell user the price

16. Dialogue plan, formal representation findout(?x.transport(x)) findout(?x.dest-city(x)) findout(?x.depart-city(x)) findout(?return) … consultDB(?x.price(x)) respond(?x.price(x))

17. Typical human-human dialogue S(alesman), C(ustomer) S: hi C: flights to paris S: when do you want to travel? C: april, as cheap as possible...

18. Accommodation Lewis (1979): –If someone says something at t which requires X to be in the conversational scoreboard, and X is not in the scoreboard at t, then (under certain conditions) X will become part of the scoreboard at t A: My cat is hungry –B accommodates “A has a cat” Has been applied to referents and propositions, as parts of the conversational scoreboard / information state

19. Question accommodation If questions are part of the information state, they too can be accommodated If –the latest move was an answer, and –there is an action in the plan to ask a matching question, –then, push that question on QUD Requires that the number of possible matching questions is not too large –or can be narrowed down by asking clarification question

20. Update rule for question accommodation QuAcc pre: eff: in($ SHARED.LM, answer(usr, A)) in($ PRIVATE.PLAN, findout(Q)) $ DOMAIN :relevant_answer(Q, A) delete( PRIVATE.PLAN, findout(Q)) push( SHARED.QUD, Q)

21. question accommodation… PRIVATE : PLAN : stackset( Action ) AGENDA : stack( Action ) SHARED : BEL : set( Prop ) TMP : (same type as SHARED) COM : set( Prop ) QUD : stack( Question ) LU: SPEAKER: Speaker MOVES: assocSet( Move )

22. Task accommodation In some cases, the system may not even know what task the user wants the system to perform If latest move was an answer, and there is currently no plan, –find (in the domain resource) a task and corresponding plan containing a matching question; –accommodate the task and load the appropriate plan Similar to plan recognition, but no dynamic plan reconstruction or assumption that the full plan is shared

23. Update rule for task accommodation taskAcc - An answer move matches a task if the plan contains a question matching the answer - More complex version generates clarification question if the number of plans > 1 pre: eff: in($ SHARED.LM, Move) $ DOMAIN :: match_task(Move, Task, Plan) add( SHARED.BEL, task(Task)) set( PRIVATE.PLAN, Plan)

24. task accommodation… DATABASE= LEXICON = DOMAIN= travel travel_english travel database travel lexicon travel english domain travel PRIVATE : PLAN : stackset( Action ) AGENDA : stack( Action ) SHARED : BEL : set( Prop ) TMP : (same type as SHARED) COM : set( Prop ) QUD : stack( Question ) LU: SPEAKER: Speaker MOVES: assocSet( Move )

25. Question and task accommodation in information-seeking dialogue S: hi U: flights to paris system finds plan containing appropriate questions, and loads it into the plan field in the information state system accommodates questions: how does user want to travel + where does user want to go, and integrates the answers “flight” and “to paris” system proceeds to next question on plan S: when do you want to travel?

26. An optimistic approach to grounding DPs assume their utterances are understood –If A asks a question with content Q, A will update SHARED.LM with ask(A,Q) If addressee indicates that the previous utterance was not understood (reqRep), backtrack –uses the PRIVATE.TMP field, which contains a copy of the SHARED field from before the latest utterance was made The alternative is a pessimistic approach –If A asks Q, A will not update SHARED.LM until B indicates understanding (implicitly or explicitly)

27. Update rule for optimistic grounding assumeSysMovesGrounded pre: eff: $LATEST-SPEAKER == sys SHARED.LU.SPEAKER := sys clear( SHARED.LU.MOVES ) forall_do( in( LATEST-MOVES, Move), add( /SHARED/LU/MOVES, Move, false) )

28. input inter- pret updateselect gene- rate output PRIVATE : PLAN : stackset( Action ) AGENDA : stack( Action ) SHARED : BEL : set( Prop ) TMP : (same type as SHARED) COM : set( Prop ) QUD : stack( Question ) LU: SPEAKER: Speaker MOVES: assocSet( Move ) INPUT : String LATEST-MOVES: Set(Move) LATEST-SPEAKER: Speaker NEXT-MOVES: Set(Move) OUTPUT: String optimistic grounding

29. Optimistic approach to acceptance DPs assume their utterances are accepted (and integrated into SHARED) –If ask(A,Q) is in SHARED.LM, A will make Q topmost on SHARED.QUD If addresse indicates rejection, backtrack –using the PRIVATE.TMP field No need to indicate acceptance explicitly; it is assumed The alternative is a pessimistic approach –If A asks a question with content Q, A will wait for an acceptance (implicit or explicit) before putting Q on top of QUD

30. optimistic acceptance PRIVATE : PLAN : stackset( Action ) AGENDA : stack( Action ) SHARED : BEL : set( Prop ) TMP : (same type as SHARED) COM : set( Prop ) QUD : stack( Question ) LU: SPEAKER: Speaker MOVES: assocSet( Move )

31. Adapted for travel agency and autoroute domains, as well as acting as interface to handheld computer or mobile phone Question and task accommodation to enable mixed initiative Simple “optimistic” grounding strategy Focus intonation based on information state contents Has been extended to handle instructional dialogue (IMDiS) Also being extended to handle negotiative dialogue (SIRIDUS) GoDiS features

32. Action- and information- oriented dialogue

33. Information- vs. action- oriented dialogue Information oriented dialogue (IOD) has the primary goal of exchanging information, regardless of how this information will be used in future actions Action oriented dialogue (AOD) has the primary goal of one or several participants performing or being obliged to perform an action (or plan, i.e. a complex action)

34. Information-oriented dialogue Information-seeking dialogue (ISD): one DP asks the questions, the other answers them Information-exchange (information oriented) dialogue: both DPs ask questions and provide answers IOD can be seen as a sequence of infoseeking dialogues, possibly with embedded subdialogues shared information is stored in shared.com

35. Action-oriented dialogue In simple AOD, only one participant becomes obliged/comitted to some action or plan Actions can either be performed ”online” while the dialogue is happening, or they may be stored as a plan to be performed after the dialogue (”offline”) Requires a store of obliged actions in the IS ( SHARED.ACTIONS ) Two subtypes; –instructional dialogue –command dialogue

36. GoDIS AOD-extended infostate type PRIVATE : PLAN : stackset( Action ) AGENDA : stack( Action ) SHARED : BEL : set( Prop ) TMP : (same type as SHARED) COM : set( Prop ) QUD : stack( Question ) ACTIONS : stack( Action ) LU:…

37. Menu-based natural command dialogue (D’Homme) system carries out actions conversion menu 2 dialogue –menu-based interfaces can be automatically or semi-automatically converted into dialogue plans flexible dialogue –dialogue plans are used in a way that enables more flexible interaction than the original menu-based interface Larsson, Cooper, Ericsson (2001): menu2dialog. IJCAI workshop paper.

38. Menus vs. dialogue: Drawbacks of menus User must follow predesigned menu structure this structure may be complicated and difficult to learn menus allow only one activity at a time

39. Menu vs. dialogue: Advantages of flexible dialogue provide information in any order provide information without pre- specifying the task not necessary to learn menu structure doing several things at once

40. Menu components multiple-choice list –the user selects one of several prespecified choices dialogue window –user types requested information tickbox –user ticks box (or not) system message –information from system; user clicks ”OK”

41. Sample menu from a Nokia 3210 mobile phone search phonebook –dialogue window to type in name Name: _______ –message showing the number Kalle 031 12 34 56 –tickbox: whether to call this person Call? –if a call is made, a message: Calling 031 12 34 56

42. Converting menus to plans MenuPlan multi-choice list L 1, L 2,..., L n findout({?L 1,?L 2,...,?L n }); case(,..., ) where T 1 is the task corresponding to L 1 tick-box +/-Pfindout(?P) dialogue window p=___ findout(?x.p(x)) pop-up message M inform(M)

43. Converting menus to plans, cont’d in addition, we need –if-then statements to check conditions and pick out propositions –device actions, e.g. “call up number N” “add name N with number M to phonebook” … these are not currently assumed to be part of the menu specification

44. the menu again search phonebook –dialogue window to type in name Name: _______ –message showing the number Kalle 031 12 34 56 –tickbox: whether to call this person Call? –if a call is made, a message: Calling 031 12 34 56

45. Corresponding plan findout(?x.name(x)); lookup_database(?x.number(x)); if number(N) then respond(?x.number(x) ), findout( ?call ), if call and name(M) then inform(calling_name(M))))

46. Another plan: change ringtone findout(?x.name(x)); findout(?x.ringtone(x)); if name(N) and ringtone(R) then save-ringtone( N, R ), inform(ringtone-changed);

47. Example 1: standard menu navigation U:Go to the phonebook S: Phonebook. Do you want to search, add, or remove an entry, or assign a ringing tone? U: Assign ringing tone S: What name? U: John S: Which ringtone? U: Bossanova S: The ringtone for John is now Bossanova.

48. Example 2: question accommodation U: Assign ringtone S: What name? U: Bossanova for John accommodate ”which ringtone?” S: The ringing tone for John is now Bossanova.

49. Example 3: task accommodation U: Bossanova for John accommodate task: assign ringtone accommodate ”which ringtone?” accommodate ”what name?” S: The ringing tone for John is now Bossanova.

50. Example 4: task clarification U: 5 o’clock –matches two tasks; ask for clarification S: Do you want to set the clock or the alarm clock? U: alarm clock –accommodate ”what time?” S: the alarm clock is now set to 5

51. Example 5 (video): task and question accommodation U: record from 19:30 to 20:00 today –accommodate task: add program –accommodate ”what starting time?” –accommodate ”what end time?” –accommodate ”what day?” S: What channel? U: Seven S: OK, I will record channel 7 from 19:30 to 20:00 today.

52. Example 6: doing several things at once U: Search the phonebook for Jim S: The number is 031 997321 U: Assign ringtone “Attraction” name(jim) is already a shared proposition S: The ringtone for Jim is now Attraction. Do you want to make a call? U: No Note: proposition name(jim) is used by both plans

53. Instructional dialogue user carries out actions goals of this research: –explore the relation between monologue and dialogue in instructional discourse –investigate possibilities of using domain task plans to generate dialogue plans providing various levels of interactivity –adapt GoDiS to handle instructional dialogue (and monologue) NB. We are not dealing with educational (a.k.a. instructional) dialogue TrindiBook chapter 9; Larsson & Zaenen (2000) SigDial paper

54. Example from a Xerox manual Reinstalling the print head Caution: make sure that the green carriage lock lever is STILL moved all the way forward before you reinstall the print head 1. Line up the hole in the print head with the green post on the printer carriage Lower the print head down gently into position 2. Gently push the green carriage lock lever up until it snaps into place This secures the print head 3. Close the tops cover and re-attach the scanner 4. Press and release the yellow LED button The printer will prepare the cartridge for printing Note: if the carriage does not move from the center position after you press the carriage change button, remove and reinstall the print head

55. dialogue mode 8 moves (6 standard GoDiS moves, + Instruct and Confirm) Confirmations are integrated by assuming that the current top- most action of the shared.actions has been performed rule integrateUsrConfirm (slightly simplified) pre: eff: val( SHARED.ACTIONS, A ) pop( SHARED.ACTIONS ) add( SHARED.COM, done(A)) in( SHARED.LU.MOVES, confirm )

56. Sample information state PRIVATE =PLAN = AGENDA = { findout(?moved(carriage)) } SHARED = if_then( not moved( carriage ), [ remove(print_head), reinstall(print_head) ] ) COM = moved_forward(carriage_lock) done(secure(print_head)) done(close(top_cover)) done(reattach(scanner)) QUD = LU.MOVES { instruct(sys, press_and_release(yellow_button) } BEL = { } TMP = (same structure as SHARED) ACTIONS=

57. Example from a Xerox manual (repeated) Reinstalling the print head Caution: make sure that the green carriage lock lever is STILL moved all the way forward before you reinstall the print head 1. Line up the hole in the print head with the green post on the printer carriage Lower the print head down gently into position 2. Gently push the green carriage lock lever up until it snaps into place This secures the print head 3. Close the tops cover and re-attach the scanner 4. Press and release the yellow LED button The printer will prepare the cartridge for printing Note: if the carriage does not move from the center position after you press the carriage change button, remove and reinstall the print head

58. dialogue behavior with minimal interactivity yes/no questions confirm (”ok”) requestRepeat (”what?”) grounding and avoidance of irrelevant information example: S: Press and release the yellow button U: ok S: Has the carriage moved from the center position? U: yes S: the print head is now installed U: what? S: the print head is now installed

59. increased interactivity skipping subtask instructions –S: Put the print head in place –U: Ok, done, what now? –S: Close the top cover requesting subtask instructions; popping out of subdialogue –S: Put the print head in place –U: how? –S: Line up the hole in the print head with the green post on the printer carriage –U: done, I now remember the rest, the print head is secured

60. conclusions GoDiS is based on QUD and dialogue plans –QUD has independent motivation in dialogue semantics & ellipsis resolution (Ginzburg) –no planning needed in many domains; use ready- made plans but allow user initiative QUD accommodation –theoretically motivated –enables mixed initiative Demonstrates reusability –Originally implemented for information-oriented dialogue –but straightforward to extend to handle simple action-oriented dialogue Demonstrate exploratory use of TrindiKit –explore relation between instructional text and dialogue