1. VoiceXML: Forms, Menus, Grammars, Form Interpretation Algorithm

2. Acknowledgements Prof. Mctear, Natural Language Processing, http://www.infj.ulst.ac.uk/nlp/index.html, University of Ulster. http://www.infj.ulst.ac.uk/nlp/index.html

3. Overview Forms Form items Form attributes Fields Field Attributes Prompts Menus Form Interpretation Algorithm

4. Forms Interpreting a form item (e.g. field) Select and play one or more prompts Collect user input Throw events (e.g. noinput) Interpret actions (procedural logic that executes when input item variables are assigned) Form attributes idname of the form e.g. reference to another form (validate is the name of a form) scopescope of the form’s grammars e.g.

5. Form Items Input Items input using speech or DTMF records user’s input as audio clip transfers user to another telephone number invokes a platform-specific object e.g. voice authentication program invokes another dialog Control Items procedural statements for prompting and computation controls initial interaction in a mixed initiative form

6. Example: A form for getting student information C: Say students, courses, or reports H: Students. C: What is the student’s name? H: John Scott. C: What is the student’s id? H: 654556. C: What information do you require? H: Address The system prompts the user for input in order to fill in the required fields. The user responds to the system’s prompts. The system accepts the responses if they match the grammars for each field.

7. Fields The form for the previous example consists of 4 fields: choice, student_name, student_id, info Each field has a prompt and a grammar Each field declares a variable which usually has the value of ‘undefined’ when the field is first interpreted Field attributes namee.g. exprthe initial value of the form item variable conda conditional expression e.g. to activate a field only when another field has been filled typee.g.

8. Headers (see next slide) Welcome … To begin say one of the following: students, courses, reports students courses reports studentsystem1a.vxml Example

9. Headers IBM Voice Toolkit: Bevocal: <!DOCTYPE vxml PUBLIC "-//BeVocal Inc//VoiceXML 2.0//EN" "http://cafe.bevocal.com/libraries/dtd/vxml2-0-bevocal.dtd">

10. Tutorial Exercises Exercise 1: Load and run the file ‘studentsystem1a.vxml’. The system will prompt you to say one of: students, courses, details. Exercise 2: Create a file called ‘studentdetails.vxml’. The file should have fields to elicit a student's name and the name of a course. You will need to construct grammars in each field containing a few names of students and courses.

11. (1) Bargein If an implementation platform supports bargein, the application author can specify whether a user can interrupt, or "bargein" on, a prompt using speech or DTMF input To disable “bargein”: Timeout Specifies the interval of silence between prompt and user input e.g. Default timeout value in Voice Toolkit is 7s. Timeout can be specified for individual prompts or set as a property at a higher level e.g. in the root document

12. (2) Count Used for prompts that change with each attempt to elicit input e.g. to provide more help what is the student’s id? please say the student’s id. It consists of 6 digits Conditional prompt Specifies that a prompt is only to be spoken if the condition is true Good morning Good evening

13. Menus A is a shorthand version of a form with a single field that prompts the user to make a choice and then transitions to another place (e.g. another dialog) based on that choice Menu attributes ide.g. scopegrammar scope of the menu dtmfwhen set to true, first nine choices are given the implicit values ‘1’, ‘2’, etc (unless explicitly specified) acceptdefault “exact” – defines the exact phrase to be recognised “approximate” – allows approximate recognition

14. Menu example C: Welcome to the Student System main menu Please choose one of the following options: You can say student details, course details, or records H: Student details. This example uses a menu to present a choice to the user. The user selects one of the options and the application moves to the next dialogue state.

15. To begin say one of the following: students courses examplemenu1.vxml

16. Choice Can specify a speech grammar – may be automatically generated Can specify a DTMF grammar Can use contents to form the prompt string Specifies an event to be thrown or URI to go to when the choice is selected reports Where to go toGrammar for this choice

17. DTMF in Menus DTMF only For reports press 1, etc DTMF or speech For reports press 1 or say ‘reports’ reports

18. State transitions and branching comment element Executable code when fields are filled Conditionals Requires documents to be specified

19. Tutorial Exercises Exercise 3: Save the file ‘studentsystem1a.vxml as studentsystem1b.vxml. Insert the additional code after the end of the field and before the end of the form. Note: To make this example work, it will be necessary to create dummy files for the options for the transitions specified in the element. Exercise 4: Load and run the file ‘examplemenu1.vxml’

20. Form Interpretation Algorithm Verification – verifying that the recognition result corresponds to what the user said boolean builtin grammar Variables referencing variables declaring variables The Form Interpretation Algorithm (FIA)

21. viewdetails.vxml you want details on looking up details on let's try again

22. Referencing a variable you want details on “student_name“ is the name of a field the expression retrieves the value of the field, i.e. the value that was recognised by the system this value is not necessarily the value that the user spoke e.g. if the input was misrecognised

23. More on Variables - declare a variable and possibly assign an initial value to it – assign a value to an existing variable

24. Variables: More examples

25. Tutorial Exercises Exercise 4: Load and run the file ‘viewdetails.vxml’. In response to the prompt 'Please say the student name', you can say one of: John, David, Rosemary, Jennifer. In response to the prompt asking for confirmation, you can say either 'yes' or 'no'. Try both options. Exercise 5: Adding verification to studentdetails.vxml Using the file viewdetails.vxml as a basis, save the file ‘studentdetails.vxml’ as ‘studentdetails_confirm.vxml’. Amend the file so that the system confirms the values elicited (studentname, coursename). If the user confirms, then the application terminates, otherwise it goes back to elicit the values again.

26. Form Interpretation Algorithm The form interpretation algorithm (FIA) determines the order of execution in a VoiceXML form or menu The FIA cycles through all of the fields in the form, asking the caller to supply values for each unfilled field. The FIA provides the default sequence of processing fields within a form; and thus hides the procedural aspects of form processing the from the dialog designer. Sometimes the fields of a form are “reset.” In this case, the FIA will select the reset field during the next cycle so all fields are filled eventually.

27. Form Interpretation Algorithm (FIA) AmountSource Destination Date ConfirmAmountSource Destination Date Each field has a prompt and a grammar (for user input) The field variable is initially set to ‘undefined’ When a field has been completed the field variable is set to ‘true’. The Form Interpretation Algorithm skips fields that are ‘true’ If a field value is disconfirmed the variable is re-set to ‘undefined’

28. Confirm Amount Source Destination Date Amount and Destination were disconfirmed Their values are re-set to ‘undefined’ The FIA visits the ‘Amount’ and ‘Destination’ fields again until their values are set to ‘true’ FIA (continued)

29. FIA: More detail Each variable (form-level elements and form item variables) is initialised to ‘undefined’ The first form item whose guard condition is false (undefined) is chosen to be visited. If no guard condition is false, then the FIA does an implicit operation. element form item variable set to true (cannot be visited again) content evaluated no input is collected

30. FIA in operation Field Name Course Level Date BeforeMid Name disconfirmed undefined true undefined true

31. More detail Note that the FIA selects one item on each iteration e.g. given the 4 fields above without any conditions on the fields: IterationBy-passed itemSelected item 1Nonename 2namecourse 3name courselevel 4name course leveldate 5course level datename 6name course level datenone: FIA exits

32. yes no You want details on Let's try again Let's continue FIA example

33. The element Defines how to process input-form-item variables after they have values e.g. to check that the input is correct or to verify it with the user (example: arrival time not earlier than departure time) If the checking involves only the value of a single field, the element is positioned within the element. If the check involves more than a single input-form-item variable, the element is placed within the element.

34. Example: Validating the input <!– The system has collected values for ‘source’ and ‘destination’ and checks that these are different --> the destination and source accounts must be different

35. let's try again Let's continue “confirm” is the name of a field The condition checks if the value of “confirm” is ‘no’ re-sets variables to ‘undefined’ namelist specifies which variables are to be re-set to ‘undefined’, otherwise all variables within the current form are re-set

36. Tutorial Exercise Exercise 5 1. Save the file ‘viewdetails.vxml’ as ‘newviewdetails.vxml’ 2. Delete the line and see what happens. 3. Insert the line but delete the value ‘confirm’. 4. Try again, this time with ‘confirm’ included and ‘student_name’ deleted.

37. Exercise 6: Asking which item to confirm Scenario: 1. User has supplied values for studentname and coursename 2. System verifies and user says ‘no’ 3. System asks: which do you want to change ‘the student name or the course name?’ which item do you wish to change: the student name or the course name?

38. so you want to details on taking which item do you wish to change: the student name or the course name? The line makes the value of “change” be ‘true’ – this ensures that the field will not be visited by the FIA.