New challenge: telephone Text To Speech & audio Speech recognition VoiceXML Homework: sign up on studio.tellme.com

Telephone Caller to system: speech recognition, –using grammars (limited vocabulary, general audience, no training) –optional use of touch tones (numbers) System to caller: recorded audio (wav files) plus TTS (text to speech) Limited bandwidth, in comparison to other applications, but very familiar, ubiquitous medium 800 long distance, some airline information systems, others?

Problems in context Speech recognition: very difficult if –no restrictions on speakers –grammar for all of English with aim of 'natural language understanding' Text to speech: much easier problem (but English is more difficult than more fully phonetic languages like Spanish. (I've been told.) (More next class)

studio.tellme.com Company that provides ‘engine’ for applications Provides developing environment –We are doing the tellme version of VoiceXML, but it appears to be standard. Register as a developer: –Provide your own id; assigned a PIN –Scratchpad for quick testing Put VoiceXML in ScratchPad place (no audio files) VXML (8965) –SAY id and then PIN. –Application URL for projects with multiple files To look at someone else's project, you change your Application URL –called pointing your account to a new source.

VoiceXML XML document (VXML header) VoiceXML has tags for flow-of-control and calculations. –Also can use for JavaScript Grammars come in different varieties. We will use the tellme way. –Grammars are included in CDATA tags to prevent XML interpretation. –Many grammars constructed for you. … will listen for yes or no. … will listen for currency. – for list

VoiceXML basics, continued element can contain – elements, which can contain,, other – which can contain (if not one of built-in grammars) tags can be at different levels (for example, document, block, or higher levels) tags elements for JavaScript (which can also appear in expressions>

VoiceXML basics: typical case a form element –, made up of, with reference to recorded wav file and backup text, if NOT using built-in grammars designated by type attribute of field. This is a CDATA section. with (follow-on) code using field for nomatch, noinput cases

Caution A form contains various elements, including a field. If a field has a grammar and the grammar is satisfied, control goes to a filled tag

obligatory… Hello, world recorded using tellme studio backup using TTS, just in case src file missing

Preparation: objects JavaScript (and other languages) use classes and objects Objects (aka object instances) are declared (created, instantiated) as members of a class Objects have –properties ('the data') –methods (functions that you can use 'on' the objects) –static methods Math.random

Example: tm_date var dt = new tm_date; creates a date/time object. Use methods to extract/manipulate information held 'in' dt. var day = dt.get_day(); Use static methods supplied to do common tasks: var dn=tm_date.to_day_of_week_name(day); or directly: var dn=tm_date.to_day_of_week_name(dt.get_day());

outline Header stuff script with external reference script (code) encased in CDATA notation Form/Block, with text to speech using value produced by script Closing stuff

<script src=" /tm_date.js"/> Will make use of data functions

<![CDATA[ var dt = new tm_date(); var monis = tm_date.to_month_name(dt.get_month()); var dateis = dt.get_date(); var dayis = tm_date.to_day_of_week_name(dt.get_day( )); var yearis = tm_date.to_year_name(dt.get_full_year() ); var houris= dt.get_hours() - 4; var minutesis=dt.get_minutes() var whole = 'The date is '+ monis+' '+dateis+'. It is ' + dayis+'. The time is ' + houris + ' ' + minutesis; ]]> brute force correction from GMT

Hello. Good bye. Can use block for audio

Example: my family Directed responses to 3 family members: – Daniel, question/response on activities –Aviva, question/response on number of cranes –Esther response Calculations (arithmetic) done using variables if tags –The cond attribute is a condition test. limited error handled: exit on no-match event –alternative is to repeat prompt, generally using count attribute

Hello. Who is calling?

<![CDATA[ [ [dan daniel (daniel meyer) (dan meyer)] { } [aviva (aviva meyer)] { } [esther (esther minkin) ] { } ] ]]>

Sorry. I didn't get that. never happens Note inner, single quote marks. Note double ='s

Congratulations on the new job. Did you work on your thesis, or do aikido or jo today? <![CDATA[ [ [aikido (i key dough)] { } [thesis (work)] { } [jo (joe) ] { } [both (all) (everything) ((i key dough) jo)]{ } [none nothing (sort of)] { } ] ]]> I didn't quite understand. Call or send .

Some aikido is fine. Good, but do other things also. don't get hit in the head. Doing some of everything is best. You deserve a break, but remember you want to be done by September. See you soon. Good bye

Hello, Aviva. How many cranes have you made? <![CDATA[ NATURAL_NUMBER_THRU_9999 ]]> Sorry. I didn't get that.

to go. You're in the home stretch More than half way Off to a good start Get a move on Good bye. can't use <

Hello, Mommy. This is all I can do now.

Application logic VoiceXML elements (for example, and. –Note: more powerful than XSLT: tag JavaScript code in attributes (for example, cond, expr) JavaScript code in –Encase in CDATA to avoid problems with certain characters external JavaScript code, cited using

Class work EVERYONE (who hasn't already) signup studio.tellme.com tonight Design simple application (you may work in groups): –Ask one question –Detect and respond to each of 2 or 3 answers –Use examples here for models –All text to speech Pick (at least) one and implement. (Do this a short time and then go on to next lecture. Resume after 9pm when minutes are free.)

Homework (Majors requirement overdue: there will be a deduction but better late than never.) Go to studio.tellme.com & signup as developer. –try examples (using scratch pad) –record some voice samples –do tellme tutorials ALSO try and report on –800 long distance or some other commercial application