SALT & The Microsoft Speech Application SDK VMK Lecture Zheng-Hua Tan (zt@kom.aau.dk) SMC – Speech and Multimedia Communication Aalborg University 18/11/2018

SALT – Speech Application Language Tags is a markup language encompassing extensions of HTML and others as opposite to VoiceXML which is a complete markup language in its own right therefore, can either add speech interface to regular Web pages for multimodal applications or make speech-only interface for telephony applications or separate them for cross-modal applications, where a single application page can be reused for different modes on different devices, e.g. GUI-only and speech-only. is designed with the philosophy that speech is introduced into the well-established GUI interface as one additional modality but not more than any others. 18/11/2018

SALT – cont. The core of SALT Speech input <listen> (for ASR configuration, execution & post-processing, and recording) <grammar> (for specifying input grammar resources) <bind> (for processing of recognition results) <record> (for recording audio input) Speech output <prompt> (for TTS configuration and prompt playing) DTMF input <dtmf> (for configuration and control of DTMF collection) Simple Message EXtension <smex> (for general purpose communication with platform components) 18/11/2018

SALT – cont. An example: <salt:listen id=“travel” onnoreco=“noReco()”> <salt:grammar src=“./city.grxml” /> <salt:bind targetElement=“textBoxCity” targetAttribute=“value” value=“/result/cityName” /> </salt:listen> <salt:prompt id=“alert”> I did not understand you. </salt:prompt> <script> function noReco() { alert.Start(); } </script> 18/11/2018

The SALT reference architecture SALT – cont. The SALT reference architecture Not for any particular device type, but rather for a range of architectural scenarios. As a consequence, a range of devices from PCs to handheld devices to the telephone can be speech-enabled. Multimodal Clients PC (Local speech engines) Handheld computer HTML + SALT + Script PSTN/VoIP Phones Web Server Telephony Server Remote Speech Server (ASR and TTS) 18/11/2018

The Microsoft® Speech Application SDK The SDK “is a set of development tools based on the SALT specification that will make it easier and faster for developers to incorporate speech functionality into Web applications ” Features: Microsoft ASP.NET Speech Controls Visual Studio .NET Speech Development Tools Speech Add-in for Microsoft IE Newsgroup: microsoft.public.netspeechsdk 18/11/2018

field trails A number of field trails in the area of speech enabled access to network hosted applications at AAU personal assistant (PA) vehicle registration database on-line query service medical application 18/11/2018

Personal Assistant Functionality Speech enabled Device-identification Scheduled Calendar Address book Speech enabled SALT Speech .Net Device-identification PC, laptop PC PDA 18/11/2018

Personal Assistant – cont. Access via Fixed/wireless networks IP network Mixed network Seamless transfer across networks AND terminals Accessibility from anytime, anywhere PA Server IP-based Core Network GPRS Access Network WLAN AP 18/11/2018

Personal Assistant – cont. FILL OUT: speech input in order of: Date of the activity, Start time, End time, Type and to add a contact  if necessary. SET followed by  DATE, START TIME, END TIME, TYPE. The user can now the requested field. ADD CONTACT. To add a contact to the contact list . SAVE. Saves the activity. BACK. Returns to the Calendar page 18/11/2018

Personal Assistant – cont. Accessible at http://192.168.113.21/pa/ http://kom.auc.dk/pa/ 18/11/2018

References The SALT Specification and The SALT Technical White Paper. http://www.saltforum.org/ The Microsoft Speech Application SDK. http://www.microsoft.com/speech Kuansan Wang, “SALT: A Spoken Language Interface For Web-Based Multimodal Dialog Systems,” ICSLP-02, Denver, USA, September 2002. Hansen H., Pannet Y., Sørensen T. and Tseveenochir A., ”Speech Enabled Web Application Using MS Speech .NET”, Student Project Report, Aalborg University, May 2003. 18/11/2018