1. XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech.

2. Issues on Interaction with Web The coming of heterogeneous devices and interaction manner access by : PC, Mobile phone, PDA, AutoPC, DTV, … interaction with: modality-rich terminals, mixed initiative,...

3. Ex. On-line Shopping Devices - order goods at Kiosk..... touch panel - confirm the order with a mobile phone..... voice - change the order at home..... DTV remote terminal XML Document Server (Contents ; View ; Interaction) Our Purpose: Seamless Services over Internet

4. XML Document Server (Contents ; View ; Interaction) How to do it? <xhtml........ VoiceXML........ CSS <xhtml....... Form <salt:......

5. CSS XISL: Extensible Interaction-Sheet Language CSS <xhtml....... Form........ contents- independent <xhtml....... Form VoiceXML........ remodels VoiceXML to meet MMI requirements

6. XISL: Device-Independent - 1/3 no modality-specific tag : input modality <input type=speech event=recog target=grammar.gxml match=/grammar/goods return=goods_name> data <input type=dtmf event=push target=grammar.gxml match=/grammar/goods_selectreturn=goods_name> digit...........

7. XISL: Device-Independent - 2/3 Front-end Module type: speech event:recognize recognize grammar rule: introduction return: word1 type: pointing : ASR Pointing Keyboard Hello! Hello word1: Hello grammar type: speech event: recognize target: ***.xml match: /page return: word1:Hello Dialog Manager

8. XISL: Device-Independent - 3/3 no modality-specific tag : output modality type event mode windowopen, close, … speechplay, stop, … TTS, rec_speech, … video play, stop, … audio play, stop, … agentmove, speak, …

9. MMI Description Language: XISL - is content ( & view ) -independent. - has no modality-specific tags that enable XISL device-independent. - has MMI control tags and attributes. realizes seamless services over internet and reuse of XML documents. meets modality-rich services and avoids patent issues.

10. XISL enables combinatorial usage of modalities. Sequential Parallel adding a new modality Alternative

11. ......... <input type=dtmf event=push target=/d_gram.gxml match=/d_gram/goods_select return=number> digit <input type=speech event=recognize target=/gram.gxml match=/gram/goods_select return=goods_name> data XISL: example -1/2

12. nancy You selected No. One. XISL: example -2/2

13. Implementation -1/4 XISL XML XSL InputModalities OutputModalities Document Server Interpreter Dialogue Manager Dialogue Manager Input Integrator Input Integrator Response Controller Response Controller System Architecture

14. Dialogue Control I/O profile XML editing transition Op. Control GLRparser Input Integration user operation Interpreter Start Dialog Dialog Manager Action Execution Implementation -2/4 Document Server Front - end Document Buffer

15. Dialogue Control by Using GLR TBL REDUCE- 2 REDUCE-1 SHIFT-5 SHIFT-4 ACCEPT SHIFT-2SHIFT-1SHIFT-3 EXCH1 BUYGOODS completetimeoutgoods_spbuy_btn GOTO TBLACTION TBL IN State SHIFT-3 SHIFT-5 REDUCE-2 Implementation -3/4

16. Implementation -4/4

17. XISL Tree View Interaction Builder - binding MMI -1/4

18. Interaction Builder - binding MMI -2/4

19. Interaction Builder - binding MMI -3/4

20. <![CDATA[ Merlin ]]> Interaction Builder - binding MMI -4/4

21. Future Work - XHTML + XISL - Implementation of 3GPP applications - MMI Requirements: more complicated usage of MMI - Rapid-prototyping: GUI-based developer support system (IPA project) http://www.vox.tutkie.tut.ac.jp/XISL/XISL-E.html