1. Using VoiceXML, XHTML, and SCXML to Build Multimodal Applications James A. Larson

2. SCXML controls the flow of an application SCXML controls modalities –VoiceXML –XHTML –Others, e.g., InkML, SVG SCXML controls Web services Agenda

3. State Chart XML Represents a state transition system –States –Transitions A B

4. Possible Actions within a A Multimodal life cycle events Executable content,,

5. SCXML Sequentially perform the actions in two states done A B

6. SCXML <transition event= "done" cond="result='OK ̍ '" target="B"/> <transition event= "done" cond="result='Opps'" target="C"/> done OK done Opps A BC Conditional branch

7. SCXML A CD B Parallel States

8. SCXML controls the flow of an application SCXML controls modalities –VoiceXML –XHTML –Others, e.g., InkML, SVG SCXML controls Web services Agenda

9. SCXML + VoiceXML <transition event= "done" target="B"/> state id="B"> done A B Say your destination airport <grammar src= “Airport.grxml” getAirport.vxml

10. SCXML + XHTML <transition event= "done" target="B"/> state id="B"> done A B {function sendDone…} enter month getAirport.xhtml

11. SCXML + XHTML + VoiceXML state id="B"> done B <transition event= "done" target="B"/> <transition event= "done" target="B"/> A A1A2 done B

12. SCXML + InkML + VoiceXML state id="B"> B <transition event= "done" target="B"/> <transition event= "done" target="B"/> A A1A2 done B Another example of multiple modalities

13. SCXML + InkML + VoiceXML state id="B"> B <transition event= "done" target="B"/> <transition event= "done" target="B"/> A A1A2 done B Tuesday

14. SCXML + InkML + VoiceXML state id="B"> B <transition event= "done" target="B"/> <transition event= "done" target="B"/> A A1A2 done B Tuesday Las Vegas Portland

15. SCXML + InkML + VXML B <transition event= "done" target="B"/> <transition event= "done" target="B"/> A A1A2 done B Tuesday Las Vegas Portland Las Vegas Portland Tuesday state id="B">

16. SCXML controls the flow of an application SCXML controls modalities –VoiceXML –XHTML –Others, e.g., InkML, SVG SCXML controls Web services Agenda

17. SCXML Control More Than Just Modalities SQL Complex calculations Web services vxml sql complexCalculation xhtml

18. References State Chart XML (SCXML): State Machine Notation for Control Abstraction –http://www.w3.org/TR/scxml/ Multimodal Architecture and Interfaces –http://www.w3.org/TR/mmi-arch/ Commons SCXML (Open Source) –http://jakarta.apache.org/commons/scxml/

19. Summary SCXML controls the flow of an application SCXML controls modalities –XHTML –VoiceXML –Others, e.g., inkML, SVG SCXML controls Web services