1. Mir Farooq Ali Computer Science, Virginia Tech May 9, 2003 Building Multi-platform User Interfaces using UIML

2. Outline Multi-platform User Interface (UI) problem User Interface Markup Language (UIML) Task Modeling Future Research

3. Introduction Variety of Hardware Variety of target platforms Netscape 6.1 for HTML Internet Explorer 6.0 for HTML Java Swing

4. Introduction Users want access to the same applications and data Traditional approaches are geared towards a single platform New approach is needed to build multi- platform UIs

5. Goals for UIML Able to describe implementations of UIs that are –Multi-platform –Multi-lingual Usable for non-traditional UIs: voice, etc Same expressive power as target languages (Java, HTML, WML, …)

6. UIML UIML is an XML-based markup language Meta-language for UIs Can be used currently in conjunction with platform-specific vocabularies to create Uis –A vocabulary is defined to be the set of UI widgets, their properties and events for one specific platform

7. UIML (Cont’d…) Provides complete description of a UI Renderers are needed to take UIML file and render it to UI Rendering can be done through –Interpretation –Translation Renderers are available for Java, HTML, WML, VoiceXML and PalmOS

8. Example UI Widgets Events associated with widgets Look and feel

9. Separation of Concerns UIML considers an UI to be composed of 4 sections –structure –style –behavior –content

10. ..................... UIML (Cont’d…) Skeleton of UIML document

11. UIML Structure section indicates the hierarchical relationship between various ‘parts’ Widget

12. UIML Style section describes “look and feel” of UI using properties for individual parts UIML Example Hello World!

13. UIML Behavior section defines what happens when you perform any action on the parts Top button pressed.

14. UIML Examples 1.Hello World example Java Swing WML HTML 2.UI with events UIML with Java Swing vocabulary Actual Java Swing program

15. Example: UI for 3 Platforms Date book: Java Palm OS WML

16. What UIML Is and Is Not UIML: is a language for implementing UIs UIML: is not a language for UI design (We assume UIML is generated by a model, specification language, design tool, or even manually)

17. Example: Vocabularies & Mappings Recall: OK! Vocabulary: class, property, event names (e.g., “Button”) Mapping: … versus …

18. How UIML Addresses Multi- Platform UIs 1.Tree structure (cascading): describe a family of UIs as in tree form 2.Generic vocabulary + transformation 3.Building block of overall methodology

19. 1. Tree Structure Factor out common elements to UIs on all platforms in form of an XML tree –Structure: changes across families –Style: changes across families –Content: may change across families –Behavior: may largely remain intact –Logic: API to logic unchanged –Presentation: differs by family

20. 2. Generic Vocabulary + Transformation Vocabulary is defined outside UIML Vocabulary represents set of abstractions. Abstractions could be… 1.identical to UI toolkit (“Platform-Specific vocabulary”) 2.cross UI toolkits (e.g., generic across Java/HTML, VoiceXML, WML) 3.domain-specific (e.g., abstractions to build UIs for news articles -- title, abstract, body, navigation)

21. Sample Generic Vocabulary (#2 in Previous List) Generic PartUIML Class Name Generic Top ContainerG:TopContainer Generic AreaG:Area Generic ListG:List Generic LabelG:Label Generic ButtonG:Button Generic IconG:Icon Generic RadiobuttonG:RadioButton

22. Transformations Transformation maps UIML document with generic vocabulary to UIML document with platform-specific vocabulary............

23. Transform Engine HTML Renderer Java for Swing toolkit Example of How UIML Is Used HTML for Web browser UIML/HTML vocabulary UIML/Java vocabulary Java Renderer UIML describing application with generic vocabulary

24. 3. UIML as Building Block Task Model UIML Family Model for desktop platform UIML Family Model for WML platform UIML for Java Swing UIML for HTML UIML for VoiceXML UIML for WML Model->UIML Transformation 1 UIML->UIML Transformation UIML Family Model for Voice platform Generic UIML Platform-Specific UIML

25. Multi-platform UIs Family model is in UIML using generic vocabulary –Generic vocabulary contains UI widgets like G:TopContainer, G:Label, etc.

26. Task Model Using ConcurTaskTree notation –Developed by Fabio Paterno, CNUCE-C.N.R., Pisa, Italy Hierarchical tree-like structure –Each node in tree indicates a task –Temporal operator exists between adjacent sibling tasks

27. Task Types Four different task types –User –Interaction –Abstract –Application Eight different kinds of temporal operators

28. Use of Task Model Developer builds task model specifying task types and temporal operators Transformation done by system to create generic UIML, mapping –tasks to UIML ‘parts’ or widgets –temporal operators to UIML ‘behavior’

29. Sample Task Model

30. Interaction design Task Model-centric Interaction Design Task Model Family Model for desktop platform Family Model for phone platform UIML for Java Swing UIML for HTML UIML for VoiceXML UIML for WML Family Model for Voice platform Task/Domain analysis Interaction s/w design Interaction s/w impl. Traditional Usability Engineering

31. Future Research Accessibility –Treat accessible toolkits as a separate platform TIDE –Enhance the current version –Introduce more platforms

32. TIDE

33. Publications CADUI’2002 Full Paper CHI’2002 Short Paper Book chapter in edited book “Multi-Device User Interfaces: Engineering and Application Frameworks”

34. Questions?