1. Multi-Path Development of User Interfaces 1 Quentin Limbourg Louvain-la-Neuve, 4th November 2004

2. Multi-Path Development of User Interfaces 2 Presentation Plan  Context  Problem Statement  Thesis  Proposal –Ontological proposal –Methodological proposal  Case Studies and tools  Conclusion & Perspectives

3. Multi-Path Development of User Interfaces 3 Context  Transformational-driven Development of User Interfaces –Endorses the motivation of SE for HCI –User-Centred Design of UIs –Deriving a UI is heuristic and is not a straight line

4. Multi-Path Development of User Interfaces 4 Context (cont’d)  UIs are running fast…after change –Task redefinitions or reallocation –Organizational adaptation –Domain evolution –Obsolescence of languages and New Computing languages –New platforms  These changes give rise to the ellicitation development scenario

5. Multi-Path Development of User Interfaces 5 Context cont’d  These development scenarios are relevant to several problem areas : –Computer-assisted derivation of UIs –UI maintenance and refactoring –Multi-context UI development –Reverse engineering of UIs

6. Multi-Path Development of User Interfaces 6 Problem Statement  Ontological shortcomings –Explicitness, rigorousness, extendibility,commitment, communication  Methodological shortcomings –Expliciteness, rigorousness, consistency in application, predictability, flexibility, modifiability, communication.

7. Multi-Path Development of User Interfaces 7 Thesis Statement  It is possible to address the previously outlined shortcommings by: –A definition of an explicit and rigorous ontological framework for defining concepts involved in UI development –A definition of an explicit and rigorous methodological framework that is transformation driven and multi-path

8. Multi-Path Development of User Interfaces 8 Multi-Path Development of UI  Multi-path development qualifies a methodology that support the realization of multiples types of development paths withing a single framework Forward engineering Reverse EngineeringAdaptationAny Relevant Composition Development step Development step Development path Development path * 1 Development Scenario Development Scenario * * Development step Development step Development path Development path * 1 Development Scenario Development Scenario * *

9. Multi-Path Development of User Interfaces 9 Proposal  To support this approaches in a single framework we need: –An ontological framework that provide a description of concepts for each development stage –A methodological framework that allows a definition, organization, and execution of development paths

10. Multi-Path Development of User Interfaces 10 Ontological Framework  An ontology is a formal specification of a conceptualisation Abstract Concepts Model description with UML class diagrams Abstract Syntax Identified, Labeled, Typed, Constrained – Graphs Concrete Syntax AGG Visual Notation + usiXML Semantic Mapping Abstract to Concrete Mapping

11. Multi-Path Development of User Interfaces 11 Ontology: Abstract Concepts  Structured in viewpoints and in models –Task (CTT + some improvements) –Domain (UML Class/Object diagram + some improvements) –Abstract User Interface (vocabulary independent of the modality) –Concrete User Interface (vocabulary independent of the toolkit) –Context of use ( ) –Inter-model relationship mappings (traceability, integration of all views)

12. Multi-Path Development of User Interfaces 12 Ontology : Syntax  Abstract syntax –Directed, labeled, attributed, typed graphs. –Nodes are concepts –Edges are relationships between these concepts –Result: a UI specification is a BIG WHOLE graph  Concrete syntax : USIXML and AGG visual notation –USer Interface eXtensible Mark-Up Language –(graph structure is achieved by defining explicitly relationships) –AGG Visual Notation

13. Multi-Path Development of User Interfaces 13 Methodological Framework   Task and Domain Task and Domain Abstract User Interface Abstract User Interface Concrete User Interface Concrete User Interface Code User Interface Code Reification Abstraction Translation Code Generation Code Reverse Engineering         Task and Domain Task and Domain Abstract User Interface Abstract User Interface Concrete User Interface Concrete User Interface Code Final User Interface Reification Abstraction Translation Code Generation Code Reverse Engineering Reification Abstraction Translation Code Generation Code Reverse Engineering    

14. Multi-Path Development of User Interfaces 14 Model-to-Model Transformation  In the literature: –Opportunistic algorithms –APIs –XSLT –…  Drawbacks –Execution semantics is poorly defined –Consistency in applying steps –Consistency of resultant model –Procedurality –Cryptic syntax –Low predicatbility –Hidden rationale (entails low re-use)

15. Multi-Path Development of User Interfaces 15 Conditional Graph Rewritting  Generalization of string grammars  Grounded execution semantics (pushout construction)  Side-effect free  Attractive syntax  Declarativeness  Determinism (application strategy)  Seamlessness with ontological world (rules manipulate patterns of specification)  Tools  Extension (PAC, NAC)

16. Multi-Path Development of User Interfaces 16 How it works…  Find an occurrence of LHS in G (this occurrence is called a match). If several occurrences exist, choose one non- deterministically.  Check preconditions of both type PAC and NAC. If not verified, then skip.  Remove the part of G which corresponds to (LHS – K), where K is the morphism specified between LHS and RHS.  Add RHS – K into G – (LHS – K) as it is given by the corresponding relation between RHS – K and K.  Check postconditions of both type PAC (and notably that the resulting graph is properly typed) and NAC. If not verified, then undo the transformation rule.

17. Multi-Path Development of User Interfaces 17 Methodological World and Transformation World A development library stores (in usiXML textual format) paths, steps and sub-steps definition and their associated transformation systems and transformation rules

18. Multi-Path Development of User Interfaces 18 Substeps: From AUI to CUI  Definition of AUI structure –Which objects should be logically grouped ?  Definition of AIC types – Which « functionnality » should assume AICs and what data do they manipulate ?  Definition of spatio-temporal arrangement –How should AIC be arranged in space and/or in time ?  Definition of dialog control –What is the valid flow of action on AIOs ?

19. Multi-Path Development of User Interfaces 19 Substeps: From AUI to CUI  Define CUI structure –Which AIC is a window ?  Define CIC type ? –Which « widget » should represent which AIC ?  Define placement ? –What layout can be specified between CICs, CCs…  Define navigation ? –Which container can be started or closed from which container ?  Define dialog control ? –What is the valid flow of action on AIOs

20. Multi-Path Development of User Interfaces 20 Typed Model-to-Model Transformation Uses language Meta-Meta-Model Graph Structure is instance of Meta-Model Our Meta-Model Meta-Model Subset 1 e.g., Task+Domain Model is instance of Meta-Model Subset 2 e.g., Concrete UI Model is instance of Initial UI Model e.g., MyTaskAndDomainModel Resultant UI Model e.g., MyConcreteUIModel Transformation Rule Our transformation catalog

21. Multi-Path Development of User Interfaces 21 Example : Widget Replacement (1) 7 Youri... <radioButton groupName=“grupo01" defaultContent="Femme" defaultState="false" id="radiobutton_0"/> <radioButton groupName="grupo01" defaultContent="Homme" defaultState="true" id="radiobutton_1"/>... Excerpt for an usiXML CUI specification.

22. Multi-Path Development of User Interfaces 22 Example : Widget Replacement (2)

23. Multi-Path Development of User Interfaces 23 Example : Widget Replacement (3) The usiXML graph before aplying any rule

24. Multi-Path Development of User Interfaces 24 Example : Widget Replacement (4) LHS RHS NAC Rule 1: Create a new comboBox with the same id and name as the name of the group of radioButtons.

25. Multi-Path Development of User Interfaces 25 Example : Widget Replacement (5) Rule 1: Create a new comboBox with the same id and name as the name of the group of radioButtons. The usiXML graph after aplying the first rule

26. Multi-Path Development of User Interfaces 26 Example : Widget Replacement (6) LHSRHS ::= Rule 2: Convert every radioButton within the group “x” into an item for the comboBox “x”, we have just created.

27. Multi-Path Development of User Interfaces 27 Example : Widget Replacement (7) Rule 2: Convert every radioButton within the group “x” into an item for the comboBox “x”, we have just created. The usiXML graph after aplying the second rule

28. Multi-Path Development of User Interfaces 28 7 Youri...... Example : Widget Replacement (8) Excerpt from the final transformated usiXML specification

29. Multi-Path Development of User Interfaces 29 Example : Widget Replacement (10)

30. Multi-Path Development of User Interfaces 30 Case Studies  2 case studies were considered  Modus Operandi

31. Multi-Path Development of User Interfaces 31 TransformiXML API/GUI  API = set of transformation functions  GUI (Video)

32. Multi-Path Development of User Interfaces 32 Tool Support  Prototypes –TransformiXML API/GUI : transformation tool (previously shown) –IdealXML: Task, Domain and AUI editors –GrafiXML : CUI Hi-Fi + Code Generator (Java Swing, XHTML) –VisiXML : CUI Lo-Fi, MS Visio Plug-in –FlashiXML : flash renderer –ReversiXML : reverse engineering from HTML to CUI

33. Multi-Path Development of User Interfaces 33 Tools : IdealXML Demo

34. Multi-Path Development of User Interfaces 34 Tools : GrafiXML

35. Multi-Path Development of User Interfaces 35 Tools : VisiXML

36. Multi-Path Development of User Interfaces 36 Tools : FlashiXML Demo

37. Multi-Path Development of User Interfaces 37 Conclusion: Validation  External: Feasiblity of the approach has been exposed by the realization of two case studies and the provision of transformation catalogs  Internal: –Ontological requirements: Expliciteness, Expressivity, Human readability, Formality, Machine readability, Separation of concerns, Verifiability, Homogeneity, Reuse, Extendibility, Standards –Methodological requirements: Expliciteness, Flexibility, Formality, Executability, Separation of concerns, Extendibility Traceability, Correctness, Tool interoperability, Reuse.

38. Multi-Path Development of User Interfaces 38 Conclusion: Main Contributions  Ontological contribution –Definition of a formal language for describing viewpoints on a UI system  Methodological contribution –Explicit, open and formal definition of development steps and paths –Unification of several problem area in a unique framework thanks to the paradigm of multi-path development of user interface –Communication and incremental research and development effort

39. Multi-Path Development of User Interfaces 39 Conclusion: Secondary contributions  For SE community: –Application of transformational development concepts to UIs.  Graph Transformation Community: –Application of theoretical concepts to a new problem domain

40. Multi-Path Development of User Interfaces 40 Obstacles and Shortcomings  Main obtacles and shortcomings : –Ontological Lack of expressivity of models –Methodological Complexity of certain sub-steps Difficulty of finding an approprite level of generalization Difficulty in ordering rules and steps Lack of disjunction in the rule expression Difficulty to get meta-information Difficulty in managing large sets of transformation catalogs

41. Multi-Path Development of User Interfaces 41 Future Works  Extension to other types of UIs  Extension to other types of models  Define high-level building blocks (“constructive” patterns)  Extension of methodological steps definition and transformation catalogs  Extend formal foundations  Validate catalogs by human factors experts  Hide complexity  Evaluate transformation for run-time scenarios  Continue the development of ongoing tools …

42. Multi-Path Development of User Interfaces 42 Thank You !