A model-based development for 3D User Interfaces Juan Manuel Gonzalez Calleros 1, Jean Vanderdonckt 1, Jaime Muñoz Arteaga 2 1. Université catholique de Louvain, School of Management (IAG) Information Systems Unit (ISYS) 2. University Autonomous of Aguascalientes 6th International Conference on Computer-Aided Design of User Interfaces CADUI´2006 June 6th-8th, 2006, Bucharest, Romania.

June 6-8CADUI Bucharest, Romania2 Contents  Introduction  Related Work  Model-based development for 3D User Interfaces  Case Study  Conclusion

June 6-8CADUI Bucharest, Romania3 Introduction - Problem  Actual methods: Rarely provide the design knowledge Rarely provide the design knowledge Are more focusing on the implementation Are more focusing on the implementation  Available tools for 3D UIs are: Toolkits, interface builders, rendering engines

June 6-8CADUI Bucharest, Romania4 Introduction – proposal  We rely on: A user-centered approach A user-centered approach Separation of concerns Separation of concerns Model-driven architecture Model-driven architecture  Our goal:  “To demonstrate the feasibility of a MDE- compliant method that is user-centered as opposed to contents-centric for developing 3D UIs”

June 6-8CADUI Bucharest, Romania5 Contents  Introduction  Related Work  Model-based development for 3D User Interfaces  Case Study  Conclusion

June 6-8CADUI Bucharest, Romania6 Related work – 3D UIs generation  Programmatic approach: OpenGL, Direct3D, C++, MCNL, VRML, X3D, …  Toolkit programming: Maya, Max3D, Crazy Eddie's GUI System, Anark, Alice  Render engines: Microsoft task gallery, infinite 3D Cube, Sphere XP, looking glass, Clara browser, 3DNA  Model based: ontoWEb (VR-Wise), ConGenIVE (VRIXML), InTML, Contigra

June 6-8CADUI Bucharest, Romania7 Related work – Programmatic Approach, VRML

June 6-8CADUI Bucharest, Romania8 Related work – Toolkit Approach, Anark

June 6-8CADUI Bucharest, Romania9 Related work – Render Engines, 3DNA

June 6-8CADUI Bucharest, Romania10 Related work – Model-based approach, CoGenIVE-VRIXML    <Tickstyle orientation =" horizontal“ position =" both "  frequency ="16"/>      

June 6-8CADUI Bucharest, Romania11 Contents  Introduction  Related work  Model-based development for 3D User Interfaces  Case Study  Conclusion

June 6-8CADUI Bucharest, Romania12 We rely on…  Cameleon reference Framework  UsiXML  UML class diagrams  Graph transformations

June 6-8CADUI Bucharest, Romania13 Method Outline AUI production (IdealXML) CUI production (TransformiXML) 3D Concrete User Interface #1 (UsiXML) Maya Development environment Java3D code HTML browser with Cortona player Java3D Generator Abstract user interface #1 Domain model Task model … … Abstract user interface #2 Abstract user interface #n 3D Concrete User Interface #3 (MEL) VRML97/ X3D code HTML browser with Cortona player VRML97/X3D Generator … Step 1 Step 2 Step 3 Step 4 Task & Concepts Abstract UI Concrete UI Final UI AUI: UI definition independent of any modality of interaction CUI: concretizes an AUI into CIOs (widget sets found in popular graphical and vocal toolkits) FUI: operational UI that runs on a particular platform either by interpretation or by execution 3D Concrete User Interface #2 (Java3D) VRML97 code HTML browser with Cortona player VUIToolkit

June 6-8CADUI Bucharest, Romania14 Software tools - IdealXML

June 6-8CADUI Bucharest, Romania15 Software tools - AGG

June 6-8CADUI Bucharest, Romania16 Software tools - Alice

June 6-8CADUI Bucharest, Romania17 Software tools - Maya

June 6-8CADUI Bucharest, Romania18 Software tools - VUIToolkit

June 6-8CADUI Bucharest, Romania19 Contents  Introduction  Related Work  Model-based development for 3D User Interfaces  Case Study  Conclusion

June 6-8CADUI Bucharest, Romania20 Case Study – Virtual polling system

June 6-8CADUI Bucharest, Romania21 Case Study – Task and domain definition

June 6-8CADUI Bucharest, Romania22 Case Study – Concrete Definition

June 6-8CADUI Bucharest, Romania23 Case Study – Concrete Model B

June 6-8CADUI Bucharest, Romania24 Case Study – From Abstract to concrete Model NAC LHS:=RHS x>2

June 6-8CADUI Bucharest, Romania25 Case Study – From Abstract to concrete Model “Show Questionnaire” Output (value unknown) Attribute, data type, domain characteristics An output text “Select Answer”Select attribute value + repetitive (selection values not known) Data type, domain characteristics A dropdown list, a group of option buttons “Send Questionnaire”ControlFeedbackA trigger “Navigation”NavigationFeedbackA trigger

June 6-8CADUI Bucharest, Romania26 Mock-up of the Concrete User Interface

June 6-8CADUI Bucharest, Romania27 Case Study – From Abstract to concrete Model <SphereTrigger defaultContent="Start" radious="1.5" solid="True" isVisible="true" isEnabled="true"> isEnabled="true"> <Material diffuseColor=" " specularColor=" " emissiveColor=" " shininess="0.3"/> emissiveColor=" " shininess="0.3"/>

June 6-8CADUI Bucharest, Romania28 Case Study – Editing the FUI in Maya

June 6-8CADUI Bucharest, Romania29 Case Study – Editing the FUI in Alice

June 6-8CADUI Bucharest, Romania30 Case Study – FUI generated with Alice

June 6-8CADUI Bucharest, Romania31 Case Study – FUI generated with the VUIToolkit

June 6-8CADUI Bucharest, Romania32 Case Study – FUI generated with the Maya

June 6-8CADUI Bucharest, Romania33 Case Study – FUI generated with the Maya

June 6-8CADUI Bucharest, Romania34 Use of the virtual polling system Case Study in a virtual world

June 6-8CADUI Bucharest, Romania35 Contents  Introduction  Related Work  Model-based development for 3D User Interfaces  Case Study  Conclusion

June 6-8CADUI Bucharest, Romania36 Conclusion –Model-based approaches Models t= task, Do = Domain Di = dialog AUI=abstract presentation CUI=concrete user interface U = user, C = context. Inter Model Transformation  Bidirectional derivation → Derivation link ≈→ Manual Derivation ≈  Manual Bidirectional der. FUI = Final User Interface FUI target languages *Indicates that it is done manually VR-Wise CUI CUI → FUI VRML, X3D CoGenIVE T, Di, CUI ( T, Di, CUI )  FUI C++ InTML Di, CUI T ≈→ (Di, CUI), (Di, CUI) ≈→ FUI Contigra CUI, Di (CUI, Di) → FUI X3D, Behavior3D, Audio3D Our methodology T, Do, C, AUI, CUI T  Do, T  AUI, AUI  CUI, CUI  AUI, T  CUI, (T, Do,AUI,CUI)  C, CUI ≈→ FUI Java, XHTML, Flash, HTML, Voice XML* Java 3D*, X3D*, VRML*.

June 6-8CADUI Bucharest, Romania37 Conclusion  The method based on graph transformation has been introduced, defined and illustrated  We proposed a method for 3D UI development whose life cycle was divided into four steps ranging from the most abstract to the most concretes according to the principles of Model- Driven Engineering.  The feasibility of the approach is much depending on the amount and the quality of the design rules that are encoded in UsiXML.

June 6-8CADUI Bucharest, Romania38 Conclusion  Future work. Exploring more design options and encode them in UsiXML Models Models Transformational rules Transformational rules Software tools Software tools More FUI More FUI

June 6-8CADUI Bucharest, Romania39 Thank you Multumesc!  Juan Manuel Gonzalez Calleros   Contact:  