George Vellis a Dimitrios Kotsalis a Demosthenes Akoumianakis a Jean Vanderdonckt b a Department of Applied Informatics & Multimedia, Technological Education Institution of Crete, Greece b Université catholique de Louvain, Louvain Interaction Laboratory ( Louvain School of Management, Belgium Model-based engineering of multi-platform, synchronous & collaborative UIs Extending UsiXML for polymorphic user interface specification

Introduction User interface challenges for software engineers – Programming intensive – Broad range of interaction components and widgets – Platform specificity (GUI objects, Web widgets, etc) – Lack of interoperability due to platform specificity (Some) Model-based engineering claims – UI engineering = Model specification – Focus on design rather than implementation – Linking (rather than direct calls) to platform-specific libraries

Introduction User interface challenges for software engineers – UI development time takes a large portion of time Software plans and requirements Validation Product design Verification Detailed design Verification Code Unit test Integration Product verification Operations and maintenance Revalidation Implementation System test System feasibility UI Design occupied 45% of the total amount of time devoted to each activity 50% 37% Mean = 44%

Retrospective assessment Claims proven useful – Model specifications can provide an engineering basis for UI engineering – Run-time linking alleviates the complexity of programming Claims proven possible – Multi-platform execution – Relatively straightforward Uis (e.g., form-based UI)

Retrospective assessment (Cont.) Several initiatives – One of them is UsiXML which is used in this work Reference Architectures – CAMELEON Reference framework Recommended by W3C Working Group on Model-based User Interface Design UsiXML is compliant with this framework Plethora of methods with some focus on collaboration – FlowiXML, AMENITES, CIAM, TOUCHE

Retrospective assessment (cont.) Challenges still pending attention – non-standard and customized widgets? – synchronous and collaborative UIs? – new affordances such as social awareness in distributed collaborative settings? – radically different interaction vocabularies? – design and run-time support tools for all the above?

Theory development – Build abstraction to capture interaction components as assemblages of capabilities (or affordances) – Substantiate such capabilities (or affordances) to polymporphic instantiation schemes – Specify polymorphic properties in a model-based fashion Engineering framework – Extend UsiXML as needed to support the new specification – Provide dedicated design tools and run-time components – Demonstrate the concept’s validity through pilot scenarios Research design

Consider various generations of ‘buttons’ – GUI buttons (in the 90s) The (primitive) assemblage of button = {two-state device to exercise control over functionality} Why abstraction?

A continuum of abstractions (Cameleon Ref. Framework) Why abstraction?

Scented button (cf. Willett et al., 2007) The (augmented) assemblage of ‘scented’ button = { Two-state device to exercise control } + { Collectivity (i.e., who else is in the same state of mind)} ‘Adding’ social scent

‘Like’ buttons (in the Web 2.0) The (new) assemblage of the ‘like’ button = {Two-state devices to exercise control} + {Expression of opinion (i.e. I like this)} ‘Adding’ expression of intention

‘Like’ buttons with social scent The (combined) assemblage = {Two-state device to exercise control} + {Expression of opinion (i.e. I like this)} + {Collectivity (i.e., who else is in the same state of mind)} Putting them all together

Why limit our creativity to conventional widgets? – elastic buttons and their interactive behaviour Generalizing the concept Note the embedded affordances

What happens when an elastic button is to be instantiated across devices and platforms Adding to complexity

How can we … – model digital assemblages in interaction components? – instantiate them as the need arises and depending on context / platform capabilities – integrate them into specifications as first class elements / properties – compile specs that are general enough Motivating question

A scheme that relies on implementation agnostic (i.e., abstract) specifications of UIs – i.e. an abstract button can be instantiated as a conventional GUI button when implemented with the ‘primitive’ assemblage a scented button when implemented with the ‘augmented’ assemblage a ‘like’ button when implemented with the ‘combined’ assemblage At run-time and once user and usage context parameters are discovered, the implementation agnostic spec is translated to context-specific interaction vocabularies using dedicated tools (One) Answer: Polymorphic instantiation

Introduce new widgets and their properties as first- class design elements Polymorphic Widget Specifications Basic concept WSLLibraries Resources Widget Archive

Specify complex capabilities & affordances in a model-based fashion – Devise required properties – Define model elements – Create specification – Mapping models The role of MBUI engineering

The role of MBUI engineering the case of a socially aware button

Extend UsiXML as needed to support the new specification – New set of models Widget Resource Model Behavior model – Additional models for collaborative work Squad model Consistency model Abstraction model Session model A new set of models

A new design tool

A new run-time environment Design Environment Platform Server Abstraction Classes Collaboration Plugin Social Proxy Polling Social Proxy Abstraction Classes Collaboration Plugin Social Proxy Polling Social Proxy Abstraction Classes Runtime Environment Server Side Framework (SSF) UI-1 Axis2-Framework Web-Services Layer Session Manager Widget Archives Repository Notification Queue WSL Dependencies Resources UI-Model Repository 2.join 3.getUIModel 4.getArchives 5.buildUI6.attachStates 1.initPS trackChanges applyChanges distributeChanges Client-1Client-N UI-N P.S deployUI UI Design Tool Widget Spec Tool

UIDL’2011 (simple) desktop scenario

MDSD’12 multiplatform scenario (in the paper) Player-1:Desktop/PCPlayer-2: Android/mobile Viewer-1: Desktop/PC

Collaborative assembly of vacation packages User roles – Business partner (desktop) – Administrator (desktop or mobile – Platforms (desktop/swing, mobile/Android) – Context (synchronous and collaborative) Complex and custom widgets (i.e., elastic buttons) Synchronous collaboration Post MDSD’12 scenario

The desktop case administrator’s view of vacation package / swing

The desktop case business partners’ view of vacation package / swing

The mobile case administrator’s view of vacation package - Android Due to space constraints certain functions become automatically adapted to vocal interaction

The designer’s role? Designing for certain affordances

The run-time instantiation the specified affordances enabled

Current and on-going work – Advanced toolkits (i.e., visualization) – Run-time adaptivity and UI plasticity in distributed and ubiquitous settings – ‘Social’ games Acknowledgements – The work is supported by ARCHIMEDES III – Thanks to Université catholique de Louvain (UCL) for supporting the first two authors’ PhD dissertations Due to be submitted in 2013 Future work & acknowledgements