1 ICAS’2008 – Gosier, March 16-21, 2008 Model-Driven Engineering of Multi-Target Plastic User Interfaces Benoît Collignon 1, Jean Vanderdonckt 1, Gaëlle Calvary 2 1 Université catholique de Louvain (UCL) Louvain School of Management (LSM) Belgian Laboratory of Computer-Human Interaction (BCHI) 2 Université Joseph Fourier – Grenoble I, Laboratoire LIG 385, rue de la Bibliothèque BP 53 - F Grenoble Cedex 9 (France)

2 ICAS’2008 – Gosier, March 16-21, 2008 Introduction The problem of designing multi-target user interfaces –Context of use = {user, computing platform, physical environment} –Multiple contexts of use => multiple variations of users, platforms, and environments Independently Simultaneously Notion of plasticity –Plasticity = user interface ability to change according to a change of the context of use while preserving predefined usability properties Context-aware adaptation = mere adaptation depending on context changes Plasticity = more than context-aware adaptation since usability should be guaranteed at a minimum level of satisfaction To address the problem, we combine –Model-driven engineering of user interfaces –Plasticity as a mean to address context-aware adaptation

3 ICAS’2008 – Gosier, March 16-21, 2008 Multiplicity of contexts of use LocationRoleDeviceExperience SportingMultimedia Travel programme WorkingTravel booking site Powerful interface for complex operations TravellingBooking notification Everywhere connectivity for simple data exchange FamilyTV is multi-media family device #1

4 ICAS’2008 – Gosier, March 16-21, 2008 Example of a Plastic User interface Property of plasticity = adaptation to the context of use while preserving predefined usability properties of interest [Grolaux et al.,2002]

5 ICAS’2008 – Gosier, March 16-21, 2008 Example of a multi-target UI Traditional approach: Atomica HTML/Java UI code Target 1: Web terminal Target 2: Personal computer Visual C++ UI code Target 3: Pocket PC Mobile Visual Basic UI

6 ICAS’2008 – Gosier, March 16-21, 2008 Related work Some solutions to address this problem: –The mediator solution (model-based approach) [Eisenstein et al.,2001]

7 ICAS’2008 – Gosier, March 16-21, 2008 Related work Some solutions to address this problem: –Multiple variations based on a model (here, a platform model) [Märtin et al.,1990]

8 ICAS’2008 – Gosier, March 16-21, 2008 Related work Some solutions to address this problem: –Adaptive UI layout: widgets are (un)displayed and laid out according to resolution [Plomp & Keranen,2002]

9 ICAS’2008 – Gosier, March 16-21, 2008 Related work Some solutions to address this problem: –Artistic resizing : widgets have different variations according to resolution [Dragicevic et al.,2005] w h wLwL hLhL yLyL xLxL r hBhB wBwB  x L = (w-w L ) / 2  y L = (h-h L ) / 2  w L = 20  h L = 10  w B = 5  h B = 5  r = 20

10 ICAS’2008 – Gosier, March 16-21, 2008 Our solution Combine model-driven engineering and plasticity domains

11 ICAS’2008 – Gosier, March 16-21, 2008 Case study Case study: multi-target plastic UI for date & time (FlexClock) [Grolaux et al.,2002]

12 ICAS’2008 – Gosier, March 16-21, 2008 Case study Dialogue modeled through a Moore machine –16 alternative variations –6 resizing operations Observations –Visualization is hard to achieve

13 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

14 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

15 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

16 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

17 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

18 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

19 ICAS’2008 – Gosier, March 16-21, 2008 W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR W1 HH:MM W2 HH:MM:SS W3 HH:MM JJ/MM W4 HH:MM:SS JJ/MM/AAAA W5 HH:MM:SS DAY JJ/MM/AAAA W6 HH:MM:SS DAY JJ MONTH AAAA W7  W8 HH:MM  JJ/MM W9 HH:MM:SS  JJ/MM/AAAA W10 HH:MM:SS  DAY JJ/MM/AAAA W11 HH:MM:SS  DAY JJ MONTH AAAA W13  HH:MM:SS DAY JJ/MM/AAAA W12  HH:MM:SS JJ/MM/AAAA W14  HH:MM:SS DAY JJ MONTH AAAA W15  HH:MM:SS DAY JJ MONTH AAAA CALENDAR W16 HH:MM:SS  DAY JJ MONTH AAAA CALENDAR

20 ICAS’2008 – Gosier, March 16-21, 2008 Case study Dialogue modeled through a Mealy machine W2W1 W2 W1 2 / / / / / / / / 16 8

21 ICAS’2008 – Gosier, March 16-21, 2008 Case study Main idea: design one UI variation for one context at a time, connect related variations afterwards Horizontal screen resolution Vertical screen resolution Plasticity domain

22 ICAS’2008 – Gosier, March 16-21, 2008 Case study Main idea: design one UI variation for one context at a time, connect related variations afterwards

23 ICAS’2008 – Gosier, March 16-21, 2008 Tool support: PlastiXML Graphical editor for plasticity domains –Direct manipulation of plasticity domains (here, rectangles) –Assignment of a UI variation to each plasticity domain –Automated generation of UsiXML for Layout Navigation between variations <ScreenSizeAspect id="scrd1" shape="rectangle" corners="{(80,10);(80,30);(ScreenSizeXLimit,10); (ScreenSizeXLimit,30)}" allowedOperations="{vertical shrinkage}"/> <ScreenSizeAspect id="scrd2" shape="polygon" corners="{(80,30);(ScreenSizeXLimit,30); (ScreenSizeXLimit,60);(100,60); (100,ScreenSizeYLimit); (80,ScreenSizeYLimit)}" allowedOperations="{vertical shrinkage}"/>

24 ICAS’2008 – Gosier, March 16-21, 2008 Scenario First plasticity domain

25 ICAS’2008 – Gosier, March 16-21, 2008 Scenario Second plasticity domain

26 ICAS’2008 – Gosier, March 16-21, 2008 Scenario Third plasticity domain

27 ICAS’2008 – Gosier, March 16-21, 2008 Scenario Final example

28 ICAS’2008 – Gosier, March 16-21, 2008 Conclusion Advantages –One UI variation is assigned to one context (here, a resolution) at a time –Connect related variations afterwards –Visual design of connected variations insted of separate (unrelated) design –Automated generation of UsiXML specifications Presentation Navigation –Mealy machine –Moore machine Limitations –Apart from copy/paste, no direct reuse of components from one UI variation to another –No factoring out of common components Possible extension –Generate a default UI variation to feed the various resolutions to be supported –Edit afterwards Generalization needed

29 ICAS’2008 – Gosier, March 16-21, 2008 Thank you very much for your attention For more information and downloading, User Interface eXtensible Markup Language European network on Multimodal UIs Special thanks to all members of the team!