Presentation is loading. Please wait.

Presentation is loading. Please wait.

Direct Manipulation of User Interfaces for Migration J. P. Molina 1,2, J. Vanderdonckt 1 and P. González 2 1 Université catholique de Louvain 2 University.

Published byHarvey Booker Modified over 9 years ago

Similar presentations

Presentation on theme: "Direct Manipulation of User Interfaces for Migration J. P. Molina 1,2, J. Vanderdonckt 1 and P. González 2 1 Université catholique de Louvain 2 University."— Presentation transcript:

1 Direct Manipulation of User Interfaces for Migration J. P. Molina 1,2, J. Vanderdonckt 1 and P. González 2 1 Université catholique de Louvain 2 University of Castilla-La Mancha www.usixml.org Intelligent User Interfaces 2006 Jan 29 - Feb 1 2005, Sydney, Australia

2 January 29 - February 1 2Intelligent User Interfaces 2006, Sydney, Australia Contents Introduction Introduction MigriXML MigriXML Case study Case study Migration process in detail Migration process in detail Conclusion Conclusion

3 January 29 - February 1 3Intelligent User Interfaces 2006, Sydney, Australia Contents Introduction Introduction MigriXML MigriXML Case study Case study Migration process in detail Migration process in detail Conclusion Conclusion

4 January 29 - February 1 4Intelligent User Interfaces 2006, Sydney, Australia Introduction End users of modern interactive systems face a wide variety of computing platforms to support their interactive tasks, from mobile phones to wall screens End users of modern interactive systems face a wide variety of computing platforms to support their interactive tasks, from mobile phones to wall screens The specific needs of the user’s environment at a given moment, or its evolution, can make necessary to remove one application from one platform and make it run in another one, that is, to migrate the application The specific needs of the user’s environment at a given moment, or its evolution, can make necessary to remove one application from one platform and make it run in another one, that is, to migrate the application

5 January 29 - February 1 5Intelligent User Interfaces 2006, Sydney, Australia Introduction Migration can be: Migration can be: Total, when the whole application is migrated Total, when the whole application is migrated Partial, only the UI is migrated; if it is decomposed into control and presentation, then migration can be: Partial, only the UI is migrated; if it is decomposed into control and presentation, then migration can be: Control-oriented, migrates only the control component Control-oriented, migrates only the control component Presentation-oriented, the control remains Presentation-oriented, the control remains Mixed, parts of both control and presentation are migrated Mixed, parts of both control and presentation are migrated A user interface (UI) is said to be migratable if it holds the migration ability A user interface (UI) is said to be migratable if it holds the migration ability

6 January 29 - February 1 6Intelligent User Interfaces 2006, Sydney, Australia Introduction A special UI is required for controlling the run- time migration of the UI of the interactive systems A special UI is required for controlling the run- time migration of the UI of the interactive systems This UI is referred to as the meta-user interface, implemented in very different ways: This UI is referred to as the meta-user interface, implemented in very different ways: Sytem-initiated, the system initiates the migration Sytem-initiated, the system initiates the migration User-initiated, the user initiates the migration User-initiated, the user initiates the migration Mixed-initiated, user and system collaborate to perform the migration Mixed-initiated, user and system collaborate to perform the migration

7 January 29 - February 1 7Intelligent User Interfaces 2006, Sydney, Australia Introduction Related work Related work MigrationInitiated by app user interface systemuser controlpresentation Bharat and Cardelli, 95 Song, Chu and Kurakake, 02 MIGRATION Project Bandelloni et al., 04 CamNote Balme et al., 04 Drag&Drop, Drag&Pick Baudish et al., 03 Aura de Sousa and Garlan, 02

8 January 29 - February 1 8Intelligent User Interfaces 2006, Sydney, Australia Introduction Elements represented in in the meta-UI Elements represented in in the meta-UI Representation of source platform target platform space between Detachable Interfaces Grolaux et al., 2004 Pick&Drop Rekimoto, 97 Augmented Surfaces Rekimoto and Masanori, 99 physical Proximal Interactions Rekimoto et al., 03 screenshot ARIS Biehl and Baily, 04 icon 2D map

9 January 29 - February 1 9Intelligent User Interfaces 2006, Sydney, Australia Introduction Tasks are usually distributed among platforms, but without any consistent way to distribute them, the space between platforms introduces a discontinuity that may disrupt the fluency of the interactive task Tasks are usually distributed among platforms, but without any consistent way to distribute them, the space between platforms introduces a discontinuity that may disrupt the fluency of the interactive task Direct manipulation is proposed to tackle that discontinuity, and thus avoid possible usability problems Direct manipulation is proposed to tackle that discontinuity, and thus avoid possible usability problems

10 January 29 - February 1 10Intelligent User Interfaces 2006, Sydney, Australia Contents Introduction Introduction MigriXML MigriXML Case study Case study Migration process in detail Migration process in detail Conclusion Conclusion

11 January 29 - February 1 11Intelligent User Interfaces 2006, Sydney, Australia MigriXML In order to make user interface migration as natural as direct manipulation, a meta-UI in the form of a virtual environment has been developed In order to make user interface migration as natural as direct manipulation, a meta-UI in the form of a virtual environment has been developed MigriXML enables the user to carry out that migration process without the physical presence of the involved platforms, in a virtual and interactive way, relaying on UsiXML language and a set of models MigriXML enables the user to carry out that migration process without the physical presence of the involved platforms, in a virtual and interactive way, relaying on UsiXML language and a set of models

12 January 29 - February 1 12Intelligent User Interfaces 2006, Sydney, Australia MigriXML To generate the virtual reality scene representing the migration environment, a suite of models is used describing relevant aspects of the problem in terms of UsiXML: To generate the virtual reality scene representing the migration environment, a suite of models is used describing relevant aspects of the problem in terms of UsiXML: The Concrete User Interface The Concrete User Interface The User Model The User Model The Platform Model The Platform Model The Environment Model The Environment Model Describes properties of interest, which can be physical, psychological and organizational Describes properties of interest, which can be physical, psychological and organizational

13 January 29 - February 1 13Intelligent User Interfaces 2006, Sydney, Australia MigriXML: Environment Model The physical part consists of: The physical part consists of: a scene model based on VRML97/X3D, a scene model based on VRML97/X3D, the multi-surface interaction ontology, the multi-surface interaction ontology, and the Stanford topology model. and the Stanford topology model. Each scene is composed of physical resources and a series of planes, which are decomposed into areas Each scene is composed of physical resources and a series of planes, which are decomposed into areas Each area may be an interaction surface: a monitor on a table, a wall screen, or any area where the UI is projected and/or recognized with computer vision Each area may be an interaction surface: a monitor on a table, a wall screen, or any area where the UI is projected and/or recognized with computer vision Each area is described by attributes like dimensions, relative position, relationships with other objects, etc Each area is described by attributes like dimensions, relative position, relationships with other objects, etc

14 January 29 - February 1 14Intelligent User Interfaces 2006, Sydney, Australia MigriXML architecture Interactive system architecture Interactive system architecture GUI application GUI application Arch meta-model or ‘Slinky’ meta-model Arch meta-model or ‘Slinky’ meta-model Platform Platform Layered structure Layered structure

15 January 29 - February 1 15Intelligent User Interfaces 2006, Sydney, Australia MigriXML architecture General MigriXML architecture General MigriXML architecture User’s environment User’s environment GUI application GUI application Arch meta-model fits the UsiXML models Arch meta-model fits the UsiXML models Platform Platform Virtual Hardware (VHW) Virtual Hardware (VHW) Virtual Machine (VM) Virtual Machine (VM) Virtual Window Manager (VWM) Virtual Window Manager (VWM) Migration Manager (MM) Migration Manager (MM)

16 January 29 - February 1 16Intelligent User Interfaces 2006, Sydney, Australia Implementation This work focuses on the original part of providing a virtual control environment for migrating parts or whole of a UI from one platform to another This work focuses on the original part of providing a virtual control environment for migrating parts or whole of a UI from one platform to another To address its implementation, the VRML97/X3D languages have been used To address its implementation, the VRML97/X3D languages have been used User Interfaces and the Arch meta-model: User Interfaces and the Arch meta-model: Interaction toolkit component: VUIToolkit Interaction toolkit component: VUIToolkit Domain-specific component: Java and Javascript Domain-specific component: Java and Javascript

17 January 29 - February 1 17Intelligent User Interfaces 2006, Sydney, Australia Implementation VUIToolkit VUIToolkit A set of VRML97/X3D PROTOs that transforms the standard plain widgets into a truly 3D representation (Virtualized User Interface) A set of VRML97/X3D PROTOs that transforms the standard plain widgets into a truly 3D representation (Virtualized User Interface) This toolkit was developed starting from the object classes described in the Concrete Model of UsiXML language This toolkit was developed starting from the object classes described in the Concrete Model of UsiXML language

18 January 29 - February 1 18Intelligent User Interfaces 2006, Sydney, Australia Implementation Java and Javascript Java and Javascript Most VRML97/X3D browsers can interpret Javascript code and execute Java code Most VRML97/X3D browsers can interpret Javascript code and execute Java code VRML97/X3D specifications describe two programming interfaces to access the scene graph: VRML97/X3D specifications describe two programming interfaces to access the scene graph: SAI (Scene Authoring Interface) SAI (Scene Authoring Interface) EAI (External Authoring Interface) EAI (External Authoring Interface) Allowed us to leave aside, for this first implementation of MigriXML, the complex integration of emulators for each specific platform Allowed us to leave aside, for this first implementation of MigriXML, the complex integration of emulators for each specific platform

19 January 29 - February 1 19Intelligent User Interfaces 2006, Sydney, Australia Implementation Interactive Java/Javascript application, virtualized user interface rendered in the 3D window of VRML97/X3D browser Interactive Java/Javascript application, virtualized user interface rendered in the 3D window of VRML97/X3D browser

20 January 29 - February 1 20Intelligent User Interfaces 2006, Sydney, Australia Implementation Interactive application, virtualized user interface and virtual platform in MigriXML Interactive application, virtualized user interface and virtual platform in MigriXML

21 January 29 - February 1 21Intelligent User Interfaces 2006, Sydney, Australia Contents Introduction Introduction MigriXML MigriXML Case study Case study Migration process in detail Migration process in detail Conclusion Conclusion

22 January 29 - February 1 22Intelligent User Interfaces 2006, Sydney, Australia Case study The user’s environment is a small office, with five different platforms The user’s environment is a small office, with five different platforms

23 January 29 - February 1 23Intelligent User Interfaces 2006, Sydney, Australia Case study Display characteristics of each platform Display characteristics of each platform

24 January 29 - February 1 24Intelligent User Interfaces 2006, Sydney, Australia Case study UsiXML was used to describe the environment UsiXML was used to describe the environment The virtual world was the result of a structured process of creation: The virtual world was the result of a structured process of creation: Requirements: from UsiXML specifications, designers knew the details of the environment Requirements: from UsiXML specifications, designers knew the details of the environment Preparation: compilation of audio-visual material; sketches and mock-ups were created Preparation: compilation of audio-visual material; sketches and mock-ups were created Design, test and optimize: model creation, edit appearance, enhance world Design, test and optimize: model creation, edit appearance, enhance world

25 January 29 - February 1 25Intelligent User Interfaces 2006, Sydney, Australia Case study User interfaces were specified in UsiXML too User interfaces were specified in UsiXML too Different tools have been developed to help the UI designer to visually create this models Different tools have been developed to help the UI designer to visually create this models IdealXML [Montero et al., 05] IdealXML [Montero et al., 05]

26 January 29 - February 1 26Intelligent User Interfaces 2006, Sydney, Australia Case study User interfaces were specified in UsiXML too User interfaces were specified in UsiXML too Different tools have been developed to help the UI designer to visually create this models Different tools have been developed to help the UI designer to visually create this models IdealXML IdealXML GrafiXML [B. Michotte] GrafiXML [B. Michotte]

27 January 29 - February 1 27Intelligent User Interfaces 2006, Sydney, Australia Case study Two different user interfaces Two different user interfaces

28 January 29 - February 1 28Intelligent User Interfaces 2006, Sydney, Australia Case study Migrating from one laptop to the other Migrating from one laptop to the other 1 2 34 Aspire 2000 15”, 16:10 1280x800 PT-LB10SU 4:3 800x600 Latitude 15”, 4:3 1024x768

29 January 29 - February 1 29Intelligent User Interfaces 2006, Sydney, Australia Case study Migrating from a laptop to the Pocket PC Migrating from a laptop to the Pocket PC 1 2 3 Aspire 2000 15”, 16:10 1280x800 e750 3,8”, 3:4 240x320

30 January 29 - February 1 30Intelligent User Interfaces 2006, Sydney, Australia Contents Introduction Introduction MigriXML MigriXML Case study Case study Migration process in detail Migration process in detail Conclusion Conclusion

31 January 29 - February 1 31Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail The migration process is divided in four stages (A, B, C and D), each one having a finite set of steps, representing a total amount of 14 steps The migration process is divided in four stages (A, B, C and D), each one having a finite set of steps, representing a total amount of 14 steps

32 January 29 - February 1 32Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage A: Select an interactive application Stage A: Select an interactive application

33 January 29 - February 1 33Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage A: Select an interactive application Stage A: Select an interactive application The user presses button (M) to grab the window, which follows the user wherever she points at

34 January 29 - February 1 34Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage A: Select an interactive application Stage A: Select an interactive application The action is transmitted to the corresponding Virtual Window Manager (VWM) by an event

35 January 29 - February 1 35Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage A: Select an interactive application Stage A: Select an interactive application The VWM forwards the information to the Migration Manager (MM)

36 January 29 - February 1 36Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage B: Select target platform Stage B: Select target platform

37 January 29 - February 1 37Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage B: Select target platform Stage B: Select target platform The MM broadcasts a message to all platforms

38 January 29 - February 1 38Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage B: Select target platform Stage B: Select target platform Platforms change their state to ‘wait-for-selection’, a TouchSensor is activated

39 January 29 - February 1 39Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage B: Select target platform Stage B: Select target platform The user selects the target platform by clicking on its screen

40 January 29 - February 1 40Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage B: Select target platform Stage B: Select target platform The Virtual Hardware (VHW) casts an event to inform the VWM

41 January 29 - February 1 41Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage B: Select target platform Stage B: Select target platform The VWM forwards the message to the MM

42 January 29 - February 1 42Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage C: Migrate to target platform Stage C: Migrate to target platform

43 January 29 - February 1 43Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage C: Migrate to target platform Stage C: Migrate to target platform The MM commands to detach the application from source platform

44 January 29 - February 1 44Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage C: Migrate to target platform Stage C: Migrate to target platform The MM broadcast a message to the rest of platforms, which change to normal execution 10

45 January 29 - February 1 45Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage C: Migrate to target platform Stage C: Migrate to target platform The MM takes the UsiXML specification and re- generates the app UI

46 January 29 - February 1 46Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage D: Adapt to target platform Stage D: Adapt to target platform

47 January 29 - February 1 47Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage D: Adapt to target platform Stage D: Adapt to target platform The MM asks the target platform to host the re- generated application

48 January 29 - February 1 48Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage D: Adapt to target platform Stage D: Adapt to target platform The application is launched, and its virtualized UI added to the scene graph of the target VHW

49 January 29 - February 1 49Intelligent User Interfaces 2006, Sydney, Australia Migration process in detail Stage D: Adapt to target platform Stage D: Adapt to target platform The virtualized UI is adapted to the target screen and windowing system

50 January 29 - February 1 50Intelligent User Interfaces 2006, Sydney, Australia Contents Introduction Introduction MigriXML MigriXML Case study Case study Migration process in detail Migration process in detail Conclusion Conclusion

51 January 29 - February 1 51Intelligent User Interfaces 2006, Sydney, Australia Conclusion MigriXML and previous work MigriXML and previous work MigrationInitiated by app user interface systemuser controlpresentation Bharat and Cardelli, 95 Song, Chu and Kurakake, 02 MIGRATION Project Bandelloni et al., 04 CamNote Balme et al., 04 Drag&Drop, Drag&Pick Baudish et al., 03 Aura de Sousa and Garlan, 02 MIGRIXML

52 January 29 - February 1 52Intelligent User Interfaces 2006, Sydney, Australia Conclusion MigriXML and previous representations MigriXML and previous representations Representation of source platform target platform space between Detachable Interfaces Grolaux et al., 2004 Pick&Drop Rekimoto, 97 Augmented Surfaces Rekimoto and Masanori, 99 physical Proximal Interactions Rekimoto et al., 03 screenshot ARIS Biehl and Baily, 04 icon 2D map MIGRIXML 3D virtual

53 January 29 - February 1 53Intelligent User Interfaces 2006, Sydney, Australia Conclusion MigriXML highlights: MigriXML highlights: As an environment for simulating migration, there is no need to have the physical device to carry on the process of adapting a previous UI to a new platform As an environment for simulating migration, there is no need to have the physical device to carry on the process of adapting a previous UI to a new platform As a meta-UI for migration, it could also render remote spaces, moving us forward tele-presence applications As a meta-UI for migration, it could also render remote spaces, moving us forward tele-presence applications In both cases, all involved elements are graphically rendered and the migration is continuously depicted during the process, avoiding disruption in the user’s task, and thus improving usability In both cases, all involved elements are graphically rendered and the migration is continuously depicted during the process, avoiding disruption in the user’s task, and thus improving usability

54 January 29 - February 1 54Intelligent User Interfaces 2006, Sydney, Australia Conclusion As future work, evaluations are desirable: As future work, evaluations are desirable: On the one hand, expert and user evaluation in order to assess the influence of the meta-display problem, that is, how the resolution of the display that renders the meta-UI can affect utility and usability of this approach On the one hand, expert and user evaluation in order to assess the influence of the meta-display problem, that is, how the resolution of the display that renders the meta-UI can affect utility and usability of this approach On the other hand, summative evaluation with other meta-UI representations, such as the iconic 2D map of ARIS, in order to assess which approach is better for a given scenario On the other hand, summative evaluation with other meta-UI representations, such as the iconic 2D map of ARIS, in order to assess which approach is better for a given scenario

55 Thank you Mercí Gracias More info: www.usixml.org Contact: jpmolina@info-ab.uclm.es

Download ppt "Direct Manipulation of User Interfaces for Migration J. P. Molina 1,2, J. Vanderdonckt 1 and P. González 2 1 Université catholique de Louvain 2 University."

Similar presentations

CONCEPTUAL WEB-BASED FRAMEWORK IN AN INTERACTIVE VIRTUAL ENVIRONMENT FOR DISTANCE LEARNING Amal Oraifige, Graham Oakes, Anthony Felton, David Heesom, Kevin.

CONCEPTUAL WEB-BASED FRAMEWORK IN AN INTERACTIVE VIRTUAL ENVIRONMENT FOR DISTANCE LEARNING Amal Oraifige, Graham Oakes, Anthony Felton, David Heesom, Kevin.
Structured Design The Structured Design Approach (also called Layered Approach) focuses on the conceptual and physical level. As discussed earlier: Conceptual.

Structured Design The Structured Design Approach (also called Layered Approach) focuses on the conceptual and physical level. As discussed earlier: Conceptual.
Technical and design issues in implementation Dr. Mohamed Ally Director and Professor Centre for Distance Education Athabasca University Canada New Zealand.

Technical and design issues in implementation Dr. Mohamed Ally Director and Professor Centre for Distance Education Athabasca University Canada New Zealand.
An Overview Of Virtual Machine Architectures Ross Rosemark.

An Overview Of Virtual Machine Architectures Ross Rosemark.
Korea Univ. Division Information Management Engineering UI Lab. Korea Univ. Division Information Management Engineering UI Lab. S E M I N A R Predictive.

Korea Univ. Division Information Management Engineering UI Lab. Korea Univ. Division Information Management Engineering UI Lab. S E M I N A R Predictive.
Université catholique de Louvain (UCL) Belgian Laboratory of Computer-Human Interaction (BCHI) Place des Doyens, 1 B-1348 Louvain-la-Neuve (Belgium) Presented.

Université catholique de Louvain (UCL) Belgian Laboratory of Computer-Human Interaction (BCHI) Place des Doyens, 1 B-1348 Louvain-la-Neuve (Belgium) Presented.
1http://img.cs.man.ac.uk/stevens Interaction Models of Humans and Computers CS2352: Lecture 7 Robert Stevens

1http://img.cs.man.ac.uk/stevens Interaction Models of Humans and Computers CS2352: Lecture 7 Robert Stevens
Software. Application Software performs useful work on general-purpose tasks such as word processing and data analysis. The user interacts with the application.

Software. Application Software performs useful work on general-purpose tasks such as word processing and data analysis. The user interacts with the application.
© Prentice Hall 2002 3.1 CHAPTER 3 Computer Software.

© Prentice Hall CHAPTER 3 Computer Software.
Introduction to HCI Marti Hearst (UCB SIMS) SIMS 213, UI Design & Development January 21, 1999.

Introduction to HCI Marti Hearst (UCB SIMS) SIMS 213, UI Design & Development January 21, 1999.
Chapter 2 Introduction to Systems Architecture. Chapter goals Discuss the development of automated computing Describe the general capabilities of a computer.

Chapter 2 Introduction to Systems Architecture. Chapter goals Discuss the development of automated computing Describe the general capabilities of a computer.
Introduction to the course January 9, 2012. Points to Cover  What is GIS?  GIS and Geographic Information Science  Components of GIS Spatial data.

Introduction to the course January 9, Points to Cover  What is GIS?  GIS and Geographic Information Science  Components of GIS Spatial data.
Microsoft Visual Basic 2012 CHAPTER ONE Introduction to Visual Basic 2012 Programming.

Microsoft Visual Basic 2012 CHAPTER ONE Introduction to Visual Basic 2012 Programming.
A Computational Framework for Multi-dimensional Context- aware Adaptation Vivian Genaro Motti LILAB – Louvain Interaction Laboratory Université catholique.

A Computational Framework for Multi-dimensional Context- aware Adaptation Vivian Genaro Motti LILAB – Louvain Interaction Laboratory Université catholique.
TC2-Computer Literacy Mr. Sencer February 8, 2010.

TC2-Computer Literacy Mr. Sencer February 8, 2010.
Sikuli Ivailo Dinkov QA Engineer PhoneX Team Telerik QA Academy.

Sikuli Ivailo Dinkov QA Engineer PhoneX Team Telerik QA Academy.
An Approach to Task Modelling for User Interface Design Costin Pribeanu National Institute for Research and Development in Informatics, Bucureşti, Romania.

An Approach to Task Modelling for User Interface Design Costin Pribeanu National Institute for Research and Development in Informatics, Bucureşti, Romania.
The Old World Meets the New: Utilizing Java Technology to Revitalize and Enhance NASA Scientific Legacy Code Michael D. Elder Furman University Hayden.

The Old World Meets the New: Utilizing Java Technology to Revitalize and Enhance NASA Scientific Legacy Code Michael D. Elder Furman University Hayden.
1 ICAS’2008 – Gosier, March 16-21, 2008 A Transformational Approach for Pattern-based Design of User Interfaces Costin Pribeanu Jean Vanderdonckt National.

1 ICAS’2008 – Gosier, March 16-21, 2008 A Transformational Approach for Pattern-based Design of User Interfaces Costin Pribeanu Jean Vanderdonckt National.
CHAPTER FOUR COMPUTER SOFTWARE.

CHAPTER FOUR COMPUTER SOFTWARE.

Similar presentations

Ads by Google