© TAMODIA’061 The Comets Inspector: Towards Run Time Plasticity Control based on a Semantic Network Alexandre Demeure, Gaëlle Calvary, Joelle Coutaz, Jean Vanderdonckt Université de Grenoble, France Université catholique de Louvain, Belgium

© TAMODIA’062 The problem UI plasticity: adaptation of a UI –to its context of use –while preserving usability E.g: Choice of a month - Distinctive criteria - Various - Variable - Unforeseeable Task NavigationGenericity

© TAMODIA’063 Objectives Provide models, methods and tools for –Describing and capitalizing UIs In an extensible way incorporating tailored UIs –Supporting “plasticity” questions both at design and run time Which UIs are HTML versions equivalent to this UI? Is this UI decomposable into smaller ones? Is there a tailored UI for choosing a month? (pattern?) What is the common functionality supported by these 2 UIs? …

© TAMODIA’064 State of the art Description Domain & Task Abstract UI Concrete UI Final UI ||| with navigation DS0 DSi ||| Cond Choice in a known set Type: a TYPE S_poss: set of Type S_eff: set of Type min, max : Integer Constraints: S_eff  S_poss #S_eff  [min; max] User Task: Specify S_eff Linear ||| A SPACE name A SPACE A SPACE name A SPACE ScrollList Pie ||| Languages XAML, XUL, … UsiXML, …

© TAMODIA’065 State of the art Capitalization –Taxonomy of interactors Rarely explicit Abstraction levels not explicit Not easily extensible –Semantic network “a graphic notation for representing knowledge in patterns of interconnected nodes and arcs. Computer implementations of semantic networks were first developed for artificial intelligence and machine translation, but earlier versions have long been used in philosophy, psychology, and linguistics” (Sowa) Explicit the UI meta models Appropriate for extension Easy to represent abstraction levels

© TAMODIA’066 Approach Based on a semantic network for –Capitalizing UIs –Supporting plasticity both at design and run time Semantic network for plasticity –Node  UI models –Edge(arcs)  Relationships between UI models

© TAMODIA’067 Semantic network for plasticity Nodes –Describe an interactive system at any level of abstraction Relationships –Describe how a node is related to another one C&TAUICUI FUI : Is deduced from Choice in a known set Choice of months

© TAMODIA’068 Semantic network for plasticity Choice at C&T level Choice in a known set Type: a TYPE S_poss: set of Type S_eff: set of Type min, max : Integer Constraints: S_eff  S_poss #S_eff  [min; max] User Task: Specify S_eff Simple choice Constraints: min = max = 1 Restriction of partial/non exclusive Choice a month Constraints: Type = MONTH |||Marks * ||| S_IS: set of OBJECT Encapsulates Marker Type: a TYPE Value: TYPE Obj: OBJECT Specializes partial/exclusive Choice by ||| and marks Constraints:  m:Marks()  m.Type = Bool #Marks() = #L_poss U{m:Marks() | m.Value = true  m.Obj} = L_eff U{m:Marks() | m.Value = false  m.Obj} = L_poss \ L_eff |||().S_IS = Marks() Accumulator Constraints: Eff().Type = Poss().Type Eff().S_poss = S_eff Poss().S_poss = S_poss\S_eff If Poss().TaskDone() then L_eff U= Poss().L_eff If Eff().TaskDone() then L_eff() \= Eff().L_eff

© TAMODIA’069 Semantic network for plasticity Interleaving at AUI level ||| S_IS: set of OBJECT ||| without navigation DS0 DSi ||| ||| with navigation DS0 DSi ||| Cond Specializes total / exclusive Cont * Container Dialog level Constraints: Represents(DS, S_IS)) DS Encapsulates Concretizes partial/non exclusive ||| Dialog level Constraints: DS = {c:Cont()  c.DS}  is:S_IS  (  c:Cont() | c.S_IS = {is})  c:Cont()  (  is:S_IS | c.S_IS = {is}) ||| DS0 DSi ||| monospace Constraints: Cond  #{i:Integer|DSi.active}=1 Specializes partial / non exclusive ||| sequence Constraints: Cond   i1,i2,i3:Integer | i1<i2<i3  DSi1.active   DSi2.active  DSi3.active ||| sequential access Constraints: Cond   i:Integer | DSi.active at t   DSi.active at t-1   DSi-1.active at t-1   DSi+1.active at t-1

© TAMODIA’0610 Plasticity questions

© TAMODIA’0611 Plasticity questions

© TAMODIA’0612 Plasticity questions

© TAMODIA’0613 Plasticity questions

© TAMODIA’0614 Plasticity questions Is there a choice tailored for months? ?n : Choice : NODE()<-REL(type~=GDD_inheritance)* <REL(type==GDD_restriction&&constraint~=“type==MONTH”)<-$n()

© TAMODIA’0615 Plasticity questions Are there pie menu equivalent UIs without navigation? ?n : InterleavingNoNav : Node()<-REL(type==GDD_inheritance)*<-$n()

© TAMODIA’0616 Plasticity questions What are the possible TK choices? ?n : Choice : NODE()<-REL(type~=GDD_inheritance)*<REL(type==GDD_implementation)<-$n() Cartesian product on Marker [TK] and Interleaving [TK]

© TAMODIA’0617 Demonstrator Comet Inspector –Comet = Polymorphic interactors

© TAMODIA’0618 Conclusion Contribution : a semantic network that –supports plasticity questions both at design and run-time –is opened to tailored UIs –Can be located inside and outside interactors Strength for plasticity –Easy to extend: Just add nodes and edges –Enlarges plasticity without extra programmation : The effort is set on the exploration mechanisms –Preserves plasticity questions Drawbacks –People must agree on a common vocabulary and structure –Who controls the network?

© TAMODIA’0619 Perspectives Theoretical –Agreement on The UI levels of abstraction The UI relationships The structure of the semantic network –Language to describe UIs and UI transformations –Language to build plasticity queries –How to include patterns Practical –Semantic network server –Visualization techniques for guiding the user (designer and/or final user) META-UI –Integration to Comets (plastic interactors) –Usability of the approach

© TAMODIA’0620 Thanks QUESTIONS?

© TAMODIA’0621 Semantic network for plasticity Who use? –Conceptors design time : Build different version Run-time : Express substitution –Users Personalize the interface (change this radio button choice with…) Who extend? –Conceptors : new systems or new way to present –Users : End user programming (on the fly application building)

© TAMODIA’0622 Semantic network for plasticity Nodes –Describe an interactive system at any level of abstraction Example; Choice at the C&T level C&TAUICUI FUI : Is deduced from Choice in a known set Type: a TYPE S_poss: set of Type S_eff: set of Type min, max : Integer Constraints: S_eff  S_poss #S_eff  [min; max] User Task: Specify S_eff

© TAMODIA’0623 Y is deduced from X Semantic network for plasticity Relationships –Describe how an interactive system is related to an other. x and y are nodes x has a X level of abstraction Y has a Y level of abstraction It can be total/partial Exclusive or not

© TAMODIA’0624 Plasticity questions

© TAMODIA’0625 Semantic network for plasticity Tailored UI –Capitalizing the know-how

© TAMODIA’0626 Semantic network for plasticity What are all implementation of a choice? Is there a choice tailored for months? What is the closest version of a TK pie-menu in HTML? What is the greatest common functionality between this 2 systems? What are equivalents to pie menu ensuring no navigation? Could this system be decomposed into smaller ones? What are the possible TK choices for choosing a month?