1 Comp150 TUI Spring 2007 Modeling and Specifying Tangible User Interfaces

2 Comp150 TUI Spring 2007 Tangible User Interfaces - Design Space Designers’ Outpost [Klemmer et al.2001] Navigation Blocks [Camarata et al. 2002] Media Blocks [Ullmer et al. 2003] Senseboard [Jacob et al. 2001] ComTouch [Jacob et al. 2001] Tangible Query Interfaces [Ullmer et al. 2003] URP [Underkoffler and Ishii. 1999] Tangible Video Editor [Zigelbaum et al. 2005] PeoplePretzel Shaer et al. 2004] Quetzal [Horn et al. 2006]

3 Comp150 TUI Spring 2007 Why to Model?  Highlight system properties: observability, predicatability, access points, etc.  Expose design issues: what is physical what is digital, physical syntax, expected and unexpected interactions.  Explore and compare alternative designs.  Communicate design within multidisciplinary teams  Design documentation  Guide implementation  Identify boundary cases Analysis Communication Implementation

4 Comp150 TUI Spring 2007 TUIML: Tangible User Interface modeling Language Interaction diagram 3 TAC Palette 12 Dialogue diagram DialogueInteraction structurebehavior

5 Comp150 TUI Spring 2007  Tokens: Physical objects that represent digital information. Users interact with tokens to access and manipulate digital information.  Constraints: Physical objects that provide context to tokens manipulation by constraining the ways in which tokens can be manipulated. A constraint constrain the manipulation of a token by:  Suggesting how to manipulate a token  Physically constraining  Providing a frame of reference  TAC: A relationship between a token and a set of constraints. TACs encapsulate a set of manipulation actions that can be performed upon a token in respect to a set of constraints. Describing the Structure of a TUI using TUIML

6 Comp150 TUI Spring 2007 TUIML Representation of Tokens and Constraints ConstraintRepresentationConceptual RelationsAction/Manipulation SurfaceIdentity, presence, position (x,y,z), spatial relations, order, number, group, containment Add, remove, move, stack, line up, cluster, separate etc. RackIdentity, presence, position (x), spatial relations, order, number, proximity Add, remove, slide IndentationPresence, identityAdd, remove Knob Position ( θ ), identity Rotate Length slider Position ( x1,x2 ), identity Slide ConnectorConnection, identityConnect, disconnect TokensConstraints Attributes: Shape Color Size Texture

7 Comp150 TUI Spring 2007 Combining Tokens and Constraints into TACs m Building model Wind tool Message Play indentation Query Parameter

8 Comp150 TUI Spring 2007 URP [Underkoffler and Ishii 1999]  A TUI for urban planning.  Combines physical building models with physical interactive tools to help urban planners perform analysis of shadows, proximities, reflection, wind and visual space.  Physical interaction objects include:  Building models  A wind tool  A material wand  A clock dial  A distance measuring tool

9 Comp150 TUI Spring 2007 The TAC Palette TACRepresentationAssociationManipulation VariableTokenConstraintTAC graphicsActionResponse 1 buildingbuilding modelsurface other buildings Adddisplays shadow according to time. RemoveRemoves related info from display moveUpdates display 2 distanceDistance toolTwo buildings Surface AddDisplays distance removeHides distance m

10 Comp150 TUI Spring 2007 The TAC Palette TACRepresentationAssociationManipulation VariableTokenConstraintTAC graphicsActionResponse 1 buildingbuilding modelsurface other buildings Adddisplays shadow according to time. RemoveRemoves related info from display moveUpdates display 2 distanceDistance toolTwo buildings Surface AddDisplays distance removeHides distance 3 wind simulation Wind toolBuildings surface AddDisplays wind RemoveHides wind moveUpdates wind m m

11 Comp150 TUI Spring 2007 Describing Behaviors Using TUIML Interaction diagram 3 TAC Palette 12 Dialogue diagram DialogueInteraction structurebehavior

12 Comp150 TUI Spring 2007 Interaction Model for TUIs

13 Comp150 TUI Spring 2007 Dialogue diagram - URP wind mov e time mate r. orie n. addBuilding /shadow(T) addBuilding /shadow(T) remove Buildin g /shadow (F) addBuilding /shadow(F) removeBuilding numOfBuilding=2 /shadow(F) removeBuilding numOfBuilding>2 /shadow(F) wind mov e time mate r. orie n. dist.

14 Comp150 TUI Spring 2007 Task Diagram - URP Wind Simulation

15 Comp150 TUI Spring 2007 Analyzing URP  Observability and predictability: Does the physical state of the system indicate to the user the internal state of the system? Does the physical state of the system contain enough information to enable the user to determine what tasks are available for him to perform? Assuming multiple users interact with the system at the same time. Does the physical state of the system indicate to users what tasks they can or cannot perform in parallel?  Modes Certain actions are meaningful only in certain contexts. Bring an example of a user action that is meaningless in one context and has a meaning in another.  Physical Syntax Does the physical state of the system indicate to users which actions are legal ¥ illegal? Does the physical state of the system indicate to users how actions can be combined into sequences in order to perform a task?

16 Comp150 TUI Spring 2007 The Marble Answering Machine (MAM) (Durrel Bishoph 1992)

17 Comp150 TUI Spring 2007 MAM - TAC Palette

18 Comp150 TUI Spring 2007 MAM - Dialogue diagram

19 Comp150 TUI Spring 2007 MAM - Task diagrams Call backPlay

20 Comp150 TUI Spring 2007 Tangible Query Interfaces Ullmer, Ishii, Jacob 2003  Using physically constrained tokens to express, manipulate and visualize parameterized database queries.

21 Comp150 TUI Spring 2007 Tangible Queries - TAC Palette RepresentationAssociationManipulation TACVariableTokenConstraintsTAC GraphicsActionResponse 1Query Parameter (upper bound) Upper slider Bar, lower slider SlidesetUpperBound() 2Query Parameter (lower bound) Lower slider Bar, upper slider SlidesetLowerBound() 3Query parameter Parameter bar Rack Other bars Add Slide Remove displayQueryRes()

22 Comp150 TUI Spring 2007 Tangible Queries - Dialogue Diagram

23 Comp150 TUI Spring 2007 Tangible Queries - Task Diagrams Slide bar Adjust upper bound

24 Comp150 TUI Spring 2007 Summary TUIML Interaction diagram 3 TAC Palette 12 Dialogue diagram DialogueInteraction structurebehavior