Modeling Tangible User Interfaces Hi, I would like to share with you this afternoon a framework we are currently developing for Reality-Based interfaces. My interest in RBI, began after I experienced their ease of use and intuitive nature. As I started developing them, I realized that hteir development was challenging as there was no comprehensive toolset and framework for building them. Comp150 TUI Spring 2006

The Design Space of TUIs Designers’ Outpost [Klemmer et al.2001] ComTouch [Jacob et al. 2001] Senseboard [Jacob et al. 2001] Media Blocks [Ullmer et al. 2003] Last week we discussed a variety of interfaces. Some of the examples are here as reminders. These interfaces are very different in their structure, functionality, interaction techniques and implementation technology. Today we’ll discuss a modeling language that will help us describe their structure and functionaly in a concise way. Navigation Blocks [Camarata et al. 2002] Tangible Query Interfaces [Ullmer et al. 2003] Comp150 TUI Spring 2006

Why to Model? Highlight system properties: observability, predicatability, modes etc. Expose design issues Compare alternative designs. Communicate design to SW engineers Communicate design to ‘customers’ Design documentation Generate code from technology independent spec. Integrate novel technologies Analysis Communication First, why do we need Implementation Comp150 TUI Spring 2006

A Framework for modeling TUIs Token and Constraints (TAC) Paradigm Tokens: Physical objects that represent digital information Constraints: Physical objects that constraint the manipulation of tokens by: Suggesting how to manipulate a token Physically constraining Providing a frame of reference TAC: A relationship between a token and a set of constraints which encapsulates a set of manipulation actions To describe these interfaces we first need to recognize a common set of constructs: what are the widgets of TUIs? After a thorough investigation we believe these constructs can be used to describe a large set of tUIs. Comp150 TUI Spring 2006

Describing a TUI using TUIVIS interaction 3 dialogue association 1 2 TAC Palette Dialogue diagram Task diagram Comp150 TUI Spring 2006

Graphical Representation of Tokens and Constraints surface rack indentation knob slider connector Tokens Constraints Comp150 TUI Spring 2006

Combining Tokens and Constraints into TACs Create a TAC palette for UrP Comp150 TUI Spring 2006

The TAC Palette 1 2 Comp150 TUI Spring 2006 TAC Representation Association Manipulation Variable Token Constraint TAC graphics Action Response 1 building building model surface other buildings Add displays shadow according to time. Remove Removes related info from display move Updates display 2 distance Distance tool Two buildings Surface Displays distance remove Hides distance Comp150 TUI Spring 2006

The TAC Palette 1 2 3 Comp150 TUI Spring 2006 TAC Representation Association Manipulation Variable Token Constraint TAC graphics Action Response 1 building building model surface other buildings Add displays shadow according to time. Remove Removes related info from display move Updates display 2 distance Distance tool Two buildings Surface Displays distance remove Hides distance 3 wind simulation Wind tool Buildings Displays wind Hides wind Updates wind Comp150 TUI Spring 2006

Describing the Tangible Dialogue High level states Transitions Element Description Internal state A vector of the current values of application variables Physical state A vector of instantiated TAC relationships Tasks A set of tasks that could be completed within this state. These tasks can be performed either sequentially or in parallel depends on the satisfaction of their preconditions wind move mater.ial orien.t time dist. Source Representation Timer System User Interaction Tacit Interaction Trigger Comp150 TUI Spring 2006

Constructing the Dialogue Diagram What is the initial state of the system? What tasks are users willing to accomplish while interacting with the URP interface? Start from the start state and repeat until you can’t find new states: What event change the set of tasks users can perform in this state? Create a new state, describe the new set of tasks. Comp150 TUI Spring 2006

Dialogue Diagram Comp150 TUI Spring 2006 UIDL description wind move time mater. orien. addBuilding /shadow(T) removeBuilding /shadow(F) numOfBuilding=2 removeBuilding numOfBuilding>2 dist. UIDL description As we discussed our UIDL is based on a two-tier model that separates reality based interaction to two level: dialogue and task. The dialogue level is sequential. It is inspired from storyboarding techniques and provides an overview of the interaction structure. It is represented as an enhanced state diagram and uses a notation based on the Statechart specification language. While the dialogue level, describes which tasks can be performed in parallel, the task level: describes how many instance of the same task can be performed in parallel. The Task diagram is inspired from colored Petri nets, a formalism for specifying concurrency: It consists of: a set of actions that comprise a task A set of places of type…. Which describes the resources needed for the executions of each action, as well as the digital and physical state of the system before and after the execution of an action. A set of flow relations between laces and actions, and finally a set of continuous relationships. Next, we will introduce the visual notation Comp150 TUI Spring 2006

Task Diagram URP - distance measuring The Task diagram is inspired from petri nets , a modling technique for parallel interacting components. Comp150 TUI Spring 2006

Constructing a Task Diagram What is the physical state of the system prior to the first action aimed at accomplishing this task? What are the actions users perform toward accomplishing this task? What physical interaction objects are needed for each action? What is the physical state of the system after an action was performed? What is the internal state of the system after an action was performed? When does the system physical state goes back to its state prior to the execution of the first action? Comp150 TUI Spring 2006

Task Diagram URP - distance measuring The Task diagram is inspired from petri nets , a modling technique for parallel interacting components. Comp150 TUI Spring 2006

Summary interaction dialogue association TAC Palette Dialogue diagram 3 dialogue association 1 2 TAC Palette Dialogue diagram Task diagram Comp150 TUI Spring 2006

Project 1, Part 2 Analyzing TUIs What to submit: A TAC Palette A dialogue diagram At least two task diagrams Discussion of the following topics: Observability and predictability: Does the physical state of the system indicate to the user the internal state of the system? Is there information in the internal state that is not presented in the physical state? Does the physical state of the system contain enough information to enable the user to determine what tasks are available for him to perform? Assuming multiple users interact with the system at the same time. Does the physical state of the system indicate to users what tasks they can or cannot perform in parallel? If possible bring an example of two tasks that different users cannot perform in parallel. Modes Certain actions are meaningful only in certain contexts. Bring an example of a user action that is meaningless in one context and has a meaning in another. Physical Syntax Does the physical state of the system indicate to users which actions are legal/illegal? Comp150 TUI Spring 2006

Next week Introduction to technologies Reading Start working in teams on a technology demo Comp150 TUI Spring 2006

Designer’s Outpost [Klemmer 2001] (An Interactive Surface) Back Comp150 TUI Spring 2006

Tangible Query Interfaces [Ullmer 2003] (A Token+Constraints System) Back Comp150 TUI Spring 2006

ComTouch [Chang 2003] (A Haptic Interface) Back Comp150 TUI Spring 2006

Web Stickers (Paper Based) Comp150 TUI Spring 2006