1. Modeling Tangible User Interfaces
Hi,
I would like to share with you this afternoon a framework we are currently developing for Reality-Based interfaces.
My interest in RBI, began after I experienced their ease of use and intuitive nature.
As I started developing them, I realized that hteir development was challenging as there was no comprehensive toolset and framework for building them.
Comp150 TUI Spring 2006

2. The Design Space of TUIs
Designers’ Outpost [Klemmer et al.2001]
ComTouch [Jacob et al. 2001]
Senseboard [Jacob et al. 2001]
Media Blocks [Ullmer et al ]
Last week we discussed a variety of interfaces. Some of the examples are here as reminders.
These interfaces are very different in their structure, functionality, interaction techniques and implementation technology. Today we’ll discuss a modeling language that will help us describe their structure and functionaly in a concise way.
Navigation Blocks [Camarata et al ]
Tangible Query Interfaces [Ullmer et al ]
Comp150 TUI Spring 2006

3. Why to Model?
Highlight system properties: observability, predicatability, modes etc.
Expose design issues
Compare alternative designs.
Communicate design to SW engineers
Communicate design to ‘customers’
Design documentation
Generate code from technology independent spec.
Integrate novel technologies
Analysis
Communication
First, why do we need
Implementation
Comp150 TUI Spring 2006

4. A Framework for modeling TUIs Token and Constraints (TAC) Paradigm
Tokens: Physical objects that represent digital information
Constraints: Physical objects that constraint the manipulation of tokens by:
Suggesting how to manipulate a token
Physically constraining
Providing a frame of reference
TAC: A relationship between a token and a set of constraints which encapsulates a set of manipulation actions
To describe these interfaces we first need to recognize a common set of constructs: what are the widgets of TUIs? After a thorough investigation we believe these constructs can be used to describe a large set of tUIs.
Comp150 TUI Spring 2006

5. Describing a TUI using TUIVIS
interaction 3
dialogue
association 1 2
TAC Palette
Dialogue diagram
Task diagram
Comp150 TUI Spring 2006

6. Graphical Representation of Tokens and Constraints
surface
rack
indentation
knob
slider
connector
Tokens
Constraints
Comp150 TUI Spring 2006

7. Combining Tokens and Constraints into TACs
Create a TAC palette for UrP
Comp150 TUI Spring 2006

8. The TAC Palette 1 2 Comp150 TUI Spring 2006 TAC Representation
Association
Manipulation
Variable
Token
Constraint
TAC graphics
Action
Response 1
building
building model
surface
other buildings
Add
displays shadow according to time.
Remove
Removes related info from display
move
Updates display 2
distance
Distance tool
Two buildings
Surface
Displays distance
remove
Hides distance
Comp150 TUI Spring 2006

9. The TAC Palette 1 2 3 Comp150 TUI Spring 2006 TAC Representation
Association
Manipulation
Variable
Token
Constraint
TAC graphics
Action
Response 1
building
building model
surface
other buildings
Add
displays shadow according to time.
Remove
Removes related info from display
move
Updates display 2
distance
Distance tool
Two buildings
Surface
Displays distance
remove
Hides distance 3
wind simulation
Wind tool
Buildings
Displays wind
Hides wind
Updates wind
Comp150 TUI Spring 2006

10. Describing the Tangible Dialogue
High level states
Transitions
Element
Description
Internal state
A vector of the current values of application variables
Physical state
A vector of instantiated TAC relationships
Tasks
A set of tasks that could be completed within this state. These tasks can be performed either sequentially or in parallel depends on the satisfaction of their preconditions
wind
move
mater.ial
orien.t
time
dist.
Source
Representation
Timer
System
User Interaction
Tacit Interaction
Trigger
Comp150 TUI Spring 2006

11. Constructing the Dialogue Diagram
What is the initial state of the system?
What tasks are users willing to accomplish while interacting with the URP interface?
Start from the start state and repeat until you can’t find new states:
What event change the set of tasks users can perform in this state?
Create a new state, describe the new set of tasks.
Comp150 TUI Spring 2006

12. Dialogue Diagram Comp150 TUI Spring 2006 UIDL description
wind
move
time
mater.
orien.
addBuilding
/shadow(T)
removeBuilding
/shadow(F)
numOfBuilding=2
removeBuilding numOfBuilding>2
dist.
UIDL description
As we discussed our UIDL is based on a two-tier model that separates reality based interaction to two level: dialogue and task.
The dialogue level is sequential. It is inspired from storyboarding techniques and provides an overview of the interaction structure.
It is represented as an enhanced state diagram and uses a notation based on the Statechart specification language.
While the dialogue level, describes which tasks can be performed in parallel,
the task level: describes how many instance of the same task can be performed in parallel.
The Task diagram is inspired from colored Petri nets, a formalism for specifying concurrency:
It consists of: a set of actions that comprise a task
A set of places of type…. Which describes the resources needed for the executions of each action, as well as the digital and physical state of the system before and after the execution of an action.
A set of flow relations between laces and actions, and finally a set of continuous relationships.
Next, we will introduce the visual notation
Comp150 TUI Spring 2006

13. Task Diagram URP - distance measuring
The Task diagram is inspired from petri nets , a modling technique for parallel interacting components.
Comp150 TUI Spring 2006

14. Constructing a Task Diagram
What is the physical state of the system prior to the first action aimed at accomplishing this task?
What are the actions users perform toward accomplishing this task?
What physical interaction objects are needed for each action?
What is the physical state of the system after an action was performed?
What is the internal state of the system after an action was performed?
When does the system physical state goes back to its state prior to the execution of the first action?
Comp150 TUI Spring 2006

15. Task Diagram URP - distance measuring
The Task diagram is inspired from petri nets , a modling technique for parallel interacting components.
Comp150 TUI Spring 2006

16. Summary interaction dialogue association TAC Palette Dialogue diagram3
dialogue
association 1 2
TAC Palette
Dialogue diagram
Task diagram
Comp150 TUI Spring 2006

17. Project 1, Part 2 Analyzing TUIs
What to submit:
A TAC Palette
A dialogue diagram
At least two task diagrams
Discussion of the following topics:
Observability and predictability:
Does the physical state of the system indicate to the user the internal state of the system? Is there information in the internal state that is not presented in the physical state?
Does the physical state of the system contain enough information to enable the user to determine what tasks are available for him to perform? Assuming multiple users interact with the system at the same time. Does the physical state of the system indicate to users what tasks they can or cannot perform in parallel? If possible bring an example of two tasks that different users cannot perform in parallel.
Modes
Certain actions are meaningful only in certain contexts. Bring an example of a user action that is meaningless in one context and has a meaning in another.
Physical Syntax
Does the physical state of the system indicate to users which actions are legal/illegal?
Comp150 TUI Spring 2006

18. Next week Introduction to technologies Reading
Start working in teams on a technology demo
Comp150 TUI Spring 2006

19. Designer’s Outpost [Klemmer 2001] (An Interactive Surface)
Back
Comp150 TUI Spring 2006

20. Tangible Query Interfaces [Ullmer 2003] (A Token+Constraints System)
Back
Comp150 TUI Spring 2006

21. ComTouch [Chang 2003] (A Haptic Interface)
Back
Comp150 TUI Spring 2006

22. Web Stickers (Paper Based)
Comp150 TUI Spring 2006