1. Ubiquitous and Pervasive Computing
HCI in the real world

2. Ubiquitous Computing
“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” - Mark Weiser, “The Computer for the 21st Century”, Scientific American
Weiser: How do technologies disappear into the background? The vanishing of electric motors may serve as an instructive example: At the turn of the century, a typical workshop or factory contained a single engine that drove dozens or hundreds of different machines through a system of shafts and pulleys. Cheap, small, efficient electric motors made it possible first to give each machine or tool its own source of motive force, then to put many motors into a single machine.
A glance through the shop manual of a typical automobile, for example, reveals twenty-two motors and twenty-five more solenoids. They start the engine, clean the windshield, lock and unlock the doors, and so on. By paying careful attention it might be possible to know whenever one activated a motor, but there would be no point to it.
CS / Psych 6750

3. Agenda
Area overview
Four themes
Challenges/issues
CS / Psych 6750

4. Pervasive/Ubiquitous Computing
Move beyond desktop machine
Computing is embedded everywhere in the environment
Nike + iPod interface for running shoes
Sensor in sole of shoe detects foot motion, reports steps to the iPod, which displays the information in both visual and auditory forms. l
CS / Psych 6750

5. Internet Kitchen Internet refrigerator and cooking appliance
Download recipes from web directly to device
CS / Psych 6750

6. Ubicomp Notions Computing capabilities, any time, any place
“Invisible” resources
Machines sense users’ presence and act accordingly
Automatic hand
soap dispenser
CS / Psych 6750

7. Video Examples Ambient Room - H. Ishii
Beyond the Desktop – J. Rekimoto
CS / Psych 6750

8. Four Themes Automated capture of experiences with easy access
Context-aware/sensitive interactions and applications
Ubiquitous services independent of devices/platforms
Natural/Implicit interfaces
CS / Psych 6750

9. 1. Automated Capture Motivation Record-taking is hard
Multiple streams of information need to be captured
Machines are better at some of these things than we are
CS / Psych 6750

10. Examples Meeting capture (scribe at Xerox PARC), Mark Weiser
CS / Psych 6750

11. LiveBoard
Liveboard allows any participant to view what is drawn on the big screen and to take their own notes.
CS / Psych 6750

12. Classroom 2000/eClass
Classroom 2000 – video capture, indexing of audio by phoneme for searching later. Automatic upload to website. Annotations (drawing on smartboard) are links into the video or audio recording.
CS / Psych 6750

13. Issues Stream integration -- At what level?
Very finest level of actions or more coarse?
Modifying a record after the fact
Can student notes be added later?
Networked interaction
Why can’t your notes be put up on the Liveboard?
CS / Psych 6750

14. 2. Context-Aware Computing
Computing services sense aspects of environment (location, user emotion,…) and tailor provided services
Walk into conference room, my is projected on a big screen there
CS / Psych 6750

15. Examples Active Badge & PARCTab Shopping assistant Cyberguide
Perception system for recognizing user moods from their facial expressions
House where position is sensed and temperature adjusted automatically
PARCTab was one of the first mobile computers (1992) and was designed for constant connectivity to workstation-based applications. It also reported location of user.
Shopping Assistant (Michael Schneider) used RFID tags to infer the goals of the user (information gathering, acquisition) and provided on-demand information about products
Cyberguide was a indoor location-based information device, could be used for example in a museum.
Irfan Essa has done work in automated facial expression recognition
CS / Psych 6750

16. Augmented Reality
AR allows internal components to be “viewed” from the outside.
CS / Psych 6750

17. Issues Registration, registration, registration
How to integrate all the different aspects of context?
What about the loss of privacy?
Registration – aligning virtual objects with physical objects – very difficult to do even with stationary objects.
CS / Psych 6750

18. 3. Ubiquitous Services Care about service, not application
Want to receive a message using whatever device is handy nearby
Message is tailored to work according to device
Text messages through a kiosk? Electronic bulletin board?
CS / Psych 6750

19. Issues What is software infrastructure for integration?
Do we get it by just adopting some standard?
Many different devices, operating systems, networks, platforms
CS / Psych 6750

20. 4. Natural/Implicit Interfaces
Computer interfaces and devices are more natural interaction tools
Pen input
Speech
Gesture
Tangible interfaces
We have already talked about pen, speech, and gesture
CS / Psych 6750

21. Examples Pen applications Speech applications Gesture pendant
H. Ishii’s tangible UI work at MIT
CS / Psych 6750

22. Gesture Pendant
The gesture pendant is a wearable device for control of home automation systems via hand gestures. This solution has many advantages over traditional home automation interfaces in that those with loss of vision, motor skills, and mobility can use it. By combining other sources of context with the pendant, we can reduce the number and complexity of gestures while maintaining functionality. As users input gestures, the system can also analyze their movements for pathological tremors. This information can then be used for medical diagnosis, therapy, and emergency services. Currently, the Gesture Pendant can recognize control gestures with an accuracy of 95% and user-defined gestures with an accuracy of 97% it can detect tremors above 2HZ within plus or minus 0.1 Hz.
CS / Psych 6750

23. Personal Ambient Displays
Personal Ambient Displays are small, physical devices worn to display information to a person in a subtle, persistent, and private manner. They can be small enough to be carried in a pocket, worn as a watch, or even adorned like jewelry. In our implementations, information is displayed solely through tactile modalities such as heating and cooling, movement and vibration, and change of shape.
Iishi’s work – paging and notification, could vibrate, heat up, or change shape
CS / Psych 6750

24. Pins and Super Cilia Skin
Super Cilia Skin is a multi-modal interactive interface, conceived as a computationally enhanced membrane coupling tactile-kinesthetic input with tactile and visual output. An array of individual actuators (cilia) use changes in orientation to display images or physical gestures as physical or tactile information.
CS / Psych 6750

25. Workbenches
This is a network-building workbench with a multi-touch interface
CS / Psych 6750

26. SandScape
SandScape is a tangible interface for designing and understanding landscapes through a variety of computational simulations using sand. Users view these simulations as they are projected on the surface of a sand model that represents the terrain. The users can choose from a variety of different simulations that highlight either the height, slope, contours, shadows, drainage or aspect of the landscape model. The users can alter the form of the landscape model by manipulating sand while seeing the resultant effects of computational analysis generated and projected on the surface of sand in real-time.
MIT Media Group
CS / Psych 6750

27. Issues
Errors are more likely (handwriting recognition, speech, …) How to discover and correct them?
Is there truly value added?
CS / Psych 6750

28. Wearable Computing
Computation devices accompany you, rather than you seeking them out
T. Starner
Micro-optical – projects screen onto retina, appears to be floating in front of the user
“Twiddler” chording keyboard in pocket
CS / Psych 6750

29. Evaluation How do we evaluate these technologies?
Challenge in Classroom 2000
CS / Psych 6750

30. Interested in More…
CS 7470, Mobile and Ubiquitous computing, usually in Spring term
Gregory Abowd
Readings, discussion, research-oriented
CS / Psych 6750