Ubiquitous and Pervasive Computing HCI in the real world

Ubiquitous Computing “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” - Mark Weiser, “The Computer for the 21st Century”, Scientific American Weiser: How do technologies disappear into the background? The vanishing of electric motors may serve as an instructive example: At the turn of the century, a typical workshop or factory contained a single engine that drove dozens or hundreds of different machines through a system of shafts and pulleys. Cheap, small, efficient electric motors made it possible first to give each machine or tool its own source of motive force, then to put many motors into a single machine. A glance through the shop manual of a typical automobile, for example, reveals twenty-two motors and twenty-five more solenoids. They start the engine, clean the windshield, lock and unlock the doors, and so on. By paying careful attention it might be possible to know whenever one activated a motor, but there would be no point to it. CS / Psych 6750

Agenda Area overview Four themes Challenges/issues CS / Psych 6750

Pervasive/Ubiquitous Computing Move beyond desktop machine Computing is embedded everywhere in the environment Nike + iPod interface for running shoes Sensor in sole of shoe detects foot motion, reports steps to the iPod, which displays the information in both visual and auditory forms. l CS / Psych 6750

Internet Kitchen Internet refrigerator and cooking appliance Download recipes from web directly to device http://www.dreamlg.com/en/ref/internet/introduction_tv.shtm CS / Psych 6750

Ubicomp Notions Computing capabilities, any time, any place “Invisible” resources Machines sense users’ presence and act accordingly Automatic hand soap dispenser CS / Psych 6750

Video Examples Ambient Room - H. Ishii Beyond the Desktop – J. Rekimoto CS / Psych 6750

Four Themes Automated capture of experiences with easy access Context-aware/sensitive interactions and applications Ubiquitous services independent of devices/platforms Natural/Implicit interfaces CS / Psych 6750

1. Automated Capture Motivation Record-taking is hard Multiple streams of information need to be captured Machines are better at some of these things than we are CS / Psych 6750

Examples Meeting capture (scribe at Xerox PARC), Mark Weiser CS / Psych 6750

LiveBoard Liveboard allows any participant to view what is drawn on the big screen and to take their own notes. CS / Psych 6750

Classroom 2000/eClass Classroom 2000 – video capture, indexing of audio by phoneme for searching later. Automatic upload to website. Annotations (drawing on smartboard) are links into the video or audio recording. CS / Psych 6750

Issues Stream integration -- At what level? Very finest level of actions or more coarse? Modifying a record after the fact Can student notes be added later? Networked interaction Why can’t your notes be put up on the Liveboard? CS / Psych 6750

2. Context-Aware Computing Computing services sense aspects of environment (location, user emotion,…) and tailor provided services Walk into conference room, my email is projected on a big screen there CS / Psych 6750

Examples Active Badge & PARCTab Shopping assistant Cyberguide Perception system for recognizing user moods from their facial expressions House where position is sensed and temperature adjusted automatically PARCTab was one of the first mobile computers (1992) and was designed for constant connectivity to workstation-based applications. It also reported location of user. Shopping Assistant (Michael Schneider) used RFID tags to infer the goals of the user (information gathering, acquisition) and provided on-demand information about products Cyberguide was a indoor location-based information device, could be used for example in a museum. Irfan Essa has done work in automated facial expression recognition CS / Psych 6750

Augmented Reality AR allows internal components to be “viewed” from the outside. CS / Psych 6750

Issues Registration, registration, registration How to integrate all the different aspects of context? What about the loss of privacy? Registration – aligning virtual objects with physical objects – very difficult to do even with stationary objects. CS / Psych 6750

3. Ubiquitous Services Care about service, not application Want to receive a message using whatever device is handy nearby Message is tailored to work according to device Text messages through a kiosk? Electronic bulletin board? CS / Psych 6750

Issues What is software infrastructure for integration? Do we get it by just adopting some standard? Many different devices, operating systems, networks, platforms CS / Psych 6750

4. Natural/Implicit Interfaces Computer interfaces and devices are more natural interaction tools Pen input Speech Gesture Tangible interfaces We have already talked about pen, speech, and gesture CS / Psych 6750

Examples Pen applications Speech applications Gesture pendant H. Ishii’s tangible UI work at MIT CS / Psych 6750

Gesture Pendant The gesture pendant is a wearable device for control of home automation systems via hand gestures. This solution has many advantages over traditional home automation interfaces in that those with loss of vision, motor skills, and mobility can use it. By combining other sources of context with the pendant, we can reduce the number and complexity of gestures while maintaining functionality. As users input gestures, the system can also analyze their movements for pathological tremors. This information can then be used for medical diagnosis, therapy, and emergency services. Currently, the Gesture Pendant can recognize control gestures with an accuracy of 95% and user-defined gestures with an accuracy of 97% it can detect tremors above 2HZ within plus or minus 0.1 Hz. CS / Psych 6750

Personal Ambient Displays Personal Ambient Displays are small, physical devices worn to display information to a person in a subtle, persistent, and private manner. They can be small enough to be carried in a pocket, worn as a watch, or even adorned like jewelry. In our implementations, information is displayed solely through tactile modalities such as heating and cooling, movement and vibration, and change of shape. Iishi’s work – paging and notification, could vibrate, heat up, or change shape CS / Psych 6750

Pins and Super Cilia Skin Super Cilia Skin is a multi-modal interactive interface, conceived as a computationally enhanced membrane coupling tactile-kinesthetic input with tactile and visual output. An array of individual actuators (cilia) use changes in orientation to display images or physical gestures as physical or tactile information. CS / Psych 6750

Workbenches This is a network-building workbench with a multi-touch interface CS / Psych 6750

SandScape SandScape is a tangible interface for designing and understanding landscapes through a variety of computational simulations using sand. Users view these simulations as they are projected on the surface of a sand model that represents the terrain. The users can choose from a variety of different simulations that highlight either the height, slope, contours, shadows, drainage or aspect of the landscape model. The users can alter the form of the landscape model by manipulating sand while seeing the resultant effects of computational analysis generated and projected on the surface of sand in real-time. MIT Media Group CS / Psych 6750

Issues Errors are more likely (handwriting recognition, speech, …) How to discover and correct them? Is there truly value added? CS / Psych 6750

Wearable Computing Computation devices accompany you, rather than you seeking them out T. Starner Micro-optical – projects screen onto retina, appears to be floating in front of the user “Twiddler” chording keyboard in pocket CS / Psych 6750

Evaluation How do we evaluate these technologies? Challenge in Classroom 2000 CS / Psych 6750

Interested in More… CS 7470, Mobile and Ubiquitous computing, usually in Spring term Gregory Abowd Readings, discussion, research-oriented CS / Psych 6750