Ubiquitous Computing: LE1

Ubiquitous Computing "Ubiquitous Computing is fundamentally characterized by the connection of things in the world with computation“ (Mark Weiser)

Terminology Ubiquitous Swedish: “allestädes närvarande” www.m-w.com: existing or being everywhere at the same time : constantly encountered : widespread omnipresent, allover, universal, constantly available pervasive to the point of subconscious

Ubiquitous Computing Coined by Mark Weiser and researchers at Xerox Palo Alto in the late 80s. Computers as we know them will be replaced by a multitude of networked computing devices embedded in our environments, and these devices will be invisible in the sense of not being perceived as computers. Ubiquitous computing pushes the user interface away from the desktop and into our everyday environments.

Ubiquitous Computing Mark Weiser’s vision (1991) “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” (Weiser)

Ubiquitous Computing In short about what happens when we become surrounded by IT-artefacts in our daily and working lives Distributed interactive computing permeating the world Perspectives Computer Science Sociological Science Interaction Design

History of Interaction Era 1: Mainframes (past) Central, “powerful” and expensive computer Many users access a single computer from “dumb” terminals Used for enterprise data processing Cobol, data bases, etc.. Computer not easily accessible

History of Interaction Era II: Personal computers (present) “Powerful” and relatively inexpensive computers At least one machine per user Used for word processing, personal productivity applications, video, audio etc Powerpoint, MS Word, Web browser etc.. Computer still not that accessible

History of Interaction Era III: Post-PC (future) Explosion in number and variety of computing devices A number of devices/machines share one user Devices (inexpensive) vary in complexity and function Used to make “our lives better” Computers become “invisible”

Three, or Four, Waves Mainframe computing (60’s-70’s) massive computers to execute big data processing applications very few computers in the world Desktop computing (80’s-90’s) one computer at every desk to help in business-related activities computers connected in intranets to a massive global network (internet), all wired Ubiquitous computing (00’s-?) tens/hundreds of computing devices in every room/person, becoming “invisible” and part of the environment WANs, LANs, PANs – networking in small spaces Cloud computing as a fourth wave, see Google Tech Talk.

Presence of IT-artefacts

Visions of the Future The Office of the Professional 1981 - before the mouse Office OfTheProfessional.wmv Weiser’s vision: UbiCompIntro.mov UbiCompClose.mov Playful Interaction in recent visions playful_interaction.avi

Software Agents Public Interactions Mobile-Nomadic Computing Invisible/Silent/Calm Interfaces Mechatronics Ubiquitous Computing Tangible interfaces Embedded computers Context sensitivity Wearable Computing Augmented Reality Adaptive services Ad hoc Networks

UbiComp enabling technologies Processing cheaper, faster, smaller, more energy efficient Storage big and fast Networking global, local, ad- hoc, low- power, high bandwidth, low latencies Displays projection, flexible materials, power consumption

* Embedded sensor networks Sensors: - in everyday environments - on people - on artefacts Sensor fusion: combining different data and placements to gather context

Embedded Computers Computers are becoming smaller and cheaper ->Everyday things will to a higher extent be “computerized”. Analogy: Electric motors used to be large, powering several appliances. They are nowadays embedded in the devices and invisible, so that the user sees a task-specific tool and not the technology of motors. Computers as well as motors are enablers and infrastructure. (Donald Norman)

Smart-Its: sensors: sound, light, acceleration (2d), pressure core board: context-recognition, communication interface (RF)

Defining Context Information used to characterize the situation of an entity Entity - person, place or object relevant to the interaction between a human and some computational service

Context and location Awareness What if your device knew from the time you approached or picked it up: WHO you are (user aware). WHERE you are (location aware). WHAT is going on around you (environment aware).

* Context-aware computing “computer-based devices [that] reach out into the real world through sensors” [Gellerson]. “A system is context-aware if it uses context to provide relevant information and/or services to the user, where relevancy depends on the user’s task.” [Dey & Abowd, 1999].

Computers that know where you are Examples Location-Aware Systems Computers that know where you are Examples Help guide blind person through an area Put location tracking on buses so you will know when next one is coming Attach reminders to places, like "Next time I am near grocery store remind me to…“

Sample Context-Aware Apps ParcTabs and Active Badges Xerox PARC Want, Schilit, et al Active Badge Olivetti / AT&T Hopper, Harter, et al

Tangible interfaces Most HC interaction is done through a big glass tube pressing keys and moving a mouse Human-human interaction uses many more channels

* Tangible computing Input, data, output and networking contained and accessed within the same tangible artefact Paper, cups, pens, umbrellas or specially designed artefacts Tangible objects as active entities that respond to the environment, to user manipulation and people’s activities in general Building on the users’ cognitive abilities

Allow users to “grasp and manipulate” bits by coupling the bits with everyday things Bridge cyberspace and physical space Bridge foreground and background of human activities Turn the physical world into the interface

Tangible Computing Bishop’s Marble Answering Machine physical interaction with digital information

* Social computing Incorporating understandings of the social world into interactive systems Social traces left by people on objects or places Mobile social networks between co-located acquaintances enhancing user awareness by providing them information about others and their activity

* Augmented reality Superimposing a digital world upon the real one User experiences both as co-existing parts of the same reality User is able to interact with their combination in real time Interfaces: 3D computer graphics seen through transparent head-mounted displays or augmented glasses Spatialised audio cues heard through headphones

“Virtual reality” cuts you off from real world * Augmented reality “Virtual reality” cuts you off from real world Augmented reality adds interaction on top of real world Examples: superimposing street names on windshield while driving a car Mixed-reality: digital world not directly overlaid on the physical one but still presented as part of the same reality, f.ex. with both realities displayed on the screen of hand-held device)

MagicBook, U. of Washington

Augmented Reality

* Wearable computing Computing incorporated into clothing Make use of body-related information or interaction forms to control processes : - body movements - biometrics Embedded displays (e.g. glasses)

Calm Technology Encalms and Informs The users remain serene and in control Engages our peripheral attention Allows us to attune to more things The user is largely unaware of interacting with a computing or communication device. Moves easily from center to periphery and back Recenter to take control Ex: Car engine; when not OK we notice it

Dangling String By Natalie Jeremijenko, Xerox PARC bridging physical and virtual displays activity level on computer network Vision, sound and touch

UC Issues How can we enhance [everyday] activities by connecting them to a computational infrastructure? How can we design IT-artefacts and IT-environments using computational technology as a material? What computational infrastructure do we need?

UC Issues HCI Security & Privacy how to address many computers (without going insane) Security & Privacy Wireless data, overcoming surveillance Communications & Networking Home Networks, Personal Area Networks, Ad-hoc Networks Operating Systems Must fit in small memories, energy aware Hardware Design Small size, low weight, low power, harsh environments Software Design Cope with large variation in hardware, partition the code to be easily customized in different environments

Open Research Challenges People Issues Avoiding embarrassing situations Active Badges + bathrooms Inconvenient phone forwarding Avoiding dangerous situations Need to take into consideration cost of mistake Smoke alarms when cooking Lights that turn off when you're still there Woman locked in "smart toilet stall" Will adding more context really help here? http://www.theregister.co.uk/content/28/18312.html

Recap Ubiquitous Computing concerns how computational technology becomes a natural part of our lives Involves both technology and interaction design Moves away from traditional desktop interaction Computation becomes embedded in the environment

Design Perspective Toward a hybrid design Combine unique capabilities of computer technology with properties of physical environments Focus on foreground activity: how people interact with their designed environment – preserve familiarity and accustomed use Interaction with Physical Artefacts / Environments Physical/Tangible Interaction Physical affordances: suggesting and guiding action Distributed interaction: actions across artefacts / space Spatial/ambient interaction Spatial organisation of action/communication Ambient interaction: “spatial attention model”

Technical Perspective Computers in the Background Computers as secondary artefacts (embedded/situated) Embedded: (only a) part of some other artefact Situated: meaningfully placed, designed for specific context (“context-made” rather than “context-aware”) The Environment is the Interface Build upon affordances of the primary artefact or environments Don’t break with accustomed uses and familiar concepts Physical I/O (sensors/actuators) to tie computers to entities in the physical environment Networking to enable coherent interaction Spatial interaction, proximate networking, etc Allow for new interactions/relationships across parts of the environment