1. Pervasive Computing: Vision and Challenges
M. Satyanarayanan
IEEE Personal Communications, 2001

2. Evolution & Related Fields
Pervasive computing represents a major evolutionary step in a line of work dating back to the mid-1970s.
Two distinct earlier steps in this evolution:
Distributed systems
Mobile computing
Some of the technical problems in pervasive computing correspond to problems already identified and studied earlier in the evolution.
In some of those cases, existing solutions apply directly; in other cases, the demands of pervasive computing are sufficiently different that new solutions have to be sought.
There are also new problems introduced by pervasive computing that have no obvious mapping to problems studied earlier.

3. Evolution & Related Fields
DISTRIBUTED SYSTEMS
The field of distributed systems arose at the intersection of personal computers and local area networks.
The research that followed from the mid-1970s through the early 1990s created a conceptual framework and algorithmic base that has proven to be of enduring value in all work involving two or more computers connected by a network — whether mobile or static, wired or wireless, sparse or pervasive.
Spans many areas that are foundational to pervasive computing:
Remote communication, including protocol layering, remote procedure call [5], the use of timeouts, and the use of endto-end arguments in placement of functionality
Fault tolerance, including atomic transactions, distributed and nested transactions, and two-phase commit
High availability, including optimistic and pessimistic replica contro, mirrored execution, and optimistic recovery
Remote information access, including caching, function shipping, distributed file systems, and distributed databases
Security, including encryption-based mutual authentication and privacy.
MOBILE COMPUTING
The appearance of full-function laptop computers and wireless LANs in the early 1990s led researchers to confront the problems that arise in building a distributed system with mobile clients. The field of mobile computing was thus born.
Many basic principles of distributed system design continued to apply.
Four key constraints of mobility forced the development of specialized techniques:
Unpredictable variation in network quality
Lowered trust and robustness of mobile elements
Limitations on local resources imposed by weight and size constraints
Concern for battery power consumption
Many research areas:
Mobile networking, including Mobile IP, ad hoc protocols, and techniques for improving TCP performance in wireless networks
Mobile information access, including disconnected operation, bandwidth-adaptive file access, and selective control of data consistency
Support for adaptative applications, including transcoding by proxies and adaptive resource management
System-level energy saving techniques, such as energy-aware adaptation, variable-speed processor scheduling, and energy-sensitive memory management
Location sensitivity, including location sensing and location-aware system behavior
Figure 2. Taxonomy of computer systems research problems in pervasive computing.

4. Evolution & Related Fields
Distributed systems
Arose at the intersection of personal computers and local area networks.
The research that followed from the mid-1970s through the early 1990s created a conceptual framework and algorithmic base (for various types of networks)
Spans many areas that are foundational to pervasive computing

5. Evolution & Related Fields
Mobile computing
Led by the appearance of full-function laptop computers and wireless LANs in the early 1990s
How to build a distributed system with mobile clients??
Many basic principles of distributed system design continued to apply
Four key constraints of mobility forced the development of specialized techniques:
Unpredictable variation in network quality
Lowered trust and robustness of mobile elements
Limitations on local resources imposed by weight and size constraints
Concern for battery power consumption

6. Other related fields Sensor networks Human-computer interaction
Artificial intelligence

7. Other related fields Sensor Networks
A sensor network consist of a large number of tiny autonomous computing devices, each equipped with sensors, a wireless radio, a processor, and a power source.
Sensor networks are envisioned to be deployed unobtrusively in the physical environment to monitor a wide range of environmental phenomena
e.g., environmental pollutions, seismic activity, wildlife

8. Other related fields Human Computer Interaction (HCI)
HCI is the study of interaction between people (users) and computers.
Goal of HCI: to improve the interaction between users and computers by making computers more user-friendly and receptive to the user's needs.
Long term goal of HCI: to design systems that minimize the barrier between the human's cognitive model of what they want to accomplish and the computer's understanding of the user's task.

9. Other related fields Artificial Intelligence
AI can be defined as intelligence exhibited by an artificial (non-natural, manufactured) entity.
AI is studied in overlapping fields of computer science, psychology and engineering, dealing with intelligent behavior, learning and adaptation in machines, generally assumed to be computers.
Research in AI is concerned with producing machines to automate tasks requiring intelligent behavior.

10. Key research thrusts
Pervasive computing incorporates four additional research thrusts:
Effective use of smart spaces
Invisibility
Localized scalability
Masking uneven conditioning

11. Effective use of smart spaces
By embedding computing infrastructure in building infrastructure, a smart space brings together two worlds that have been disjoint until now.
The fusion of these worlds enables sensing and control of one world by the other.
Automatic adjustment of heating, cooling, and lighting levels in a room based on an occupant’s electronic profile.

12. Invisibility
The ideal expressed by Weiser is complete disappearance of pervasive computing technology from a user’s consciousness (minimal user distraction).
If a pervasive computing environment continuously meets user expectations and rarely presents him with surprises, it allows him to interact almost at a subconscious level.
Görünmezlik

13. Localized scalability
As smart spaces grow in sophistication, the intensity of interactions between a user’s personal computing space and his/her surroundings increases.
This has severe bandwidth, energy, and distraction implications for a wireless mobile user.
The presence of multiple users will further complicate this problem.
Good system design has to achieve scalability by severely reducing interactions between distant entities.
Sınırlandırılmış Ölçeklenirlik

14. Masking uneven conditioning
Huge differences in the “smartness” of different environments — what is available in a well-equipped conference room, office, or classroom may be more sophisticated than in other locations.
This large dynamic range of “smartness” can be jarring to a user, detracting from the goal of making pervasive computing technology invisible.
One way to reduce the amount of variation seen by a user is to have his/her personal computing space compensate for “dumb” environments.
Değişken Durumların Gizlenmesi

15. Detailed design and implementation issues
User intent
Cyber foraging
Adaptation strategy
High-level energy management
Client thickness
Context awareness
Balancing proactivity and transparency
Impact on layering
Privacy and trust

16. User intent
For proactivity to be effective, it is crucial that a pervasive computing system track user intent.
Otherwise, it will be almost impossible to determine which system actions will help rather than hinder the user.
For example, suppose a user is viewing video over a network connection whose bandwidth suddenly drops. Should the system:
Reduce the fidelity of the video?
Pause briefly to find another higher-bandwidth connection?
Advise the user that the task can no longer be accomplished?
The correct choice will depend on what the user is trying to accomplish.

17. Cyber foraging (aka “living off the land”)
Dynamically augment the computing resources of a wireless mobile computer by exploiting wired hardware infrastructure.
As computing becomes cheaper and more plentiful, it makes economic sense to “waste” computing resources to improve user experience.
In the forseeable future, public spaces such as airport lounges and coffee shops will be equipped with compute servers or data staging servers for the benefit of customers, much as table lamps are today.
Today, many shopping centers and cafeterias offer their customers free wireless internet access.

18. Adaptation strategy
Adaptation is necessary when there is a significant mismatch between the supply and demand of a resource (e.g. wireless network bandwidth, energy, computing cycles or memory).
There are three alternative strategies for adaptation in pervasive computing:
A client can guide applications in changing their behavior so that they use less of a scarce resource. This change usually reduces the user-perceived quality, or fidelity, of an application.
A client can ask the environment to guarantee a certain level of a resource (reservation-based QoS systems). From the viewpoint of the client, this effectively increases the supply of a scarce resource to meet the client’s demand.
A client can suggest a corrective action to the user. If the user acts on this suggestion, it is likely (but not certain) that resource supply will become adequate to meet demand.

19. High-level energy management
Sophisticated capabilities such as proactivity and self-tuning increase the energy demand of software on a mobile computer in one’s personal computing space.
Making such computers lighter and more compact places severe restrictions on battery capacity, so the higher levels of the system must be involved in memory management.
One example is energy-aware memory management, where the operating system dynamically controls the amount of physical memory that has to be refreshed.
Another example is energy-aware adaptation, where individual applications switch to modes of operation with lower fidelity and energy demand under operating system control.

20. Client thickness (or client’s hardware capabilities)
For a given application, the minimum acceptable thickness of a client is determined by the worst-case environmental conditions under which the application must run satisfactorily.
A very thin client suffices if one can always count on high-bandwidth low-latency wireless communication to nearby computing infrastructure, and batteries can be recharged or replaced easily.
If there exists even a single location visited by a user where these assumptions do not hold, the client will have to be thick enough to compensate at that location.
This is especially true for interactive applications where crisp response is important.

21. Context awareness
A pervasive computing system must be cognizant of its user’s state and surroundings, and must modify its behavior based on this information.
A user’s context can be quite rich, consisting of various attributes
Ex) physical location, physiological state (e.g., body temperature and heart rate), emotional state (e.g., angry, distraught, or calm), personal history, daily behavioral patterns, and so on.
If a human assistant were given such context, he or she would make decisions in a proactive fashion, anticipating user needs.
In making these decisions, the assistant would typically not disturb the user at inopportune moments except in an emergency.
A pervasive computing system should emulate such a human assistant.

22. Balancing proactivity and transparency
Unless carefully designed, a proactive system can annoy a user and thus defeat the goal of invisibility.
A mobile user’s need and tolerance for proactivity are likely to be closely related to his/her level of expertise on a task and familiarity with his/her environment.
A system that can infer these factors by observing user behavior and context is better positioned to strike the right balance.
For transparency, a user patience model can be implemented to predict whether the user will respond positively to a fetch request.
So the user interaction is suppressed and the fetch is handled transparently.

23. Impact on layering
Proactivity and adaptation based on corrective actions seem to imply exposure of much more information across layers than is typical in systems today.
Layering cleanly separates abstraction from implementation and is thus consistent with sound software engineering.
Layering is also conducive to standardization since it encourages the creation of modular software components.

24. Privacy and trust
As a user becomes more dependent on a pervasive computing system, it becomes more knowledgeable about that user’s movements, behavior patterns and habits.
Exploiting this information is critical to successful proactivity and self-tuning (invisibility), but also may cause serious loss of privacy.
User must trust the infrastructure to a considerable extent and the infrastructure needs to be confident of the user’s identity and authorization level before responding to his/her requests.
It is a difficult challenge to establish this mutual trust in a manner that is minimally intrusive and thus preserves invisibility.

25. Charting Past, Present, and Future Research in Ubiquitous Computing
Gregory Abowd and Elizabeth Mynatt
ACM TOCHI 2000

26. Natural interfaces
Context-aware computing
Automated capture and access
Everyday computing
System evaluation challenges

27. Computing with natural interfaces
Ubicomp inspires “off-the-desktop” applications
Needs “off-the-desktop” means of interaction
Speech, gestures, writing
More accessible
Easier to use???

28. Computing with natural interfaces
Need for first-class natural data types
To ease the development of more apps w/ natural interfaces
Should be easy keyboard and mouse
Support for “fundamental” operations
E.g., text and speech recognitions
Support for “extra” operations
Tivoli Live Board: special input like structured gestures such as dragging and freeform selection
Classroom 2000: annotating pen strokes with timestamps

29. Computing with natural interfaces
Error prone interaction
Permit new and numerous mistakes
People do not have perfect recognition
As low as 54%; cursive handwriting 88%; printed handwriting 96.8%
Recognition accuracy == user satisfaction??
Not really: complexity of error recovery dialogues and value-added benefit of any given efforts
Entering a command vs. writing journal entries
Several research areas
Error reduction (about 5-10%)
Error detection
Reusable toolkit for error handling

30. Context aware computing
Current Systems
Generally using position and identification of objects
Still do not provide a complete context
Definition of context is limited
Research areas
Context toolkits
Toolkit for sensing environment
Explicit use of sensed information is up to program
What is context?
How is context represented?

31. What is context? Who What Where When Why
Currently generally tailored to one user
How important are others in determining our behavior
How could this be captured?
What
Attempt to figure out what is currently happening
Sense environment, use calendar software etc.
Where
Location based information, e.g., GPS
Most explored context information
When
Easily obtained information -- Computer is good at remembering time
Although determining when one event stops and another begins is not easy
Why
Even harder than the “what” question, biometric sensors might help (e.g., body temperature, heart rate, etc)

32. Toward context aware computing
Context representation
Requires universal context schemes or toolkits with standard context representations
Context sensing and fusion
How to make context-aware computing “ubiquitous”?
In practice, there are few truly ubiquitous, single-source context services
E.g., GPS does not work indoors; different indoor localization schemes have different characteristics (e.g., cost, range)
Like sensor fusion, context fusion handles seamless handling of sensing responsibility between boundaries of different context services
Combining multiple context sources can increase the accuracy of context information

33. Automated capture and access
Recording information and data as it occurs
Computers are inherently good at recording, people are not
People freed up to summarize and understand
Most work in academic/ classroom settings
Time stamping lectures, digital whiteboards
Challenges in “capture and access”
Sometime we don’t know we want to capture something until after its already happened
How could the computer know that?
If it captures everything then we need a system of sorting and filtering (access)
Gmail Problem
Access is a problem because capturing of raw data can be burdensome for sifting through; systems need to recognize important events facilitate access

34. Everyday computing
Continuous interactions (i.e., no clear beginning or end)
Both fundamental activities like communication and long-term endeavors do not have predefined starts and ends; information from past can be recycled
Very different traditional HCI design which assumes “closure” with clear goals like spell checking, dialogue, etc.
Interruption is expected:
People are constantly interrupted
Computer systems must recognize interruption and change state
Also computers must appropriately inform users
Multiple activities operate concurrently:
People multitask and rapidly switch task based on external unpredictable environment
Systems need to adapt to this opportunistic behavior and change accordingly

35. Toward everyday computing
Develop continuously present interface
No current model of continuously present interfaces, even people are not continuously present
Create an interface that doesn’t get annoying (e.g., wearable devices)
Determine what information should require my attention and what should be display peripherally
Connect events in the physical and virtual worlds (e.g., face to face vs. , document, webs)
Modify/fuse existing HCI schemes to efficiently support everyday computing (but evaluation is challenging and laborious)

36. System evaluation challenges
Hard to evaluate Ubicomp Systems
Little publish on ubicomp evaluation
Systems often required to be fully connected leading to systems that are hard to build
Lack of development toolkits make system creation difficult
Systems often need to be integrated into peoples lives which using big clunky prototypes does not lead itself well too
Task/Goal centric approaches don’t work in ubicomp

37. Example Projects
Pervasive computing projects have emerged at major universities and in industry:
Project Aura (Carnegie Mellon University)
Oxygen (Massachusetts Institute of Technology)
Portalano (University of Washington)
Endeavour (University of California at Berkeley)
Place Lab (Intel Research Laboratory at Seattle)

38. Example Projects : Project Aura (1)
Aura (Carnegie Mellon University)
Distraction-free (Invisible) Ubiquitous Computing.
The most precious resource in a computer system is no longer its processor, memory, disk or network. Rather, it is a resource not subject to Moore's law: User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his effectiveness.
Project Aura will fundamentally rethink system design to address this problem. Aura's goal is to provide each user with an invisible halo of computing and information services that persists regardless of location. Meeting this goal will require effort at every level: from the hardware and network layers, through the operating system and middleware, to the user interface and applications.
Project Aura will design, implement, deploy, and evaluate a large-scale system demonstrating the concept of a “personal information aura” that spans wearable, handheld, desktop and infrastructure computers.
PROJECT WEBSITE:

39. Example Projects : Project Aura (2)
Moore’s Law Reigns Supreme
Processor density
Processor speed
Memory capacity
Disk capacity
Memory cost
...
Glaring Exception
Human Attention
Human Attention
The most precious resource in a computer system is no longer its processor, memory, disk or network. Rather, it is a resource not subject to Moore's law: User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his effectiveness.
Project Aura will fundamentally rethink system design to address this problem. Aura's goal is to provide each user with an invisible halo of computing and information services that persists regardless of location. Meeting this goal will require effort at every level: from the hardware and network layers, through the operating system and middleware, to the user interface and applications.
Project Aura will design, implement, deploy, and evaluate a large-scale system demonstrating the concept of a “personal information aura” that spans wearable, handheld, desktop and infrastructure computers.
PROJECT WEBSITE:
Adam & Eve
2000 AD

40. Example Projects : Project Aura (3)
Aura Thesis:
The most precious resource in computing is human attention.
Aura Goals:
Reduce user distraction.
Trade-off plentiful resources of Moore’s law for human attention.
Achieve this scalably for mobile users in a failure-prone, variable-resource environment.
The most precious resource in a computer system is no longer its processor, memory, disk or network. Rather, it is a resource not subject to Moore's law: User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his effectiveness.
Project Aura will fundamentally rethink system design to address this problem. Aura's goal is to provide each user with an invisible halo of computing and information services that persists regardless of location. Meeting this goal will require effort at every level: from the hardware and network layers, through the operating system and middleware, to the user interface and applications.
Project Aura will design, implement, deploy, and evaluate a large-scale system demonstrating the concept of a “personal information aura” that spans wearable, handheld, desktop and infrastructure computers.
PROJECT WEBSITE:

41. Example Projects : Project Aura (4)
The Airport Scenario
Jane wants to send from the airport before her flight leaves.
She has several large enclosures
She is using a wireless interface
She has many options.
Simply send the
Is there enough bandwidth?
Compress the data first
Will that help enough?
Pay extra to get reserved bandwidth
Are reservations available?
Send the “diff” relative to older file
Are the old versions around?
Walk to a gate with more bandwidth
Where is there enough bandwidth?
How do we choose automatically?
The most precious resource in a computer system is no longer its processor, memory, disk or network. Rather, it is a resource not subject to Moore's law: User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his effectiveness.
Project Aura will fundamentally rethink system design to address this problem. Aura's goal is to provide each user with an invisible halo of computing and information services that persists regardless of location. Meeting this goal will require effort at every level: from the hardware and network layers, through the operating system and middleware, to the user interface and applications.
Project Aura will design, implement, deploy, and evaluate a large-scale system demonstrating the concept of a “personal information aura” that spans wearable, handheld, desktop and infrastructure computers.
PROJECT WEBSITE:

42. Example Projects : Project Aura (5)
The Mobile Task Scenario
Aura saves Scott’s task.
Scott enters office and gets strong authentication and secure access.
Aura restores Scott’s task on desktop machine and uses a large display.
Scott controls application by voice.
Bradley enters room.
Bradley gets weak authentication, Scott’s access changes to insecure.
Aura denies voice access to sensitive application.
Scott has multi-modal control of PowerPoint application.
Aura logs Scott out when he leaves the room.
The most precious resource in a computer system is no longer its processor, memory, disk or network. Rather, it is a resource not subject to Moore's law: User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his effectiveness.
Project Aura will fundamentally rethink system design to address this problem. Aura's goal is to provide each user with an invisible halo of computing and information services that persists regardless of location. Meeting this goal will require effort at every level: from the hardware and network layers, through the operating system and middleware, to the user interface and applications.
Project Aura will design, implement, deploy, and evaluate a large-scale system demonstrating the concept of a “personal information aura” that spans wearable, handheld, desktop and infrastructure computers.
PROJECT WEBSITE:

43. Example Projects : Oxygen
Oxygen (MIT)
Pervasive human-centered computing.
Goal of Oxygen is bringing abundant computation and communication, as pervasive and free as air, naturally into people's lives.

44. Example Projects : Oxygen (2)
To support highly dynamic and varied human activities, the Oxygen system must be
pervasive— it must be everywhere, with every portal reaching into the same information base;
embedded— it must live in our world, sensing and affecting it;
nomadic— it must allow users and computations to move around freely, according to their needs;
adaptable— it must provide flexibility and spontaneity, in response to changes in user requirements and operating conditions;
powerful, yet efficient— it must free itself from constraints imposed by bounded hardware resources, addressing instead system constraints imposed by user demands and available power or communication bandwidth;
intentional— it must enable people to name services and software objects by intent, for example, "the nearest printer," as opposed to by address;
eternal— it must never shut down or reboot; components may come and go in response to demand, errors, and upgrades, but Oxygen as a whole must be available all the time.

45. Related Projects: Portalano
Portolano (University of Washington)
An expedition into invisible computing.
Expedition goals:
Connecting the physical world to the world-wide information fabric
Instrument the environment: sensors, locators, actuators
Universal plug-and-play at all levels: devices to services
Optimize for power: computation partitioning, comm. opt.
Intermittent communication: new networking strategies
Get computers out of the way
Don’t interfere with user’s tasks
Diverse task-specific devices with optimized form-factors
Wide range of input/output modalities
Robust, trustworthy services
High-productivity software development
Self-organizing, active middleware, maintenance, monitoring
Higher-level, meaningful services
PROJECT WEBSITE:

46. Related Projects: Portalano (2)
Scenario
Alice begins the day with a cup of coffee and her personalized newspaper.
When her carpool arrives, she switches to reading the news on her handheld display, where she notices an advertisement for a new 3-D digital camera.
It looks like something that would interest her shutterbug-friend Bob, so Alice asks her address book to place the call.
PROJECT WEBSITE:

47. Related Projects: Portalano (3)
Scenario (2)
Bob's home entertainment system softens the volume of his custom music file as his phone rings.
Alice begins telling Bob about the camera, and forwards him a copy of the advertisement which pops up on his home display.
Bob is sold on the product, and after hanging up with her, he asks his electronic shopping agent to check his favorite photography stores for the lowest price and make the purchase.
PROJECT WEBSITE:

48. Related Projects: Portalano (4)
Scenario (3)
When the camera arrives, Bob snaps some photos of his neighbor's collection of antique Portuguese navigation instruments.
After reviewing the photo album generated automatically by a web-based service, Bob directs a copy of his favorite image to the art display in his foyer.
He also sends a pointer to the photo album to Alice and instructs his scheduling agent to set up a lunch date so that he can thank her for the suggestion.
PROJECT WEBSITE:

49. Example Projects : Endeavour
The Endeavour Expedition (UC Berkeley)
Charting the Fluid Information Utility
Endeavour Goal:
Enhancing human understanding through the use of information technology.
PROJECT WEBSITE:

50. Principles of Pervasive Computing
Evolution & Related Fields
Example Projects
Other Scenarios
References

51. Other Scenarios Buy drinks by Friday (1) Take out the last can of soda
Swipe the can’s UPC label, which adds soda to your shopping list
Make a note that you need soda for the guests you are having over this weekend

52. Other Scenarios Buy drinks by Friday (2) Approach a local supermarket
AutoPC informs you that you are near a supermarket
Opportunistic reminder: “If it is convenient, stop by to buy drinks.”

53. Other Scenarios Buy drinks by Friday (3)
Friday rolls around and you have not bought drinks
Deadline-based reminder sent to your pager

54. Other Scenarios Screen Fridge Provides: Email Video messages
Web surfing
Food management TV
Radio
Virtual keyboard
Digital cook book
Surveillance camera

55. Other Scenarios The Active Badge
This harbinger of inch-scale computers contains a small microprocessor and an infrared transmitter.
The badge broadcasts the identity of its wearer and so can trigger automatic doors, automatic telephone forwarding and computer displays customized to each person reading them.
The active badge and other networked tiny computers are called tabs.

56. Other Scenarios
The Active Badge

57. Other Scenarios Edible computers: The pill-cam
Miniature camera
Diagnostic device
It is swallowed
Try this with an ENIAC computer!

58. Other Scenarios Artificial Retina Direct interface with nervous system
Whole new computational paradigm (who’s the computer?)

59. Other Scenarios Smart Dust Nano computers that couple:
Sensors
Computing
Communication
Grids of motes (“nano computers”)