2. Ubiquitous Communications Henning Schulzrinne (with Knarig Arabshian, Stefan Berger, Stelios Sidiroglou, Kundan Singh, Xiaotao Wu, Weibin Zhao and the RPIDS authors) Columbia University IRT Lab Univ. of Pennsylvania – February 2004

3. Overview What is ubiquitous computing? Core ubiquitous communications functionality Brief introduction to SIP Ubiquitous computing in SIP and SLP On-going work at Columbia

4. What is ubiquitous computing? “Ubiquitous computing has as its goal the enhancing computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user.” (Weiser, 1993) “Ubiquitous computing is not virtual reality, it is not a Personal Digital Assistant (PDA) such as Apple's Newton, it is not a personal or intimate computer with agents doing your bidding. Unlike virtual reality, ubiquitous computing endeavers to integrate information displays into the everyday physical world. It considers the nuances of the real world to be wonderful, and aims only to augment them.” (Weiser, 1993)

5. Ubiquitous computing aspects Also related to pervasive computing Mobility, but not just cell phones Computation and communications Integration of devices “borrow” capabilities found in the environment  composition into logical devices seamless mobility  session mobility adaptation to local capabilities environment senses instead of explicit user interaction from small dumb devices to PCs light switches and smart wallpaper

6. Components of ubiquitous communications Service discovery  discover devices Service mobility  configuration information moves to new devices Event notification  for context awareness Context-awareness  location, user actions, location properties, …

7. Example “ubicomp” projects Ambient Devices EU IST Disappearing Computer Project Aura, CMU  user attention UNC “office of real soon now” augmented surfaces [Reki99] Microsoft Easy Living Oxygen, MIT Portolano, Univ. of Washington Endeavour, Berkeley CoolTown, HP Labs

8. Ubiquitous computing using SIP – what’s different? Traditionally, focus on closed environments (lab, single company, home, …) Often, proprietary protocols  self-contained environment Here, operate across Internet (  no Corba…) trusted, semi-trusted and untrusted participants use standard protocol mechanisms where possible make minimal assumptions on homogeneity respect user privacy

9. Context-aware communications Traditional emphasis: communicate anywhere, anytime, any media  largely possible today New challenge: tailor reachability Context-aware communications modify when, how, where to be reached  machine: context-dependent call routing  human: convey as part of call for human usage context-aware services leveraging local resources awareness of other users sources of location information voluntary and automatic location-based services  privacy concerns applies to other personal information activity, reachability, capabilities, bio sensor data, … emergency services as a location-based service

10. Context context = “the interrelated conditions in which something exists or occurs” anything known about the participants in the (potential) communication relationship both at caller and callee timeCPL capabilitiescaller preferences locationlocation-based call routing location events activity/availabilitypresence sensor data (mood, bio)not yet, but similar in many aspects to location data

11. What is SIP? Session Initiation Protocol  protocol that establishes, manages (multimedia) sessions also used for IM, presence & event notification uses SDP to describe multimedia sessions Developed at Columbia U. (with others) Standardized by IETF, 3GPP (for 3G wireless), PacketCable About 60 companies produce SIP products Microsoft’s Windows Messenger (4.7) includes SIP

12. Basic SIP message flow

13. SIP trapezoid outbound proxy a@foo.com: 128.59.16.1 registrar

14. SIP event notification Named events Typically, used for events within conferences (“Alice joined”) and call legs (e.g., call transfer) Supports arbitrary notification bodies, typically XML SUBSCRIBE sip:alice@vmail.example.com SIP/2.0 To: From: ;tag=78923 Call-Id: 1349882@alice-phone.example.com Contact: NOTIFY sip:alice@alice-phone.example.com SIP/2.0 … Event: message-summary Subscription-State: active Messages-Waiting: yes Message-Account: sip:alice@vmail.example.com Voice-Message: 2/8 (0/2)

15. SIP event architecture Does not try to route notifications (“application layer multicast”) as in SIENA Filtering at PA under discussion (for low-bandwidth devices) rate content But most ubicomp notification groups are probably small and message volume not likely to provide much bandwidth saving via network-based filtering Greatly simplifies trust model: no intermediaries that need to inspect content can encrypt via S/MIME However, can build redistribution “exploders” and list subscriptions (“subscribe to engineering@hp.com”)

16. SIP presence architecture PA a@foo.com: 128.59.16.1 watcher PUAs Alice Bob PUBLISH REGISTER SUBSCRIBE NOTIFY <p:presence xmlns:p="urn:…" entity="pres:alice@example.com"> open tel:09012345678

17. Session mobility Walk into office, switch from cell phone to desk phone call transfer problem  SIP REFER related problem: split session across end devices e.g., wall display + desk phone + PC for collaborative application assume devices (or stand-ins) are SIP- enabled third-party call control

18. Session mobility via 3PCC INVITE speakerphone m=audio c=pc42 INVITE display m=video c=pc42 192.0.2.1 192.0.2.7 INVITE pc42 m=video c=192.0.2.7 m=audio c=192.0.2.1 pc42

19. How to find services? Two complementary developments: smaller devices carried on user instead of stationary devices devices that can be time-shared large plasma displays projector hi-res cameras echo-canceling speaker systems wide-area network access Need to discover services in local environment SLP (Service Location Protocol) allows querying for services “find all color displays with at least XGA resolution” slp://example.com/SrvRqst?public?type=printer SLP in multicast mode SLP in DA mode Need to discover services before getting to environment “is there a camera in the meeting room?” SLP extension: find remote DA via DNS SRV

20. Service Location Protocol (SLP) Version 2 standardized June 1999 UA DA SA SrvReg SrvRply SrvRqst SrvReg DAAdvert

21. SLP attribute example URL service:printer:lpr://igore.wco.ftp.com/draft scope-list Development Language tag en Attributes (Name=Igore),(Description=For developers only), (Protocol=LPR),(location- description=12th floor), (Operator=James Dornan \3cdornan@monster\3e), (media- size=na-letter),(resolution=res-600),x-OK

22. Other service location mechanism DNS SRV/NAPTR DNS TXT records (Apple Rendezvous)  DNS-SD UPnP uses SSDP: multicast HTTP over UDP M-SEARCH * HTTP/1.1 S: uuid:ijklmnop-7dec-11d0-a765-00a0c91e6bf6 Host: 239.255.255.250:reservedSSDPport Man: "ssdp:discover“ ST: ge:fridge MX: 3 HTTP/1.1 200 OK S: uuid:ijklmnop-7dec-11d0-a765-00a0c91e6bf6 Ext: Cache-Control: no-cache="Ext", max-age = 5000 ST: ge:fridge USN: uuid:abcdefgh-7dec-11d0-a765-00a0c91e6bf6 AL:

23. Service mobility Allow access to service parameters anywhere – “payphone problem” address book incoming call rules source name (SIP From ) Existing solutions: SIM card  cumbersome to change synchronization (e.g., Palm)  not suitable for borrowed devices Server-based services  easier with SIP (service-routing), if carrier allows Emerging solutions for SIP systems: Small user token (smart card, RFID, i-button) identifying user Temporarily download configuration from home server

24. Context-based communication services Observable state and actions State: location of users user activities Derive state from sensors (time, location, environment, user interaction) data (calendars, address books) network inputs (messages) Actions incoming and outgoing calls incoming and outgoing IMs, SMS, email, … Initially, focusing on location at key context

25. Location-based services Finding services based on location physical services (stores, restaurants, ATMs, …) electronic services (media I/O, printer, display, …) not covered here Using location to improve (network) services communication incoming communications changes based on where I am configuration devices in room adapt to their current users awareness others are (selectively) made aware of my location security proximity grants temporary access Privacy rules for access to context data

26. Location-based services Presence-based approach: UA publishes location to presence agent (PA) becomes part of general user context other users (human and machines) subscribe to context call handling and direction location-based anycast (“anybody in the room”) location-based service directory Languages for location-based services building on experience with our XML-based service creation languages CPL for user-location services LESS for end system services

27. Location-based SIP services Services: Location-aware call routing “do not forward call if time at callee location is [11 pm, 8 am]” “only forward time-for-lunch if destination is on campus” “contact nearest emergency call center” “do not ring phone if I’m in a theater” “send delivery@pizza.com to nearest branch”delivery@pizza.com Location-based events subscribe to locations, not people “Alice has entered the meeting room” subscriber may be device in room  our lab stereo changes CDs for each person that enters the room Person + location events We’re implementing SIP, caller-preferences and CPL extensions for these services

28. Locations Geographic location latitude, longitude, altitude, velocity, heading Civil location (≠ postal location!) time zone, street address, city some countries are a bit difficult… Categorical office, library, theater, hospital, … Behavioral “public location, don't expect privacy” “silence is encouraged, don't ring the phone”

29. Determining locations SIP entities are often far away from physical user or his current network (intentionally) For many devices, can’t afford hardware to determine location different precision requirements: “in Fayette County” (within driving distance of service or person) “on campus” “in room 815” “in corner, talking to Bob” GPS doesn’t work indoors, but Assisted GPS (A-GPS) may Use location beacons: BlueTooth, 802.11 may not offer network connectivity see our 7DS project: offer local content + location Physically close by network entities: DHCP (same broadcast domain) PPP (tail circuit) Not always true with VPNs, but end system knows that it’s using a VPN

30. Determining location Two types of sensors: end system determines location “handset-based”  GPS, 802.11 triangulation network conveys location to end system or other component MAC backtracking AP-based 802.11 triangulation swipe cards, iButtons, active badges Two modes: explicit user action: swipe card, touch iButton involuntary: network-based tracking GPS may not be practical (cost, power, topology) Add location beacons extrapolate based on distance moved odometer, pedometer, time-since-sighting idea: meet other mobile location beacons estimate location based on third-party information

31. Determining locations For many devices, can’t afford hardware to determine location Implementing BlueTooth- based location sensor networks CU 7DS project: offer local content + location Developing programmable active badges with IR and RF capabilities

32. DHCP for locations modified dhcpd (ISC) to generate location information use MAC address backtracing to get location information DHCP server 458/17  Rm. 815 458/18  Rm. 816 DHCP answer: sta=DC loc=Rm815 lat=38.89868 long=77.03723 8:0:20:ab:d5:d CDP + SNMP 8:0:20:ab:d5:d  458/17

33. Location-based services & SIP We’re using SIP (and SIMPLE) as generic protocols for effecting change (“actuators”) send MESSAGE to devices distributing event information (“sensors”) Advantages: people and rooms identified by URIs sip:hgs@cs.columbia.edu sip:cepsr815@cs.columbia.edu cross-domain, with extensive security mechanisms domains don’t need to trust each other scalable to global system many other systems are mostly local

34. Architectures for (geo) information access Claim: all using protocols fall into one of these categories Presence or event notification “circuit-switched” model subscription: binary decision Messaging email, SMS basically, event notification without (explicit) subscription but often out-of-band subscription (mailing list) Request-response RPC, HTTP; also DNS, LDAP typically, already has session-level access control (if any at all) Presence is superset of other two

35. SIP URIs for locations Identify confined locations by a SIP URI, e.g., sip:rm815@cs.columbia.e du Register all users or devices in room Allows geographic anycast: reach any party in the room a@foo.com: 128.59.16.1 Room 815 sip:rm815 location beacon Contact: alice Contact: bob

36. RPID: rich presence data Basic IETF presence (CPIM) only gives you contact information (SIP, tel URI) priority “open” or “closed” Want to use presence to guide communications PA watcher PUA watcher PUBLISH NOTIFY everything "vague" CPL INVITE

37. Aside: SIP caller preferences  SIP core philosophy: many devices, one identifier  Address people, not plastic

38. Aside: SIP caller preferences But caller sometimes has preferences among devices SIP caller guides call routing: “I hate voicemail!” “I hate people!” “I prefer voicemail” Multilingual lines “Caller proposes, callee disposes” a@foo.com: 128.59.16.1 sip:isabel@a.com;languages="es" sip:isabel@a.com;languages="en";q=0.2 sip:bob@a.com;languages="en" INVITE sip:sales@a.com Accept-Contact: *;languages="en" INVITE REGISTER

39. RPID: Rich presence data Integrates caller preferences information into presence announcements : on-the-phone, away, appointment, holiday, meal, meeting, steering, in-transit, travel, vacation, busy, permanent-absence : home, office, public : public, private, quiet : facet of life (home, work, …) : activity for device : family, associate, assistant, supervisor : grouping

40. Event filtering Events are core attribute of ubiquitous computing systems tell devices about people actions tell people about device presence e.g., “Alice has entered Room 815” devices that know Alice’s preferences subscribe to Alice locations may also have presence e.g., for occupancy sensors, switches

41. Location filtering language SIP presence information will be updated using REGISTER and UPDATE Need to constrain who is allowed to see what detail  presentity privacy who wants to see what detail how often what granularity of change Proposal to allow SUBSCRIBE to include frequency limitation Working on CPL-like language invoked (logically) at publication time classes of users, e.g., based on entry in my address book classes get mapped to restriction “12 bits of long/lat resolution, 6 bits of altitude resolution, 0 bits of velocity” “time zone only”, “category only” watchers can then add filters that restrict the delivery: location difference > threshold entering or leaving certain area entering or leaving category or behavioral type

42. Presence model subscription policy event generator policy subscriber filter rate limiter change to previous notification? for each watcher subscriber (watcher) SUBSCRIBE NOTIFY

43. Policy rules There is no sharp geospatial boundary Presence contains other sensitive data (activity, icons, …) and others may be added Example: future extensions to personal medical data “only my cardiologist may see heart rate, but notify everybody in building if heart rate = 0” Thus, generic policies are necessary

44. GEOPRIV and SIMPLE architectures target location server location recipient rule maker presentity caller presence agent watcher callee GEOPRIV SIP presence SIP call PUBLISH NOTIFY SUBSCRIBE INVITE publication interface notification interface rule interface INVITE

45. Columbia SIP servers (CINEMA) Internal Telephone Extn: 7040 SIP/PSTN Gateway Department PBX Web based configuration Web server Telephone switch SQL database sipd: Proxy, redirect, registrar server Extn: 7134 xiaotaow@cs NetMeeting H.323 rtspd: media server sipum: Unified messaging Quicktime RTSP clients RTSP Extn: 7136 713x Single machine SNMP (Network Management) sipconf: Conference server siph323: SIP-H.323 translator Local/long distance 1-212-5551212

46. Location-based services in CINEMA Initial proof-of-concept implementation Integrate devices: lava lamp via X10 controller  set personalized light mood setting Pingtel phone  add outgoing line to phone and register user painful: needs to be done via HTTP POST request stereo  change to audio CD track based on user Sense user presence and identity: passive infrared (PIR) occupancy sensor magnetic swipe card ibutton BlueTooth equipped PDA IR+RF badge (in progress) RFID (future) biometrics (future)

47. PA device controller SUBSCRIBE to each room SUBSCRIBE to configuration for users currently in rooms 1.discover room URI 2.REGISTER as contact for room URI tftp HTTP Example: user-adaptive device configuration SLP “all devices that are in the building” RFC 3082? 802.11 signal strength  location REGISTER To: 815cepsr Contact: alice@cs SIP room 815

48. Emergency calling as an LBS Existing emergency call systems (“911”) will no longer work in IP-based networks current 911 system uses outmoded operator trunk technology Emergency calling = call identification  sip:sos@domain or tel:112 destination identification is this really an emergency call center? call routing to nearest emergency call center (ECC) Call routing is hardest must work internationally end system + network-based location determination Once solved: roadside emergency (AAA, ADAC, …) pizza emergency (nearest PizzaHut) but different privacy trade-offs  voluntary disclosure

49. Location-based call routing – UA knows its location GPS 48° 49' N 2° 29' E INVITE sips:sos@ DHCP outbound proxy server 48° 49' N 2° 29' E  Paris fire department

50. Location-based call routing – network knows location IP 48° 49' N 2° 29' E TOA include location info in 302 INVITE sips:sos@ INVITE sips:sos@paris.gendarme.fr map location to (SIP) domain outbound proxy

51. Service creation programmer, carrier end user network servers SIP servlets, sip-cgi CPL end systemVoiceXMLVoiceXML (voice), LESS Tailor a shared infrastructure to individual users traditionally, only vendors (and sometimes carriers) learn from web models

52. Call Processing Language (CPL) XML-based “language” for processing requests intentionally restricted to branching and subroutines no variables (may change), no loops thus, easily represented graphically and most bugs can be detected statically termination assured mostly used for SIP, but protocol-independent integrates notion of calendaring (time ranges) structured tree describing actions performed on call setup event top-level events: incoming and outgoing

53. CPL Location set stored as implicit global variable operations can add, filter and delete entries Switches: address language time, using CALSCH notation (e.g., exported from Outlook) priority Proxy node proxies request and then branches on response (busy, redirection, noanswer,...) Reject and redirect perform corresponding protocol actions Supports abstract logging and email operation

54. CPL example

55. <lookup source="http://www.example.com/cgi- bin/locate.cgi?user=jones" timeout="8">

56. CPL example: anonymous call screening <reject status="reject" reason="I don't accept anonymous calls" />

57. Service creation for presence services (work-in-progress) Accept or deny subscriptions Shape presence notifications different level of detail for family, friends and colleagues particularly important for geo data Subscriber can filter detail primarily, wireless bandwidth constraints rate limit notifications XPath? Mostly, condition/reaction  CPL can be extended to most of these functions

58. Pushing context-sensitive data to users User with mobile device should get location information when entering city, campus or building flight and gate information maps and directions local weather forecast special advisories (“choose security checkpoint 2”) Often does not require knowing user but interface with (e.g.) calendar Example Columbia implementation: OBEX data exchange over BlueTooth PDA pushes current appointment or event name base station delivers directions and map

59. Conclusion SIP + auxiliary protocols supports many of the core requirements for ubiquitous computing and communications: mobility modalities: terminal, user, session, service service negotiation for devices with different capabilities automatic configuration and discovery with SLP or similar event notification and triggered actions automatic actions: event filtering, CPL, LESS (for end system services) SIP offers a loosely-coupled approach (cf. Jini or object models) Also need data push functionality Avoid tendency to assume SIP users are human – want to interconnect different components and devices SIP device configuration needs automation, rather than screen- scraping