April 21, 2004 Internet2 RTC Forum Henning Schulzrinne Xiaotao Wu & CINEMA crew Columbia University From multimedia conferencing to context-aware communications

2 April 21, 2004 Overview Old challenge: any media, anywhere, anytime New challenge: appropriate and context-sensitive communications not just telephony not just videoconferencing on-demand, not special equipment, setup, arrangements Status of multimedia communications filling in the protocol matrix On-going work: presence-enabled multimedia communications mobility  terminal, personal, session, service creating new services in the web model, not the COBOL model Location-based services Challenges ahead

3 April 21, 2004 Internet services – the missing entry Service/deliverysynchronousasynchronous pushinstant messaging presence event notification session setup media-on-demand messaging pulldata retrieval file download remote procedure call peer-to-peer file sharing

4 April 21, 2004 Filling in the protocol gap Service/deliverysynchronousasynchronous pushSIP RTSP, RTP SMTP pullHTTP ftp SunRPC, Corba, SOAP (not yet standardized)

5 April 21, 2004 SIP as service enabler Rendezvous protocol lets users find each other by only knowing a permanent identifier Mobility enabler: personal mobility one person, multiple terminals terminal mobility one terminal, multiple IP addresses session mobility one user, multiple terminals in sequence or in parallel service mobility services move with user

6 April 21, 2004 Example SIP phones about $85

7 April 21, 2004 Ubiquitous computing aspects Also related to pervasive computing Mobility, but not just cell phones Computation and communications Integration of devices “borrow” capabilities found in the environment  composition into logical devices seamless mobility  session mobility adaptation to local capabilities environment senses instead of explicit user interaction from small dumb devices to PCs light switches and smart wallpaper

8 April 21, 2004 Context-aware communications Traditional emphasis: communicate anywhere, anytime, any media  largely possible today New challenge: tailor reachability Context-aware communications modify when, how, where to be reached  machine: context-dependent call routing  human: convey as part of call for human usage context-aware services leveraging local resources awareness of other users sources of location information voluntary and automatic location-based services  privacy concerns applies to other personal information activity, reachability, capabilities, bio sensor data, … emergency services as a location-based service

9 April 21, 2004 Context context = “the interrelated conditions in which something exists or occurs” anything known about the participants in the (potential) communication relationship both at caller and callee timeCPL capabilitiescaller preferences locationlocation-based call routing location events activity/availabilitypresence sensor data (mood, bio)not yet, but similar in many aspects to location data

10 April 21, 2004 “Legacy” IM & presence systems  SIP-based systems centralized systems (single name space) federated systems, similar to mostly instant text messages media-agnostic – transmit any media object separate from session-based services (VoIP, video conferencing) integrated: use IM as part of media sessions use presence to facilitate session setup limited presence status, mostly manually set rich presence, with time information imported from sensors, calendars, backend systems, … proprietary systems (AOL, Yahoo!, MSN, ICQ, …) standards-based systems

11 April 21, 2004 Presence and event notification Presence = special case of event notification “user Alice is available for communication” Human users: multiple contacts per presentity device (cell, PDA, phone, …) service (“audio”) activities, current and planned surroundings (noise, privacy, vehicle, …) contact information composing (typing, recording audio/video IM, …) Multimedia systems: REFER (call transfer) message waiting indication conference floor control conference membership push-to-talk system configuration General events: emergency alert (“reverse 911”) industrial sensors (“boiler pressure too high”) business events (“more than 20 people waiting for service”)

12 April 21, 2004 IETF efforts SIP, SIPPING and SIMPLE working groups but also XCON (conferencing) Define SIP methods PUBLISH, SUBSCRIBE, NOTIFY GEOPRIV: geospatial privacy location determination via DHCP information delivery via SIP, HTTP, … privacy policies SIMPLE: architecture for events and presence configuration (XCAP) session-oriented IM (↔ page mode) filtering, rate limiting and authorization

13 April 21, 2004 RPID: rich presence Provide watchers with better information about the what, where, how of presentities facilitate appropriate communications: “wait until end of meeting” “use text messaging instead of phone call” “make quick call before flight takes off” designed to be derivable from calendar information or provided by sensors in the environment allow filtering by “sphere” – the parts of our life don’t show recreation details to colleagues

14 April 21, 2004 RPID: rich presence Classification: contact-type device, in-person, service, presentity class for labeling sphere “work”, “home”, … relationship “family”, “associate”, “assistant”, “supervisor” Activities: activity “on-the-phone”, “away”, “appointment”, … idle last usage of device Surroundings: placetype “home”, “office”, “industrial”, … privacy “public”, “private”

15 April 21, 2004 CIPID: Contact Information More long-term identification of contacts Elements: card – contact Information home page icon – to represent user map – pointer to map for user sound – presentity is available

16 April 21, 2004 Timed Status Presence is about here & now but often only have (recent) past – e.g., calendar or future “will be traveling in two hours” “will be back shortly” allows watcher to plan communication loose synchronization of calendars open <fs:timed-status from=" T10:20: :00“ until=" T19:30: :00"> closed I'll be in Tokyo next week

17 April 21, 2004 GEOPRIV and SIMPLE architectures target location server location recipient rule maker presentity caller presence agent watcher callee GEOPRIV SIP presence SIP call PUBLISH NOTIFY SUBSCRIBE INVITE publication interface notification interface XCAP (rules) INVITE DHCP

18 April 21, 2004 Location-based services Finding services based on location physical services (stores, restaurants, ATMs, …) electronic services (media I/O, printer, display, …) not covered here Using location to improve (network) services communication incoming communications changes based on where I am configuration devices in room adapt to their current users awareness others are (selectively) made aware of my location security proximity grants temporary access to local resources

19 April 21, 2004 Location-based SIP services Location-aware inbound routing do not forward call if time at callee location is [11 pm, 8 am] only forward time-for-lunch if destination is on campus do not ring phone if I’m in a theater outbound call routing contact nearest emergency call center send to nearest location-based events subscribe to locations, not people Alice has entered the meeting room subscriber may be device in room  our lab stereo changes CDs for each person that enters the room

20 April 21, 2004 SIP URIs for locations Identify confined locations by a SIP URI, e.g., Register all users or devices in room Allows geographic anycast: reach any party in the room Room 815 sip:rm815 location beacon Contact: alice Contact: bob

21 April 21, Location Tracking Standard access points No client software “Skiff” monitors SA110 single board computer running Linux Report signal strength, MAC address of all packets seen by Jamey from HP

22 April 21, 2004 Privacy  Presence policy subscription policy event generator policy subscriber filter rate limiter change to previous notification? for each watcher subscriber (watcher) SUBSCRIBE NOTIFY

23 April 21, 2004 Policy relationships geopriv-specificpresence-specific common policy RPIDCIPID future

24 April 21, 2004 Privacy rules Conditions identity, sphere, validity time of day current location identity as or + Actions watcher confirmation Transformations include information reduced accuracy User gets maximum of permissions across all matching rules Extendable to new presence data rich presence biological sensors mood sensors

25 April 21, 2004 Example: user-adaptive device configuration “all devices that are in the building” RFC 3082? PA device controller SUBSCRIBE to each room SUBSCRIBE to configuration for users currently in rooms 1.discover room URI 2.REGISTER as contact for room URI tftp HTTP SLP signal strength  location REGISTER To: 815cepsr Contact: SIP room 815

26 April 21, 2004 Location-based IM & presence

27 April 21, 2004 Location-based call routing – UA knows its location GPS N 73.98E CN=us A1=NJ A2=Bergen INVITE DHCP outbound proxy server provided by local ISP? 40.86N 73.98E: Leonia, NJ fire dept. leonia.nj.us.sos.arpa POLY NAPTR …

28 April 21, 2004 Service creation programmer, carrier end user network serversSIP servlets, sip-cgi CPL end systemVoiceXMLVoiceXML (voice), LESS Tailor a shared infrastructure to individual users traditionally, only vendors (and sometimes carriers) learn from web models

29 April 21, 2004 Service creation environment for CPL and LESS

30 April 21, 2004 location-switch for CPL

31 April 21, 2004 Challenges Systems are still too hard to use without wizard assistance: lack of interoperability (improving) NAT and other configuration volume mismatch, echo, … audio problems not much changed since 1992 network/system fault diagnosis Closed wireless systems – would be very nice presence sensors Threat of “spim” and nuisance calls Provider platforms remain largely closed promise of open service creation remains to be fulfilled

32 April 21, 2004 Conclusion Standardization mostly complete even if drafts don’t have RFC numbers yet Many commercial-grade, second-generation products emerging both open-source and commercial emphasis on interoperability Increasingly hostile network multi-layer NATs, random port blocking, “transparent” proxies Usability and reliability remain too low dial-in audio conference still common LCD problem (cf. MIME for )