1. Ubiquitous Programmable Internet Telephony End System Services Xiaotao Wu Internet Real Time Laboratory Thesis defense 02/06/2007

2. 2 Overview Communication services and where to run the services End system services Programmable – Service creation Analyzable – Feature interaction handling Intelligent – Feature learning Ubiquitous – Context-based services Implementations Other work

3. 3 Services and where to run services I analyzed the difference between end system services and services in the network (IPTS’00) End system services Programmable Analyzable Intelligent Ubiquitous Implementations Other work

4. 4 Where to run services? Internet

5. 5 Run services on end systems End devicesEnd serversNet UAsProxyBack-to-back user agent Number of users SingleSingle/trust ed users Multiple Call statesYes NoYes MediaYesYes/noYesNoYes/no Number of devices SingleMultipleSingleMultiple User interaction DirectIndirect*Indirect ManagingEnd user Admin. IPTS ‘00

6. 6 Services and where to run services End system services Programmable I defined a language for end system service creation. (ICC’03, RFC3880) Analyzable Intelligent Ubiquitous Implementations Other work

7. 7 End system service programming What do we need? A language and a tool End system service creation language Easy to understand Automatic service creation Portable Create once, run on different devices Easy to manage Facilitate feature interaction handling CPL, CCXML, APPEL, SPL…

8. 8 How to represent services? ECA (event – condition – action) Natural thinking of a decision making – a policy/rule set When I am in a conference, I will vibrate my device for my boss’s calls and reject all the other calls. Using decision trees to represent policy/rule sets (O(logN) execution time) For an incoming call If I am in a conference Vibrate my device Reject the call YES NO If the call is from my boss

9. 9 Use LESS effort to create more services LESS: Language for End System Services CPL: Call Processing Language (Jonathan, Henning, and myself) Simplicity Four kinds of elements: trigger, switch, action, modifier Tree-like structure, easy for feature interaction analysis Safety Type safety: XML-based, no user defined variables Control flow safety: tree-like structure without back-reference No direct memory access Default behavior for every tree branch Portability Handle user interactions and media operations Extensibility not just new elements, but can apply existing algorithms IEEE ICC’03 RFC3880

10. 10 LESS elements trigger an incoming call switch check the caller switch check time switch check priority action redirect action accept action reject modifier to Bob = ≠ = ≠ ≠

11. 11 CUTE (Columbia University Telecommunication service Editor)

12. 12 Survey on end user service creation Group1: IRT members, Group2: CS undergraduates, Group3: Other people 85% would like to create their own services, 90% like to use CUTE to create services 90% can correctly create service-1, 65% srv-2, 80% srv-3, 65% easy to understand LESS code

13. 13 Services and where to run services End system services Programmable Analyzable I defined an algorithm handling feature interactions (FI) for LESS. ICFI’05, JCN) Intelligent Ubiquitous Implementations Other work

14. 14 ring What is FI and how to handle it? Tree merging + = If time is between 10:00AM and 11:00AM If address is hgs Forward to conf Incoming call If time is between 10:00AM and 11:00AM If address is hgs reject Forward to conf reject accept Take actions from both scripts. Simply setting precedence rules cannot work.

15. 15 FI handling for LESS Action conflict tables Services conflict only if their actions conflict Tree merging algorithm Detect and help to resolve Resolve conflicts ICFI’05 (FIW) JCN

16. 16 Pre-condition and expected results pre-conditionexpected results accept The call setup is pending. The audio device is available. The call setup is finalize. The communication session is setup. The audio device is occupied. reject The call setup is pending.The call setup is finalized. redirect The call setup is pending.The call setup is finalized on the current end system. call The audio device is available.If the callee side accepts the call, a communication session is setup and audio device is occupied.

17. 17 Action conflict table acceptrejectredirectcall accept A(media)CCR reject CA(reason)C- redirect CCA(target)- call R--A(target) -: no interaction, A: attribute conflict, C: action conflict, E: enabling, R: resource competition

18. 18 Tree merging (overall) set base-rule-set empty foreach LESS-tree { convert the LESS-tree into a rule set foreach rule in the rule set { normalize the rule } merge the normalized rule set into base-rule-set } convert base-rule-set into a decision tree conflicting-free rule set

19. 19 Tree merging (merging) if (two rules have different triggers) { no rule conflict except timer trigger } else if actions in two rules do not conflict { no rule conflict } else if no overlap between rule path in two rules { no rule conflict } else { two rules conflict with each other, return the rule path overlap and action conflict information prompt to the script owner to judge } Ahmed Layouni from Univ. of Ottwa verified my work by using a model-checking tool

20. 20 Resolving interactions condition decision options with lower risks

21. 21 Services and where to run services End system services Programmable Analyzable Intelligent I proposed a method for automatic service creation. (ICC’05) Ubiquitous Implementations Other work

22. 22 No idea about services? Learning burden caused by new services What and how Help, not bypass Causal relationship between call information and call decisions SIP headers Different information sources Examples Spam filtering, calendar-based services

23. 23 What (learn from), what (generate), how Users’ communication history – LESS decision trees – decision tree induction find switches that can best partition actions Algorithm Incremental Prune Quality measurement 30*3+10*2+7=117 30*2+3*2+10*3+4*3=108

24. 24 Incremental Tree Induction Incremental Incorporating Transposition Virtual prune Direct metrics Expected number of tests Leaf counts Minimum description length Expected misclassication cost

25. 25 Simulation 40 services Each for 300 calls 80% match 10% different way 10% mismatch

26. 26 Performance Fast vs. incremental (20 samples) training IBM ThinkPad, Linux 1GHz PIII Mobile 256MB memory

27. 27 How to handle service risks? Identify Lose connection: reject, redirect, transfer, accept on wrong branch Lose privacy: accept, call, notify Lose money: accept, transfer to higher rate endpoint Lose attention: alert, accept, appliance control Analyze Possibility: number of occurrence in the decision tree Impact: (connection, privacy) > money > attention, customizable Resolve Change communication methods Change communication targets Reduce overall risk: avoiding high impact risks, though may cause low impact risks Contingency plan Backup

28. 28 Services and where to run services End system services Programmable Analyzable Intelligent Ubiquitous Joint work on several related projects – a ubiquitous computing architecture, location-based services, SIP 911 (Nossdav’03, IEEE Communications, CCNC’05, ICCCN’05) Implementations Other work

29. 29 -- Mark Weiser Ubiquitous computing using SIP What is ubiquitous computing Enhance computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user. where are we what’s available how to control automate the control Service location server room 123 what’s available in room 123? video display microphone video camera Bob I am in room 123 Bob can use devices in 123 use the devices in room 123 to talk to Tom Tom send media to Tom

30. 30 System Architecture Location Sensing Resource Discovery Resource Control Call Control GPS BlueTooth DHCP Wireless triangulation NOTIFY PUBLISH REGISTER SIP Home domain Registrar and AAA server Location Server Local domain SIP server SLP DA INVITE SLP SA SLP query result NOSSDAV03 IEEE Communications Stylianos, Stefan, Henning, myself

31. 31 Room conf Location agent Bob RFID reader iButton reader Proxy LS Bob is in conf NOTIFY Location You are In conf SLP DA SLP SA Device GW SLinke X10 Turn on conf’s light Turn on light What’s available sip:conf_pingtel for audio sip:conf Tracking Trigger an action Resource discovery IEEE CCNC’05 Location-based Services in our lab Ron,Matthew, Kundan, and myself

32. 32 Location-based Services in our lab Location agent Bob RFID reader iButton reader Proxy LS Bob is in conf NOTIFY Location You are In conf SLP DA SLP SA Device GW SLinke X10 Turn on conf’s light Turn on light What’s available sip:conf_pingtel for audio sip:conf INVITE sip:anyone_roomconf Guard communicatio n behavior ‘Talk’ to a location Room conf IEEE CCNC’05

33. 33 Where to put services End system services Programmable Analyzable Intelligent Ubiquitous Implementations SIPc – multi-function integration (MMNS’04) Other work

34. 34 Implementations

35. 35

36. 36 Function overview SIP Multimedia call control Real time streaming Location sensing Network appliance control Floor control SIP for presence SAP Instant message SIP CGI engine LESS/CPL engine Third party call control Emergency handling Service Location Detection (SLP) audio video white board desktop sharing location sensors Email clients RTP: RFC 1889 SDP: RFC 2327RTSP: RFC 2326 SIP Event Arch.: RFC 3265 RFC3903 SAP: RFC 2974SIP: RFC 3261 SLP: RFC 2608 CPL, SIP 3PCC, SIP Device Control GEOPRIV location format, SIP for IM Web browsers MMNS’04

37. 37 Call SIP SDPRTP Session broadcasting SAP RTSP SIP event notification Location sensing Emergency handling Location tracking Device control ir/x10 MapLynx Message waiting indication Voicemail handling Presence notification Conferencing floor control Service detection SLP Instant messaging xcon Function sharing

38. 38 SDP SAP RTP RTSP SIP location SLP 3pcc SIP DO SLP SIP NOTIFY MESSAGE DO SIP location Function interaction

39. 39 Where to put services End system services Programmable Analyzable Intelligent Ubiquitous Implementations Other work

40. 40 Other work Building Box project (AT&T Labs Research) Conferencing floor control (RFC4376) Networked appliance control Shared web browsing Distributed conferencing and MOC integration (Avaya Labs Research) Joined several other projects (CINEMA, SIP 911, session mobility, FAA pilot training)

41. 41 Summary I defined LESS and built CUTE. I also conducted a survey on end system service creation. The survey shows LESS and CUTE are good for end system service creation. I developed an algorithm to handle feature interactions in LESS. The algorithm is easy to implement and can detect and help to resolve feature interactions in LESS. I proposed to use decision tree induction to automatically create services for end users and defined ways to handle service risks. Automatic service creation can help users to create their desired services. I investigated how to perform ubiquitous communication and location- based services in end systems. The ubiquitous communication architecture we proposed is scalable and can use off-the-shelf hardware and software for communication. I implemented a SIP user agent, SIPc, and integrated my research result into SIPc. Multi-function integration in SIPc shows that convergence and multi-function interaction can bring new communication services.

42. 42 Some links SIPc: http://www.cs.columbia.edu/IRT/sipchttp://www.cs.columbia.edu/IRT/sipc CINEMA: http://www.cs.columbia.edu/IRT/cinemahttp://www.cs.columbia.edu/IRT/cinema LESS: http://www.ietf.org/internet-drafts/draft-wu-iptel-less-00.txthttp://www.ietf.org/internet-drafts/draft-wu-iptel-less-00.txt CUTE: http://www.cs.columbia.edu/~xiaotaow/cutehttp://www.cs.columbia.edu/~xiaotaow/cute Ubiquitous Computing: http://www1.cs.columbia.edu/~xiaotaow/ rer/Research/Paper/ieeecomm_inhome.pdfhttp://www1.cs.columbia.edu/~xiaotaow/ rer/Research/Paper/ieeecomm_inhome.pdf Service examples: http://www.cs.columbia.edu/~library/TR- repository/reports/reports-2004/cucs-048-04.pdfhttp://www.cs.columbia.edu/~library/TR- repository/reports/reports-2004/cucs-048-04.pdf Feature interaction handling: http://www.cs.columbia.edu/ ~xiaotaow/rer/Research/Paper/fiw.pdfhttp://www.cs.columbia.edu/ ~xiaotaow/rer/Research/Paper/fiw.pdf Service learning: http://www.cs.columbia.edu/~xiaotaow/rer /Research/Paper/icc2005.pdfhttp://www.cs.columbia.edu/~xiaotaow/rer /Research/Paper/icc2005.pdf

43. 43 Acknowledgements Mentor: Dr. Henning Schulzrinne All the thesis committee members People in IRT and the CS Department Colleagues in AT&T Labs Research and Avaya Labs Research SIPQuest

44. 44

45. 45 End-to-end v.s. master/slave Pickup call D1D2 D3 C D1D2 D3 C notification triggered INVITE MGC MGCP Megaco

46. 46 Timer triggered outgoing call <less xmlns="urn:ietf:params:xml:ns:less “ xmlns:IM="urn:ietf:params:xml:ns:less:im “ xmlns:xsi= “… " xsi:schemaLocation= “… "> <status-switch uri="sip:bob@example.com" status-name="presence"> Hi, please call me back. I am in office …………….

47. 47 LESS elements (triggers) Triggers incoming: incoming call handling outgoing: user invoked outgoing call timer: timer triggered actions UI:command: user interaction commands IM:message: incoming instant messaging Event:subscription: incoming subscription Event:notification: incoming notification

48. 48 LESS elements (switches) Switches time-switch: make decisions based on time address-switch: make decisions based on caller, callee priority-switch: make decisions based on call priority string-switch: make decisions based on subject, … language-switch: make decisions based on languages status-switch: make decisions based on users’ status (remote user or local user, status includes presence, activity, mood, …, as listed in RPID) Event:event-switch: check values in event notifications LOC:where-switch: check users’ physical location information (remote or local user) LOC:where-relation-switch: check relative physical locations between two people

49. 49 LESS elements (actions) Actions accept: accept an incoming call reject: reject an incoming call redirect: redirect an incoming call authenticate: authenticate an incoming request call: make an outgoing call terminate: disconnect a call wait: wait for a certain time before next action mail: send email log: log request handling process Media:mediaupdate: update media attributes Midcall:transfer: transfer a call Midcall:merge: merge multiple calls UI:alert: alert user UI:getinput: get user input IM:sendmsg: send an instant message Event:approve: approve subscription Event:deny: deny event subscription Event:defer: defer the decision on event subscription Event:subscribe: send subscription out Event:notify: send notification out Queue:enqueue: put a call and its context into a queue Queue:dequeue: get a call and its context from a queue

50. 50 LESS elements (modifiers) Two smaller concepts might be simpler and more flexible than one more powerful but complicated concept Modifiers location: to which a request to be directed lookup: lookup locations from a source remove-location: remove locations from location set Media:media: provide media attributes

51. 51

52. 52 LESS script customization xsl:if LESS editor service.less (template) XSLT less.xsl configuration editor service.html translate.cgi service_foo.less address is=$var

53. 53 LESS elements

54. 54 Example: Automatic Call Back (ACB) <less xmlns="urn:ietf:params:xml:ns:less“ xmlns:Event="urn:ietf:params:xml:ns:less:event“ xmlns:Queue="urn:ietf:params:xml:ns:less:queue“ xmlns:xsi=“….“ xsi:schemaLocation=“……"> <status-switch status-name=“activity”> <Queue:enqueue queue="callback"/> In ITU Q.1211 “This feature allows the called party to automatically call back the calling party of the last call directed to the called party.” Check my activity for an incoming call Use Event and Queue extension If I am on-the-phone Reject and enqueue

55. 55 <Event:event package=“presence" name=“activity" is=“normal"> <Queue:dequeue queue="callback"> A event notification for myself I am available Dequeue and make a call Automatic Call Back (ACB) (cont.)

56. 56

57. 57

58. 58 Rich signaling information SIP headers Caller preference and callee capabilities MIME contents Event notification Other means Web calendar, Directory services

59. 59 Rich services Be able to handle services in PSTN networks ITU Q.1211 ABD, ACB, CFC, CHA, QUE, CRG, OCS, … Services in 5ESS switches Attendant camp-on, Automatic recall, … Services in CSTA Phase III defined as signaling actions in LESS, e.g., mediaupdate Emergency provide location information New services Interact with existing Internet services web, email, SLP, SAP, IM, presence, location, networked appliance control, directory service, calendar service, conferencing Not named services, but programmable services Programmable conferencing services

60. 60 Definition What is service learning Automatically generate user desired services Help users, not bypass users Services on both proxy servers and end systems What is service risk management Risk caused by automation How to reduce the overall risks IEEE ICC’05

61. 61 Decision tree induction Entropy: -Σ P log2P 30 rejects, 17 accepts -30/47*log2(30/47) -17/47*log2(17/47) = 0.944087 Split on caller=Bob -30/33*log2(30/33) -3/33*log2(3/33) -14/14*log2(14/14) = 0.439497 Information gain: entropy change after splitting Information gain on the splitting is 0.50459

62. 62 Decision tree induction Find the splitting that can get the largest information gain Repeat for all sub-trees until no more information gain Noisy data and prune Splitting causes higher error

63. 63 Why not just a service creation tool Portability Java and scripting languages are also “portable” “I am happy to see a piece of technology that works so well that I’m free to ignore its inner details.” Maybe not some complicated work ”I also want to have the low-level innards visible, controllable when possible, and modifiable by anyone who’s willing and able to do that work.” Lower the bar for understanding the low-level innards

64. 64 Open issues Can we use LESS for B2BUA? coordinate multiple sessions multi-user feature interaction handling Loop and user-defined variables needed? Based on my exercises, no But, what about unknown new services? Convert loop to a high-level abstraction? What’s the impact on feature interaction handling

65. 65 Location in Emergency Services Emergency Call Center Call Flow Prototype Architecture SIP Proxy Internet ALI ServerDHCP Server DNS Server 911 112 sip:sos@domain w/location or w/out location geo location POTS/Wireless Network IP Network DHCP Inform MAC Address Location Info TCP Socket Telephone Number PSAP Info HTTP SOAP geo location verified civil location civil location** PSAP Info DNS Query civil location IEEE ICCCN’05 Henning, Anshuman, Matthew, Amrita, Jong Yul, Wonsang, and myself

66. 66 Program location-based services <LOC:where country="USA" A1="New York" A3="New York" A6="West 120th" HNO="450" LOC="Room 563">

67. 67 Related work on CPL FI handling Related work Syntax correct, semantic warnings e.g., parent switch and child switch mutually exclusive Translate to formal languages to check FI with other complex services

68. 68 Performance 20 vs. 250 incremental samples IBM ThinkPad, Linux 1GHz PIII Mobile 256MB memory

69. 69 Integration Gather information SIP transaction history Calendar Location sensing Idle time User activities Timestamp Alert users Service risk management