October 4, 2000 © 2000, Columbia University Kundan Singh Internet Real-Time Lab. Columbia University What it means ? What can we do ? How can we do ? What we have done ? Q/A Multimedia Communication Applications

October 4, 2000 © 2000, Columbia University Multimedia + Communication Audio (voice + music) Video Data Internet (IP) Telephone (PSTN) Interactive Vs non-interactive

October 4, 2000 © 2000, Columbia University Applications Yahoo audio chat Telephony over Internet Dialpad /phone Unified messaging Internet radio/TV Video conferencing Instant messaging and presence Multi-user games

October 4, 2000 © 2000, Columbia University Approaches Telco (ITU-T) IETF CollaborativeWork Games Web based services …add multimedia to my PSTN, ISDN, ATM, … use it for internet… …signaling and media are different … …develop and use mbone tools: vic, vat, rat,… …add new services ( , voice mail, fax) accessible from web … …add any network support to my multi-user games…

October 4, 2000 © 2000, Columbia University Signaling Protocols Telco (ITU-T) IETF H.320 (ISDN) H.324 (POTS) H.323 (LAN Internet) (H.245, H.225, H.450.x, H.332, …) RTP/RTCP (Media transport) HTTP (Web) RTSP (Streaming) SIP (Session initiation) … RMI different approach

October 4, 2000 © 2000, Columbia University Scenarios Client/server – RTSP media player and RTSP serverClient/server – RTSP media player and RTSP server Distributed infrastructure – VoIPDistributed infrastructure – VoIP Multiparty work - conferencingMultiparty work - conferencing Messaging - Messaging

October 4, 2000 © 2000, Columbia University Another approach Focus on multimedia communication components instead of specific applications Audio input Audio output Video input Display Storage Different communication protocols (SIP, RTSP, H.323,…)

October 4, 2000 © 2000, Columbia University What you need ? Ease of development and experimentation - Examples - … write a software which calls me on my phone whenever I get an important mail … … call a number and leave a message. The message is converted to text and sent as …

October 4, 2000 © 2000, Columbia University What you need ? Framework for development Properties: scalable, extensible, interoperable, easy to use API Streaming application, Multimedia conferences, Gateways, … RTP/RTCP, SIP, RTSP, DNS, H.323, … Applications Libraries

October 4, 2000 © 2000, Columbia University CINEMA Columbia InterNet Extensible Multimedia Architecture SIP, RTSP (and HTTP) Additional external libraries: RTP, H.323, … API: C++ and Java

October 4, 2000 © 2000, Columbia University SIP/SDP Parser Authentication User registration Dynamic session change SIP/SDP parser Authentication Basic and Digest User registration CGI/CPL upload Dynamic session change Components to be added... Call transfer Three party call Instant messaging and presence Easy to use !

October 4, 2000 © 2000, Columbia University Related work API JTAPI Java Servlet JAIN SIP API MASH

October 4, 2000 © 2000, Columbia University CINEMA sipdsip323sipconfsipumsipgwrtspd CINEMA libNT Win32 stub libcine Utilities parsing libsip Basic SIP library libsip++ SIP UA library libmixer RTP audio mixer libdict Hash table libdb++ mySQL intf RTSP media server SIP proxy server SIP/H.323 gateway SIP conferencing SIP/RTSP unified messaging SIP/MGCP gateway LDAP Berkeley DB xml4j OpenH323 PGP PWLib Resparse

October 4, 2000 © 2000, Columbia University Example: audio conferencing Types Centralized Distributed Multicast Local media mixing

October 4, 2000 © 2000, Columbia University Example: audio conferencing Types Centralized Distributed Multicast Local media mixing A BC D

October 4, 2000 © 2000, Columbia University Example: audio conferencing Types Centralized Distributed Multicast Local media mixing Use multicast for media traffic.

October 4, 2000 © 2000, Columbia University Example: audio conferencing Types Centralized Distributed Multicast Local media mixing Mix audio A B C

October 4, 2000 © 2000, Columbia University Sipconf - 1 Features SIP/SDP and RTP/RTCP Audio mixing – sender does not get his own audio Play-out delay algorithm Web based setup G.711 A and Mu law, G.721, DVI ADPCM Multiple simultaneous conferences SIP based audio conference server

October 4, 2000 © 2000, Columbia University Sipconf - 2 Comparison with other types Multicast – deployment Simple endpoints – no local mixing Overall bandwidth Add all and subtract one – O(N); N=participants Extensible – codecs, video Heterogeneous participants Silence suppression - scalability

October 4, 2000 © 2000, Columbia University What else ? SIP based Audio/Video conferencing bridge Web based SIP client SIP/PSTN gateway SIP/RTSP based voice and video mail SIP for instant messaging and presence SIP based /phone

October 4, 2000 © 2000, Columbia University Missing pieces QoS Billing Security Telephone interface...

October 4, 2000 © 2000, Columbia University Summary What it means ? -multimedia + communication What can we do ? -chat, radio/TV, games, telephony, … How can we do ? - SIP, RTSP, RTP, H.323, … What we have done ? - sipc, sipd, rtspd, libsip++, sip323, sipum, sipconf, …

October 4, 2000 © 2000, Columbia University Additional material

October 4, 2000 © 2000, Columbia University e*phone H.323 SIP PSTN sipgw PBX SIP/RTP based embedded ethernet phone device (user-agent) Desktop based SIP user-agent Audio, video, white-board, chat, address book SIP server (proxy, redirect, registrar) Authentication, sip-cgi, Ldap, unix and windows SIP/H.323 signaling gateway built-in gatekeeper, fast-start as well as non- fast-start, multiple calls, transparent media, unix and windows SIP audio conference server audio mixing, play-out delay, multiple conferences, web setup SIP/RTSP based unified messaging voice messaging, answering machine, web based setup, and web integration Missing component … SIP/PSTN gateway sipc

October 4, 2000 © 2000, Columbia University Related work - Parlay Technology independent interface for telecom Services developed outside core network Access network function in secure way APIs : IT applications  Communication world Service interface: Network capabilities and information Call management, messaging, user interaction, billing, policy management… Framework interface: external interface in secure and manageable way Service registration, subscription, discovery Authentication, authorization Integrity management and other management support Call model  not suited for Internet telephony

October 4, 2000 © 2000, Columbia University Related work - JTAPI (Java) Telephony API – connection, call, numbers, … Not suited for Internet Telephony

October 4, 2000 © 2000, Columbia University Related work – SIP-CGI, CPL Control of server behavior SIP-CGI  similar to HTTP CGI Call processing language is XML based

October 4, 2000 © 2000, Columbia University Related work – SIP Servlet Similar to HTTP (Java) Servlet Advantages: stateful, low overhead, typed API, other Java based services like directory service, database, JMF… Example: location manager, redirection or proxy server, … More close to SIP-CGI

October 4, 2000 © 2000, Columbia University Related work – JAIN Java API for Integrated Networks  PSTN, wireless, Internet. Very extensive framework Protocol API spec: ISUP, MGCP, SIP, … Application API spec: JCC/JCAT  Call model vs transaction model, JSPA  Parlay, SCE/SLEE  service creation and logic execution environment, … JAIN SIP API: Stack provider (JainSipProvider) and Application (JainSipListener)

October 4, 2000 © 2000, Columbia University Related work – MASH Multimedia architecture for communication and collaboration over scalable and heterogeneous networks using IP multicast Technologies SRM Active objects Scalable consensus based bandwidth allocation Media archival and playback Coordination and control Transmission over heterogeneous networks Mash shell  OTcl multimedia components Multimedia collaboration tools  vic, vat, …

October 4, 2000 © 2000, Columbia University Related work CINEMA concept is close to MASH – SIP and RTSP based API Provides different levels of abstraction Low level – like JAIN SIP API and SIP Servlet API High level – Call model for user agents (and servers) Can inter-work with SIP-CGI, CPL

October 4, 2000 © 2000, Columbia University Thank you