1. CINEMA Columbia InterNet Extensible Multimedia Architecture
Agenda
Basic IP telephony
Infrastructure and components
Additional services
Current and future work
Kundan Singh
Joint work with Wenyu Jiang, Jonathan Lennox, Sankaran Narayanan, Henning Schulzrinne, Xiaotao Wu and Min Yan
“Internet Real Time Lab”

2. Basic IP Telephony Audio Packet Transfer…
Digitization (e.g., sampling at 8kHz, 16 bits per sample, i.e, 128 kb/s or 320 bytes per 20 ms)
Real-time compression/encoding (e.g., G.729A at 8 kb/s)
Transport to remote IP address and port number over UDP (Why not TCP?)
Processing on receiver side is the reverse

3. Basic IP Telephony Sampling, Quantization, Encoding
+127 …
Encode each quantized sample into 8 bit code word
PCM: 8000 x 8 bits = 64 kb/s
Other techniques (differential coding, linear prediction)
2.4 kb/s to 64 kb/s +0
-127
Sample at twice the highest voice frequency
2 x 4000=8000
Round off samples to one of 256 levels (introduces noise)

4. Basic IP Telephony Problems with UDP
Sender 1 2 3 4 5 6 7
(a)
(b) 1 2 3 5 7 6
timeline
Receiver
Unreliable UDP
Packet loss
Out-of-order (very rarely)
Jitter (delay variation)

5. Basic IP Telephony Receive buffer
Sequence number: to detect packet loss; Just ignore the loss!
Receive buffer: to absorb jitter
Sender 1 2 3 4 5 6 7 8 9 1 2 3 4 1 2 3 5 7 6 8 9 2 3 4
Receiver 1 2 7 1 2 5 7 8 9 2 3 1 2 3 5 6 7 8 9 2

6. Basic IP Telephony Timestamp Vs sequence number
Silence suppression
Variable length packets
Sender t1 t2 t3 t4 t5 t6 t7 t8 t9 1 2 3
Silence … 4 5 6 7 1 2 3 4 5 6 7
Receiver
Playout time vs packet loss detection

7. Basic IP Telephony Real-time Transport Protocol - RTP
IP header V CC M
Payload type
Sequence number P X
UDP header
Timestamp (proportional to sampling time)
RTP Header
Source identifier (SSrc)
Encoded
Audio
Optional contributors’ list (CSrc)
msg
sendto(…, msg, …)
recvfrom(…, msg, …)

8. Basic IP Telephony RTP based conference
PCMU
Mixer
Transcoder 
PCMU 
G.729
G.729
PCMU
PCMU
An example RTP conference (without multicast)

9. Basic IP Telephony Why do we need signaling?
Register as
Alice=>
Query for Alice
Bob’s host
Alice’s host
What is the IP address of Alice’s host?
What audio encoding can it support?

10. Basic IP Telephony Session Initiation Protocol - SIP
Address based on
DNS SRV for SIP home.compc1.home.com
pc1.home.com 
INVITE
INVITE (proxy mode)
office.com
Bob
Alice
(2)
(3)
m2.home.com
Columbia.edu
Cisco.com
home.com

11. Basic IP Telephony SIP Message Format
Request
INVITE SIP/2.0
From: “Bob”
To: “Alice”
Subject: do you know SIP?
...
Response
SIP/ OK
From: “Bob”
To: “Alice”
Subject: do you know SIP?
...

12. Basic IP Telephony Session Description Protocol - SDP
INVITE Alice
I can support PCMU and G.729
Send me audio at :6780
Bob
Alice
OK; I can support PCMU
Send me audio at :8000
RTP
To port 8000
RTP
To port 6780

13. Basic IP Telephony SDP Message Format
Request
INVITE SIP/2.0
...
v=0
o=bob IN IP
s=SIP call
c=IN IP
t=0 0
m=audio 6780 RTP/AVP 0 8 5
m=video 6790 RTP/AVP 31
Response
SIP/ OK
...
c=IN IP
t=0 0
m=audio 8000 RTP/AVP 0 8
m=video 0 RTP/AVP 31

14. Basic IP Telephony Example scenario
Alice
pc5
(4)
Cisco.com
(3)
Office.com
Bob
(2)
(1)
Invite
Moved to
Invite
Invite
Home.com

15. Basic IP Telephony VoIP Gateways
Corporate/Campus
Another campus
7040
8151
External line
8152
(Plain Old) Telephone
Network
PBX
PBX
7042
8153
8154
7043
Gateway
Internet
LAN
LAN

16. Test-bed Architecture
OpenSource SQL database: MySQL
User information:
Contact location
Profile (e.g., password)
Aliases
Address book
System information
Configuration
SIP proxy,
redirect
server
SQL
database
sipd
Proxy, Redirect, Registration server.
Authentication
Programmable (SIP- CGI)

17. Test-bed Architecture
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
User profile can be modified using web browser.
Creating new user (admin/normal)
Changing profile and contact information (“follow me” service).
Web CGI scripts
Both sipd and web scripts use the database

18. Test-bed Architecture
IP phones connected to the departmental LAN. Users are identified by id, e.g.,
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
Software (sipc) for desktop. Allows audio, video, chat, white board, device control, instant message, presence and desktop sharing.

19. Test-bed Architecture
Registration can also be altered from the web interface
Web based
configuration
Web server
Phones register themselves with sipd when powered up.
SIP proxy,
redirect
server
SQL
database
sipd
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
Phones
Sipd stores the contact information in the database table:
=>
There can be multiple contacts. All registered phones ring, and the first to pick up is connected.

20. Test-bed Architecture
Another IP phone
(1) When somebody calls sipd gets the INVITE message and “proxies” the call to the current location.
Web based
configuration
Web server
(2) Phone rings, the user picks up the call and can talk to the caller.
SIP proxy,
redirect
server
SQL
database
sipd
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
Phones
Based on user profile, sipd may ask for caller authentication.

21. Test-bed Architecture
Telephone
Telephone
switch
External T1
Departmental PBX (Nortel Meridian) connects both internal and external lines to the gateway.
Dial “8” to reach outside line
Department
PBX
Web based
configuration
Web server
Internal T1
SIP proxy,
redirect
server
SQL
database
sipd
(Extension:713x)
T1/E1
RTP/SIP
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
Cisco 2600 router with SIP/PSTN gateway connects the departmental LAN with the PBX.

22. Test-bed Architecture
Dials
Telephone
Telephone
switch
(1) When PBX receives a call for , it forwards the call to extension is assigned to the gateway.
Department
PBX
Web based
configuration
Web server
Internal T1
SIP proxy,
redirect
server
SQL
database
sipd
(Extension:713x)
T1/E1
RTP/SIP
(2) The gateway forwards all PSTN calls to sipd;
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
(3) Sipd looks into the dialplan, finds a mapping and forwards the call to the current location of “hgs”.

23. Test-bed Architecture
Telephone
Telephone
switch
(4) PSTN user receives the call.
External T1
Dial “8” to reach outside line
(3) PBX forwards the call to external line.
Department
PBX
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
(2) Sipd authenticates the caller and checks permissions.
Sipd maps the number to adding the prefix “8” and the gateway address.
T1/E1
RTP/SIP
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
(1) The IP user dials

24. Test-bed Architecture
Telephone
Telephone
switch
Telephone number mappings and privileges modifiable from the web
External T1
Department
PBX
Web based
configuration
Web server
Internal T1
SIP proxy,
redirect
server
SQL
database
sipd
T1/E1
RTP/SIP
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
Can use no-DID (direct inward dialing) mode for more numbers.

25. Additional Services Advantage: cost savings + new services
“Think of receiving your voic messages in an that you can later play out in a conference to show it to others”
Easy integration of , web, instant messaging, etc.
Open architecture vs Close architecture (traditional telephones)

26. Services Programmable servers
“Forward my call to office phone during day time and to my mobile in the evening” or “do not allow to call me”
Server side scripts – similar to HTTP
SIP-CGI vs HTTP-CGI
SIP Java Servlet
Call Processing Language (CPL)

27. Test-bed Architecture
Telephone
Telephone
switch
Programmable server:
SIP-CGI and Call Processing Language (CPL)
Department
PBX
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
T1/E1
RTP/SIP
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
Scripts can be uploaded by clients also.

28. Services Management - SNMP
Constantly monitor the SIP server from remote
Report any problem (e.g., unauthorized registration request)
Use Simple Network Management Protocol (SNMP) – client server
SIP Management Information Base (MIB) – describe variables and and parameters for monitoring and control
SIP server implements MIB, can be controlled remotely by an SNMP client.

29. Test-bed Architecture
Telephone
Telephone
switch
SNMP agent for SIP MIB. Allows remote monitoring and control of the SIP server. (e.g., prompt when an unauthorized registration is attempted)
Department
PBX
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
T1/E1
RTP/SIP
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
SNMP
(Network
Management)

30. Services Interoperability with H.323
H.323 – Proposed by ITU-T
Based on previous H.324/H.320 architecture for video conferencing
Collection of other specifications (for signaling, control, transport, etc.)
Defines everything, from supported codecs to controlling remote camera. (vs SIP, define only what is needed)
NetMeeting – most commonly known client
- good introduction on H.323 (biased comparison with SIP!!)

31. Test-bed Architecture
Telephone
Telephone
switch
Allows an H323 client (Netmeeting) to use the services of our SIP infrastructure.
Department
PBX
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
T1/E1
RTP/SIP
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
SIPH.323
convertor
NetMeeting
siph323
H.323
SNMP
(Network
Management)

32. Services Unified messaging
Single messaging system – , voic , fax/video, etc.
Voice mail and answering machine – first step
Streaming media for storage and retrieval – Real-Time Streaming Protocol
RTSP – similar to HTTP, request-response, uses RTP
RealPlayer, QuickTime – popular clients
SIP for signaling of IP telephony calls
Access from telephone using a gateway

33. Test-bed Architecture
Telephone
Telephone
switch
rtspd
SIP/RTSP
Unified
messaging
RTSP media
server
sipum
Quicktime
RTSP clients
RTSP
Generic media server for playback and recording of messages. Can work with existing RTSP client, Apple’s QuickTime.
Department
PBX
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
T1/E1
RTP/SIP
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
SIPH.323
convertor
NetMeeting
siph323
H.323
SNMP
(Network
Management)
Provides a voice mail and answering machine service to all the registered users. Has web interface for accessing voice mails.

34. Services Conferencing
Multiparty conferencing using SIP and RTP
Models – centralized, decentralized, multicast
Audio mixing, you don’t get back your own audio
Decode-add-encode
Video replication
Centralized SIP control
Centralized RTP mixer

35. Test-bed Architecture
Telephone
Telephone
switch
rtspd
SIP/RTSP
Unified
messaging
RTSP media
server
sipum
Quicktime
RTSP clients
RTSP
Centralized conferencing server for audio and video. Users can join from IP as well as PSTN.
SIP
conference
server
sipconf
Department
PBX
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
T1/E1
RTP/SIP
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
SIPH.323
convertor
NetMeeting
siph323
H.323
SNMP
(Network
Management)

36. Test-bed Architecture
Sipd maps 7139=>
Telephone
Telephone
switch
PSTN user dials
rtspd
SIP/RTSP
Unified
messaging
RTSP media
server
sipum
Quicktime
RTSP clients
RTSP
SIP
conference
server
sipconf
Department
PBX
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
T1/E1
RTP/SIP
Netmeeting user dials
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
SIPH.323
convertor
NetMeeting
siph323
H.323
SNMP
(Network
Management)
SIP user dials

37. Services (What more?) Okay, SIP can do so many thing…
but can it turn on my lamp?
Answer is yes!!

38. Services SIP Extensions
SIP is just a session signaling protocol.
Can send message to a logical address (URI) and initiate any session
Extensions to do new things
Instant messaging; presence; event notification
Call Transfer
Device control (identify device as URI)

39. Test-bed Architecture
Telephone
Telephone
switch
rtspd
SIP/RTSP
Unified
messaging
RTSP media
server
sipum
Quicktime
RTSP clients
RTSP
SIP
conference
server
sipconf
Department
PBX
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
T1/E1
RTP/SIP
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
SIPH.323
convertor
NetMeeting
siph323
H.323
SNMP
(Network
Management)
Device gateway allows control of X10 devices. We can control a “lamp” using SIP DO message.
Device GW
X 10

40. Complete Architecture
Telephone
Telephone
switch
rtspd
SIP/RTSP
Unified
messaging
RTSP media
server
sipum
Quicktime
RTSP clients
RTSP
SIP
conference
server
sipconf
Department
PBX
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
T1/E1
RTP/SIP
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
SIPH.323
convertor
NetMeeting
siph323
H.323
SNMP
(Network
Management)
Device GW
X 10

41. (In)Complete Architecture
Telephone
Telephone
switch
rtspd
SIP/RTSP
Unified
messaging
RTSP media
server
sipum
Quicktime
RTSP clients
RTSP
Communication portal:
Address book, calendar, message board, file sharing, conferencing, etc.
Accounting and billing services
SIP
conference
server
sipconf
Department
PBX
Web based
configuration
Web server
SIP proxy,
redirect
server
SQL
database
sipd
T1/E1
RTP/SIP
e*phone
sipc
Software SIP
user agents
Hardware
Internet (SIP)
phones
SIP/PSTN Gateway
SIPH.323
convertor
NetMeeting
siph323
H.323
SNMP
(Network
Management)
Performance measurement and scalability for very large call/session volumes (for sipd, sipconf and rtspd).
SIP based Voice XML browser to allow access to voice mail and other services from touch tone phones
Device GW
X 10

42. Summary Internet Telephony is not only telephony on Internet !
Basic IP telephony (PC-to-PC)
VoIP gateway (PC-to-phone and phone-to-PC)
Additional services:
Multimedia
Web and integration
Instant messaging
Presence and event notification
Conferencing
Collaborative work environment
Media streaming
Unified messaging

43. Summary World of Protocols
Session Initiation Protocol (SIP) – signaling of Internet sessions
Real-time Transport Protocol (RTP) – transport of real-time multimedia
Real Time Streaming Protocol (RTSP) – signaling for multimedia streaming, both playing and recording
Session Announcement Protocol (SAP) – announcing Internet radio session information
Multimedia encoding, G.711, GSM, ADPCM, G.729, H.261, H.263, MPEG.

44. Summary Our IP telephony test-bed
For implementing new services
For test and performance measurement
For replacing the plain old PBX

45. Summary More Information
More information on the system design
More information on the software components:

46. Questions asked during the talks
Some of the questions that (I think) were raised and answered by the speaker during the talk.
Why not use IP address instead of SSRC in RTP?
What are the authentication mechanisms used in SIP?
Why is it not LGPL licensed? Are there other LGPL’ed SIP implementations? Does Columbia want to make money out of this?
What is the capacity of the system? How many users, calls can it handle?
Actual quality of audio, if we have tested between two IP phones on the real Internet?
Do you support any kind of encryption?

47. Additional slides

48. Components VoIP: Call from POTS to IP
Convert telephone number to a SIP address
ENUM: Use DNS lookup
If you dialed
Convert to
Do DNS lookup for arpa.in
Result will be a SIP URL, e.g.,
Private mapping within a domain by SIP server

49. Components VoIP: Call from IP to POTS
If you know gateway’s IP address, dial
TRIP: Telephone routing over IP – to discover a gateway that can route your number correctly
Gateway ??? free
Gateway ??????? 2¢/min
Gateway ??????? 2¢/min
Gateway ?????????? 4¢/min

50. Components Interactive Voice Response - IVR
Receives incoming PSTN5 call
Responds back with prompts
Accepts user input (DTMF or speech)
Takes action based on user input
(Usually the service logic is programmed for the specific application, say weather report)
PSTN
End user
IVR1 platform
Voice and telephony functions
(ASR2, TTS3, DTMF4)
Service logic (application specific)
[1] Interactive voice response
[2] Automated speech recognition
[3] Text to speech
[4] Dual tone multi-frequency (touch tone)
[5] Public switched telephone network

51. Components IVR – on internet
Voice gateway
Voice and telephony functions
Internet
Web server
Service logic
PSTN
End user
IVR platform
Voice and telephony functions
(ASR, TTS, DTMF)
Service logic (application specific)
End user

52. Components IVR – on Internet
PSTN
End user
Internet
Voice gateway
Web server
Service logic (CGI, servlet, JSP)
Voice and telephony functions
VoiceXML browser
VXML
HTML DB
Multimedia
Audio/
grammar
Scripts
Web server

53. Components VoiceXML Example
<form>
<field name=“drink”>
<prompt>Would you like Coffee, Tea, Milk or Nothing.</prompt>
<option value=“coffee”>coffee</option>
<option value=“tea”>tea</option>
<option value=“milk”>milk</option>
<option value=“nothing”>nothing</option>
</field>
<block>
<submit next=“ namelist=“drink”/>
</block>
</form>

54. Components Miscellaneous
Media Streaming
Multiparty conferencing
Voice mail and answering machine
Unified messaging
Event notification and presence
Collaborative work environment