Carrier-grade vs. Internet VoIP Henning Schulzrinne (with Wenyu Jiang) Columbia University FCC Technical Advisory Council III Washington, DC – October 20, 2003

Overview Previous talk: interactive communication services signaling & media Now focus on overall architecture: network & service availability signaling services: SIP, H.323 supporting services: DNS, DHCP, LDAP, … network transport network quality-of-service packet loss, delay, jitter

Overview (on-going work, preliminary results, still looking for measurement sites, …) Service availability Measurement setup Measurement results call success probability overall network loss network outages outage induced call abortion probability

Service availability Users do not care about QoS at least not about packet loss, jitter, delay rather, it’s service availability  how likely is it that I can place a call and not get interrupted? availability = MTBF / (MTBF + MTTR) MTBF = mean time between failures MTTR = mean time to repair availability = successful calls / first call attempts equipment availability: % (“5 nines”)  5 minutes/year AT&T (2003): Sprint IP frame relay SLA: 99.5% Long-distance voice99.978% ATM data99.999% Frame relay data99.998% IP99.991%

Availability – PSTN metrics PSTN metrics (Worldbank study): fault rate “should be less than 0.2 per main line” fault clearance (~ MTTR) “next business day” call completion rate during network busy hour “varies from about 60% - 75%” dial tone delay

Example PSTN statistics Source: Worldbank

Measurement setup Node nameLocationConnectivityNetwork columbiaColumbia University, NY>= OC3I2 wustlWashington U., St. LouisI2 unmUniv. of New MexicoI2 epflEPFL, Lausanne, CHI2+ hutHelsinki University of TechnologyI2+ rrNYCcable modemISP rrqueensQueens, NYcable modemISP njcableNew Jerseycable modemISP newportNew JerseyADSLISP sanjoseSan Jose, Californiacable modemISP sunaKitakyushu, Japan3 Mb/sISP shShanghai, Chinacable modemISP ShanghaihomeShanghai, Chinacable modemISP ShanghaiofficeShanghai, ChinaADSLISP

Measurement setup Active measurements call duration 3 or 7 minutes UDP packets: 36 bytes alternating with 72 bytes (FEC) 40 ms spacing September 10 to December 6, ,500 call hours

Call success probability 62,027 calls succeeded, 292 failed  99.53% availability roughly constant across I2, I2+, commercial ISPs All99.53% Internet299.52% Internet % Commercial99.51% Domestic (US)99.45% International99.58% Domestic commercial 99.39% International commercial 99.59%

Overall network loss PSTN: once connected, call usually of good quality exception: mobile phones compute periods of time below loss threshold 5% causes degradation for many codecs others acceptable till 20% loss0%5%10%20% All ISP I I US Int US ISP Int. ISP

Network outages sustained packet losses arbitrarily defined at 8 packets far beyond any recoverable loss (FEC, interpolation) 23% outages make up significant part of 0.25% unavailability symmetric: A  B  B  A  spatially correlated: A  B   A  X  not correlated across networks (e.g., I2 and commercial)

Network outages

no. of outages % symmetric duration (mean) duration (median) total (all, h:m) outages > 1000 packets all10,75330% :2010:58 I %360253:172:33 I2+2,70810%259267:475:37 ISP8,04537%107249:334:58 US1,77718%269205:183:53 Int.8,97633% :026:42

Outage-induced call abortion probability Long interruption  user likely to abandon call from E.855 survey: P[holding] = e -t/17.26 (t in seconds)  half the users will abandon call after 12s 2,566 have at least one outage 946 of 2,566 expected to be dropped  1.53% of all calls all1.53% I21.16% I2+1.15% ISP1.82% US0.99% Int.1.78% US ISP0.86% Int. ISP2.30%

Conclusions from measurement Availability in space is (mostly) solved  availability in time restricts usability for new applications initial investigation into service availability for VoIP need to define metrics for, say, web access unify packet loss and “no Internet dial tone’’ far less than “5 nines” working on identifying fault sources and locations looking for additional measurement sites

What’s next? Existing SLAs are mostly useless too many exceptions wrong time scales: month vs. minutes no guarantees for interconnects Existing measurements similarly dubious Limited ability to learn from mistakes what are the primary causes of service unavailability? what can I do to protect myself – multi-homing via same fiber? diverse access mechanisms? Consumers of services have no good ways to compare service availability only some very large customers may get access to carrier-internal data Thus, market failure Need published metrics similar to switch availability reporting