1. 1 How is the Internet Performing? Les Cottrell – SLAC Ecole SIG at nouvelles Technologies en Democratic Republic Congo, 12-17 Septembre, Organisee par l’Universite de Kinshasa www.slac.stanford.edu/grp/scs/net/talk11/perform.ppt

2. 2 Overview Internet characteristics –Users, capacities, satellites, packet sizes, protocols, routing, flows How is it used apps etc. How the Internet worldwide is performing as seen by various measurements and metrics Application requirements

3. USERS 3

4. Internet Usage growth ‘95-’10 4 95 00 05 09 Millions of Users Penetration % 0 200 400 600 Asia Europe N Amer L Amer Africa M East Austrlasia N America Australia Europe L America M East Asia Africa World 0 20% 50% 80% Millions of users Year 1500 1000 500 2.09B Mar 2011

5. Example: China China not connected to the Internet until May 1994 –1 st permanent IHEP/Beijing used satellite via SLAC –www.computerworld.com.au/article/128099/china_cel ebrates_10_years_being_connected_internetwww.computerworld.com.au/article/128099/china_cel ebrates_10_years_being_connected_internet 5

6. Where are they 6 Internet city connections Internet Users 2002 2.8% growth/year ~¼ world pop uses Internet Developed world saturating Developing catching up 73% penetration US 43% users from Asia

7. Capacities 7

8. What have they got? 8 Capacity From Telegeography Capacity

9. Who is still on Satellite 9 Terrestrial GEOS Min RTT (ms) GEOS (Geostationary Earth Orbit Satellite) good coverage, but expensive in $/Mbps broadband costs 50 times that in US, >800% of monthly salary c.f. 20% in US AND long delays min RTT > 450ms, usually much larger due to congestion Easy to spot Clear signature

10. Packet sizes & types 10

11. 11 Packet size primarily 3 sizes: WHY? Packet size (bytes) Cu,mulative probability % Packets Bytes Mean ~ 420Bytes, median ~ 80Bytes Measured Feb 2000 at Ames Internet eXchange ~ 84M packets, < 0.05% fragmented close to minimum=telnet and ACKs, 1500 (max Ethernet payload, e.g. FTP, HTTP); ~ 560Bytes for TCP implementations not using max transmission unit discovery

12. 12 Internet protocol use There are 3 main protocols in use on the Internet: –UDP (connectionless datagrams, best effort delivery), –TCP (Connection oriented, “guaranteed” delivery in order) –ICMP (Control Message protocol) Time Feb-May 2001 Flows/10min In Out TCP dominates today SLAC protocol flows TCP UDP ICMP

13. Routing 13

14. 14 Hops Hop counts seen from 4 Skitter sites (Japan, S. Cal, N. Cal, E. Canada, i.e. 10-15 hops on average Hop Count Weak RTT dependence on hop count 95% 50% 5% RTT Hops

15. 15 Richness of connectivity Angle = longitude of AS HQ in whois records Radius=1-log(outdegree(AS)+1)/(maxoutdegree + 1) –Outdegree = number of next Hops As’ accepting traffic Deeper blue & red more connections All except 1 of top 15 AS’ are in US, exception in Canada Few links between ISPs in Europe and Asia

16. Today’s routing less via US www.nytimes.com/2008/08/30/business/30pipes.html 16 Invented in US 1 st 30 yrs most traffic thru US 70%=>20% in 10yrs No central control Patriot act=>store info outside US China, India, Japan making larger investments More level playing field Harder for CIA!

17. 17 Routes are not symmetric Min, 50% & 90% RTT measured by Surveyor Notice big differences in RTTs May be due to different paths in the 2 directions or to different loading Advanced to U. Chicago RTT ms U. Chicago to Advanced

18. Flows 18

19. 19 Flow sizes Heavy tailed, in ~ out, UDP flows shorter than TCP, packet~bytes 75% TCP-in < 5kBytes, 75% TCP-out < 1.5kBytes (<10pkts) UDP 80% < 600Bytes (75% < 3 pkts), ~10 * more TCP than UDP Top UDP = AFS (>55%), Real(~25%), SNMP(~1.4%) Can roughly characterize as power law with slope & intercept SNMP Real A/V AFS file server

20. 20 Flow lengths 60% of TCP flows less than 1 second Would expect TCP streams longer lived –But 60% of UDP flows over 10 seconds, maybe due to heavy use of AFS at SLAC –Another (CAIDA) study indicates UDP flows are shorter than TCP flows TCP outbound flows Active time in secs Measured by Netflow flows tied off at 30 mins

21. Applications 21

22. Web 22 By mid 2012: IE < 50%, Chrome overtakes Firefox

23. Usage P2p hit by RIAA law suits Moving to video, social networking –Video on demand double/2 years ’08-’13 iPhones (only peripherally a phone) –Mobile traffic doubles each year 23 Yahoo Google Facebook YouTube

24. Growth of Video P2P traffic, still the largest share of Internet traffic today, will decrease as a percentage of overall Internet traffic. 24 Internet video streaming and downloads are beginning to take a larger share of bandwidth, and will grow to nearly 60 percent of all consumer Internet traffic in 2014.

25. How Internet is used & when 25 Enterprise & tier 1 asert.arbornetworks.com/2009/08/the- internet-after-dark

26. 26 Web use characteristics Size of web objects varies from site to site, server to server and by time of day. –Typical medians in 2000 varied from 1500 to 4000 bytes Also varies by object type, e.g. medians for –movies few 100KB to MBs, postscript & audio few 100KB, text, html, applets and images few thousand KB Bytes Size of average web page tripled in 5 years 2003-2008 www.websiteoptimization.com/speed/tweak/av erage-web-page/www.websiteoptimization.com/speed/tweak/av erage-web-page/

27. Why increasing New users (easier for user, more coverage) New apps: You-Tube, climate modeling … New tools: manual(hand tuned)  Automatic generation –Web 2: Ajax, Javascript, CSS Broadband  more elaborate/attractive designs possible 27 desktop to web apps e.g. mail, calendars, photo albums, games...

28. 28 Log Plot of ESnet Monthly Accepted Traffic, January 1990 – December 2008 Impact on backbones: e.g. Current and Historical ESnet Traffic Patterns Terabytes / month Oct 1993 1 TBy/mo. Aug 1990 100 MBy/mo. Jul 1998 10 TBy/mo. 38 months 57 months 40 months Nov 2001 100 TBy/mo. Apr 2006 1 PBy/mo. 53 months  ESnet Traffic Increases by 10X Every 47 Months, on Average July 2010 10 PBy/mo.

29. Performance by Metric 29

30. 30 What does performance depend on? End-to end internet performance seen by applications depends on: –round trip times –packet loss –jitter –reachability –bottleneck bandwidth –implementation/configurations –application requirements Data transmitted in packets

31. msec. ITU G.114 300 ms RTT limit for voice 31 RTT from SLAC to the World RTT ~ distance/(0.6*c) + hops * router delay Router delay = queuing + clocking in & out + processing 2/3 countries of world Ok for voice, rest mainly in Africa What is the problem with > 300ms?

32. 32 RTT from California to world Longitude (degrees) 300ms RTT (ms.) Frequency RTT (ms) Source = Palo Alto CA, W. Coast E. Coast US W. Coast US Europe & S. America Europe 0.3*0.6c Brazil E. Coast Data from CAIDA Skitter project WHY these distributions?

33. 33 Longitude RTT(ms) Seen from Japan RTT from Japan to world

34. Jitter Variability of RTT, many ways to measure “Jitter” = IQR(ipdv); ipdv(i) =RTT(i) – RTT(i-1) Usually at edges, so ~distance independent Impacts smooth flows e.g. VoIP, video, real-time Haptics (surgery) < 1ms; H.323 <40ms with buffer 34 Internet Jitter seen from SLAC to World Sep’08 Can improve voice with de-jitter buffer, e.g. 70ms to smooth the flow But….

35. Losses On good lines usually congestion Wireless dB loss, net devices Usually last mile 35 Distance independent Big effect Realtime, games, Voice, typing echo 1% loss VoIP annoying

36. Derived Throughput 36 Behind Europe 5 Yrs: Russia, Latin America, Mid East 6 Yrs: SE Asia 9 Yrs: South Asia 12 Yrs: Cent. Asia 16 Yrs: Africa Central Asia, and Africa are in Danger of Falling Even Farther behind In 10 years at the current rate Africa will be 1000 times worse than Europe Derived throughput ~ 8 * 1460 /(RTT * sqrt(loss)) Mathis et. al 1993

37. Where is best Throughput? 37

38. Voice over IP Affected by: –Loss, RTT, Jitter, Quality measured by Mean Opinion Score (MOS) 38 –Can convert from RTT, loss & jitter to MOS –MOS values: 1=bad; 2=poor; 3=fair; 4=good; 5=excellent. –Typical reasonable range for Voice over IP (VoIP) is 3.5 to 4.2. –Russia and L.America improved dramatically in 2000-2002 as moved from GEOS to terrestrial. –US, Europe, E. Asia, Russia and the M East (all above MOS = 3.5) good. S.E. Asia marginal, S. Asia, C Asia need a lot of patience –Africa beginning to get there.

39. 39 Application requirements Based on ITU Y1541 & Stanford (Haptics) The VoIP loss of 10^-3 used to be 0.25 but that assumed random flat loss –actual loss is often bursty Tail drop in routers Sync loss in circuits, bridge spanning tree reconfiguration, route changes Applicatio n Real time VoIPWAN connectivity Web free services Stream video Haptics (remote surgery) 1 way delay150ms 1000msundefined400ms160ms ‘jitter”50ms1000msundefined17ms1ms Loss10 -3 undefined10 -5 0.1

40. What’s next Mobile devices 40G (transAtlantic, US) & 100Gb backbones On demand dynamic dedicated services (layers 1 & 2) –Reserve a path at some bandwidth for some time –Use QoS to deliver –HEP, Radio Astronomy, climate research IPv6 40

41. Questions & more study www.internetworld.stats.comwww.internetworld.stats.com www-iepm.slac.stanford.edu/pingerwww-iepm.slac.stanford.edu/pinger www.slac.stanford.edu/comp/net/wan- mon/tutorial.htmlwww.slac.stanford.edu/comp/net/wan- mon/tutorial.html www.slac.stanford.edu/xorg/icfa/icfa-net-paper- jan09/report-jan09.docwww.slac.stanford.edu/xorg/icfa/icfa-net-paper- jan09/report-jan09.doc http://www.cablemap.info/ 41

42. Compare with Development Indices Abv.NameOrganizationCountries Date of Data GDP Gross Domestic Product per capita CIA 2292001-2006 HDIHuman Development IndexUNDP1752004 DAIDigital Access IndexITU 180 1995-2003 NRINetwork Readiness Index World Economic Forum 1202007 TAI Technology Achievement Index UNDP721995-2000 DOIDigital IndexITU1802004-2005 OIIndexITU1391996-2003 CPICorruption Perception Index Transparency Organization 1802007 42 Choose most: up-to-date, countries, important factors HDI & DOI