1. Is your Service Available? or Common Network Metrics Nevil Brownlee, CAIDA NANOG 19, Albuquerque, June 2000

2. NANOG 19, June 2000Nevil Brownlee 1 / 15 Overview CAIDA Metrics Working Group - Co-chairs from the networking industry Sue Moon (SprintLabs) Brett Watson (MFN/Abovenet) Measurement FAQ Service Definitions Common Metrics Availability

3. NANOG 19, June 2000Nevil Brownlee 2 / 15 Metrics WG Goals Education - Publish ‘Measurement FAQ’ - Publish ‘Metrics and Measurement Survey’ Service Metrics - Define metrics for new / emerging services - Recommend a ‘Service Measurement Toolkit,’ encourage implementors - Publish revised ‘Measurement Requirements for Hardware/Software Vendors’ User / customer participation needed !

4. NANOG 19, June 2000Nevil Brownlee 3 / 15 FAQ Contents (1) Target audience - Corporate users, smaller providers, hosting service users Generally Accepted Terms - Networking, types of service, faults and failures Measurement Topics - Active vs passive, one point vs many, sampling, statistics

5. NANOG 19, June 2000Nevil Brownlee 4 / 15 FAQ Contents (2) The Most Common metrics - Latency, packet loss, throughput, link utilisation, availability Common Measurement Tools - ping, traceroute, SNMP, flow measurement application monitoring, visualization Comparing Service Offerings - Provider ‘net status’ pages - Internet ‘weather maps’ - Rating services

6. NANOG 19, June 2000Nevil Brownlee 5 / 15 Defining Network Service Service definitions (SLAs) on the Web - Many providers publish these, e.g. AT&T, PSInet, UUNET and MCI WorldCom - They describe service offered to customers We’re only interested in describing the service, not in contractual aspects Metrics used in the service descriptions are often poorly defined

7. NANOG 19, June 2000Nevil Brownlee 6 / 15 Service Example: UUNET (1) - Network Quality Average monthly latency of no more than 85ms roundtrip within UUNET's network in North America and of no more than 120ms between New York and UUNET's international gateway hub in London Comments - Restricted to provider’s own network - Latency not defined - Nothing said about packet loss %

8. NANOG 19, June 2000Nevil Brownlee 7 / 15 Service Example: UUNET (2) - Service Quality 100 percent availability that covers the UUNET backbone and the UUNET-ordered customer access circuit. Scheduled maintenance.. will take place.. Tuesdays and Thursdays with at least 48 hours advance notice Comments - 1 minute per week = 0.01% - Does ‘available’ time include maintenance? - ‘100% availability’ - but what is availability?

9. NANOG 19, June 2000Nevil Brownlee 8 / 15 ISP Network Report Pages Many providers publish these - Sean Donelan email thread lists 13, e.g. Abovenet, AT&T, C&W, UUNET, ELI, Jet Net.. Amount of detail varies - Outage information, NOC contacts - Latency, packet loss matrices (or averages) Such ‘overall’ reports don’t say much about performance as seen from your network - You need some measurements at your site

10. NANOG 19, June 2000Nevil Brownlee 9 / 15 Common Metrics Throughput, link utilisation - Commonly measured with SNMP, RRDtool Latency, packet loss - Latency is round-trip, transit + server delay - Commonly measured with ping Availability - WG definition based on IETF IP Performance Measurement (IPPM) connectivity metrics - Need to specify what’s available, how to measure it, and what values are acceptable

11. NANOG 19, June 2000Nevil Brownlee 10 / 15 Levels of Availability Service availability: being able to send packets for a specified service - say WWW request packets - to a particular Internet host, and to receive answering packets Host availability: being able to send packets, say ping packets, to a particular Internet host, and to receive answering packets Network availability: being able to send packets from your network to the Internet, and to receive answering packets

12. NANOG 19, June 2000Nevil Brownlee 11 / 15 Measuring Availability Web service availability test: download specified pages from target web server using web browser, measure latency, packet loss and throughput Host availability test: ping the target host, having made sure that it will respond to ICMP packets Network availability test: traceroute to the target host, so as to determine whether there is connectivity to the target network

13. NANOG 19, June 2000Nevil Brownlee 12 / 15 Defining Availability Requires negotiation between provider and customer - What services are important? - What performance level is acceptable? Be realistic! - Providers only control their own networks - Some packet loss is inevitable Measurements are important - Both sides should work together on this - Make some ‘baseline’ measurements

14. NANOG 19, June 2000Nevil Brownlee 13 / 15 Conclusion The CAIDA Metrics WG has begun by producing its ‘Measurement FAQ’, which provides background material on many measurement topics The FAQ attempts a new definition of Service Availability - the Metrics WG needs feedback on this ! The WG seeks input for its other goals, especially for defining new metrics

15. NANOG 19, June 2000Nevil Brownlee 14 / 15 More Information (CAIDA) Metrics WG (FAQ, mailing list) www.caida.org/outreach/metricswg FAQ Contributors Cindy Bickerstaff, Carter Bullard, Les Cottrell, Sean Donelan, Dave O’Leary, Brett Watson,.. CAIDA tools taxonomy www.caida.org/tools

16. NANOG 19, June 2000Nevil Brownlee 15 / 15 More Information (Net Status) Provider web pages are listed in FAQ [ISP_SERV]service definitions [ISP_REPT]network performance reports NOC pages (search ‘xxx network status’) www.pictureview.com/support/PVTS2.html www.psinet.com/netstatus/ www.sprintlink.net/netstat.html Network performance pages stats.sjc.above.net/traffic/ ipnetwork.bgtmo.ip.att.net traffic.cwusa.com/index.html