1 ESnet/HENP Active Internet End-to-end Performance & ESnet/University performance Les Cottrell – SLAC Presented at the ESSC meeting Albuquerque, August 1999 Partially funded by DOE/MICS Field Work Proposal on Internet End-to-end Performance Monitoring (IEPM)

2 Overview New stuff –New validation results –New monitoring features –Current deployment ESnet to US university performance Improved routing Comparisons of ESnet & other networks

3 New stuff since last meeting Presentations to: –NGI testbed workshop, ICFA/SCIC (2), NLANR Workshops (2), ESCC, Cisco workshop on peering in Amsterdam, Stanford Comp. Science seminar Increased collaboration with other projects –Installed Surveyor, AMP & RIPE engines at SLAC Validated ping/PingER measurements –Correlation with thruput –High statistics measurements –Correlations with Surveyor & AMP New IEPM web site: – Web pages focused on collaborations –Added more groups (HENP, Fusion, DOE, NSPs, measurers…) Improved user interface

4 Validating PingER Sanity checks: –host pings itself, host pings host at same site –high statistics between a few sites & inside site: see /comp/net/wan-mon/ping-hi-stat.html /comp/net/wan-mon/ping-hi-stat.html look at subtle behaviors, e.g. RTT distribution tails –check “wire time” (sniffer) vs. ping reported times, at client & server, calibrtae & understand sources of errors see Correlated with Surveyor one-way measures

5 TCP bandwidth < (MSS/RTT)*(1/sqrt(loss)) Residual = GET - 2 * min (ping RTT) Relates to Web performance (small files dominated by RTT)

6 Web Interface Choose metric loss, RTT, variability, reachability Choice of time ticks hour, daily, monthly ; can also select day, month etc. Choice of group Export data to Excel Drill down to plots Other stuff Sort on columns Help /xorg/iepm/pinger/table.html

7 Effect of STAR-TAP on KEK.jp SLAC ITU G msec RTT limit for voice

8 Deployment in ESnet/HENP Over 50% of HENP collaborator sites are explicitly monitored as remote sites by PingER –Atlas (37%), BaBar (68%), Belle (23%), CDF (73%), CMS (31%), D0 (60%), LEP (44%), Zeus (35%), PPDG (100%), RHIC(64%) –Created focused PingER pages for BaBar, CDF, CIT- Fusion, D0, PPDG, and RHICBaBarCDFCIT- FusionD0PPDGRHIC Remainder represented by beacon sites –Selected to represent countries/R&E nets –About 50 beacon sites in 29 countries Just been updated

9 DOE funded sites’ performance Originally worried: if > 5%, then if > 2.5%, now if > 1%

10 Probability of loss

11 Details

12 Possible improved routing: 1/2 Looked at sites of interest to BaBar –which are on Internet 2 (vBNS or Abilene) and –have big differences in RTT seen from Stanford campus vs. from SLAC Stanford on Internet 2, SLAC on ESnet ESnet peers with Internet 2 at the Chicago NAP –BW TCP < MSS/(RTT*sqrt(loss))  1/RTT Need to minimize packet loss Need shorter routes => more direct peering Need larger MSS (beyond Ethernet standard)

13 Possible improved routing: 2/2 U Colorado & Colorado State RTT improves by factor 2 (60msec=SLAC=2*Stanford) –All are flat, and low loss (~ 1%), ESnet less variable RTT Stanford U - ColoState SLAC - ColoState (via Chicago) Loss RTT msec.

14 Possible improved routing: 3/3 U Texas at Dallas –SLAC – UT Dallas congested (RTT varies ms), 5-25% loss, similar to SLAC - Rice –Stanford - Rice (on Abilene) 50 msec and flat, low loss –Infer if UTD on Abilene & ESnet peered on W. Coast then similar to Stanford – Rice, i.e. low loss msec improvement However, U Tenn., Knoxville –SLAC – UTK 70msec & flat, –Stanford – UTK 100msec & flat (via Southern Georgia Tech)

15 Comparisons of Networks

16 Top dozen site utilizations ANL has direct peering thru 1 arm router, only count multicast

17 Work in Progress 1/2 Redouble efforts to get HEPNRC more involved Get new Beacon sites deployed at all monitor sites Random scheduling of pings (in beta at 2 sites) Recording more information (in production at XIWT) More flexibility in choice of packet sizes, frequencies (tailor to bandwidth between pair) Look for ICMP rate limiting signatures Gather historical traceroute information for PingER Calibrate ping jitter against VoIP jitter

18 Work in Progress 2/2 Passive RTT using SYN/ACK, FIN & PSH Calibrate using ping to measure QoS effects –setting up QoS testbed between SLAC & LBNL Measure frequency of outage lengths Continue work with XIWT, Surveyor, AMP & RIPE Other possibilities: –Thinking about extending framework to other “apps”, e.g. following IPMP work, TCP/UDP echo, http (CERN interested, XIWT also interested, there are also commercial tools, but expensive) –Generate alerts (HEPNRC)

19 Conclusions Ping/PingER agrees well with more heavyweight mechanisms (Surveyor, AMP, RIPE …) Ping/PingER validated versus wire time measurements Ping/PingER relates well to applications Performance is getting better ESnet is well provisioned Within R&E networks things are good Transoceanic, needs special care Peering is critical

20 More information IEPM home site: –