Technical Rehearsals Matter! Bob Riddle, Internet2 14 January 2004 Network Trouble-Shooting

Rehearse, rehearse, & rehearse Questions to answer before opening night Can I “stretch” the stage from here to there? – do you need a 2 lane road or an expressway? – will you need the highway during rush hour? –do you need to be able to “talk”, “listen”, or both? Do I live in the right kind of “neighborhood” ? – time to think about FDX, multicast, & switches – time to make some new friends & acquire new toys What sort of guarantees can I offer my boss?

Stretching the “Stage” Speed: do you need a freeway, city street, or a dirt road? (mbps) Latency: is it a “telephone” call or a “movie”? (rtt) Packet Loss: how much of the content do you care about? (100% … 90% … 50% … 10% …) Issues to consider when “stretching” the stage

Stretching the “Stage” -cont- Plan for a meaningful rehearsal What kind of highway do you need? H.323~=384 kbps Mpeg1~= 1.5 mpbs Mpeg2~=4 – 16 mbps DV~=30 mbps HDTV~=19.2 – 270 mbps What kind street do you really need? One way?(streaming, broadcast) Two way?(interactive voice & video) Many way?(more than 2 end points) Do the math! (and budget for the asphalt!)

Rush hour traffic or 3 a.m. traffic? Some lessons learned from the Bing Theater Good idea - Test early & test often – Sunday evening exercising the network – Monday morning “exercise” – Monday afternoon “exercise” Result – Hayward suffered from congestion Lessons learned: – Sun night traffic != to weeknight network traffic – Mon a.m. traffic != to Mon p.m. network traffic it’s important to rehearse the network during show time

You have to live in the Right Neighborhood Take a look at your local neighborhood 1.Run internal tests across your LAN 2.Make sure to test what you hope to use! 3.Find a friend “next door” to test with 4.Learn about ping, traceroute, iperf, Internet2 Detective, VRVS & what they can tell you 5.Start thinking about what “roads” you’ll need to travel to the next neighborhood(s)

My list or “normal” E2E tests Bandwidth tests iperf ( – allows for tcp, udp, multicast, bi-directional – allows testing for FDX needs – allows testing at various rates (KkMm) Multicast Tests iperf ( Vic, rat ( inSORS ( rcBridge (

Use the tools! Results of such tests – NWS example! bandwidth tests: – discovered DHCP server gateway problem at NWS – discovered 30 mbps with 0% packet loss was the best FDX testing: – discovered misconfiguration between PC & switch in Ann Arbor Internet2 office – PC diagnostics said everything was fine, switch display indicated HDX connection to the PC Human tests: – involve participants in “successful” network testing – need someone with the correct event perspective

Can I buy these tools at Wal-Mart? What did we learn from the Rialto event? 2 days to learn/fix/configure the network (help!) What if we could have shipped a no-brainer PC to “almost” endpoint that would all you to remotely exercise all these tools? Introducing the “cakebox” No operator, no monitor, keyboard, or mouse required Small, inexpensive, and easy to ship PC Just plug in a network cable & a power cable Provide user interface for non-network geeks 

Tools - cont -  Runs on Windows XP  Runs on Mac OSX  Runs E2E on detective machines  Its purpose is to tell you if there is any “hope”!

Other tools you can use      E2EP (end-to-end performance group)  Questions?