1. Videoconferencing: Network Tools & Troubleshooting Dave Devereaux-Weber University of Wisconsin-Madison Internet2 Commons Site Coordinator Training March 31, 2005 Atlanta, Georgia

2. 2 Delay and Jitter  Delay is the elapsed time from send to receive  Jitter is variation in delay for each packet

3. 3 Lost Packets  When a packet arrives too late to be displayed, the codec assumes it is lost. If network issues delay a packet, the codec calls it lost, but the router does not.

4. 4 TCP and UDP  Transmission Control Protocol This protocol guarantees receipt  Universal Datagram Protocol This protocol does NOT guarantee receipt  Video is so time sensitive that retransmission does not help, so UDP is used  Email, WWW and file transfer use TCP  TCP can mask a poor connection, video shows all the network faults.

5. 5 Duplex Mismatch  Using thicknet Ethernet, the same wire was used for sending and receiving; if more than 1 talker at same time, packet errors.  Using twisted pair or fiber, different wire for transmit and receive, but ports need to be told to wait for quiet (half duplex) or talk anytime (full duplex).  Mismatch when 1 at half, 1 at full.

6. 6 Auto Speed, Duplex  Ports can be told to automatically set speed & duplex.  The standard says: if you are in auto and the other end is not auto, drop to default of 10/half.  The danger is that if 1 end is set for 100/full and the other is auto, the auto end, detecting non-auto on the other end must drop to 10/half

7. 7 Duplex Mismatch – Detection  Show port statistics on the Ethernet switch. When mismatched, the full-duplex end will report a high level of CRC or alignment errors; the half-duplex end will report a high number of late collisions.  PortAlignFCSXmitRcvUnderSize  ErrErrErrErr  2/11-0030770   Port Single-Multi-Late-Excess-CarriSenRuntsGiants  CollCollCollColl  2/11 3233025880024890

8. 8 Firewall  Consult your vendor website for firewall recommendations

9. 9 Ping  Test for availability, loss, and roundtrip time  ICMP Echo Request Plus optional dummy payload – only in the direction of the ping, i.e., source to destination

10. 10 Sample Ping from Windows  C:\WINDOWS>ping 10.1.1.1  Pinging 10.1.1.1 with 32 bytes of data:  Reply from 10.1.1.1: bytes=32 time=88ms TTL=112  Reply from 10.1.1.1: bytes=32 time=72ms TTL=112  Reply from 10.1.1.1: bytes=32 time=69ms TTL=112  Ping statistics for 10.1.1.1:  Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),  Approximate round trip times in milli-seconds:  Minimum = 69ms, Maximum = 88ms, Average = 74ms  C:\WINDOWS>

11. 11 Sample Ping from Windows  C:\WINDOWS>ping -l 40000 10.1.1.1  Pinging 10.1.1.1 with 40000 bytes of data:  Reply from 10.1.1.1: bytes=40000 time=2412ms TTL=112  Reply from 10.1.1.1: bytes=40000 time=2721ms TTL=112  Reply from 10.1.1.1: bytes=40000 time=2761ms TTL=112  Reply from 10.1.1.1: bytes=40000 time=2714ms TTL=112  Ping statistics for 10.1.1.1:  Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),  Approximate round trip times in milli-seconds:  Minimum = 2412ms, Maximum = 2761ms, Average = 2652ms  C:\WINDOWS>

12. 12 Ping Big and Fast on Linux  ping -c2000 -i.03 -s1470 -q [destination] Count of 2000 packets Interval of.03 seconds between packet starts Packet size of 1470 bytes (2000)(.03) = 60 second long test (1/(.03 sec/packet))(1470 bytes/packet)(8 bits/byte) = 392 Kbps

13. 13 Ping Big and Fast on Linux  $ ping -c2000 -i.03 -s1470 -q [hostname]  PING [hostname] ([hostaddr]) from [hostaddr2] : 1470(1498) bytes of data.  --- [hostname] ping statistics ---  2000 packets transmitted, 2000 packets received, 0% packet loss  round-trip min/avg/max = 4.8/5.1/13.2 ms

14. 14 Traceroute  Used to discover the layer-3 network path (routers) between the two endpoints  Doesn’t identify layer-2 devices (switches)  Take baselines – know what your path should be in advance of trouble

15. 15 Traceroute  Microsoft Windows tracert uses ICMP  Unix traceroute uses UDP  If Microsoft Windows tracert appears to show continuous timeouts, the router may be filtering ICMP traffic – try a Unix/Linux traceroute.

16. 16 Traceroute; Sample Output  [dodpears@huck dodpears]$ traceroute www.internet2.edu  traceroute to www.internet2.edu (209.211.239.208), 30 hops max, 38 byte packets  1 wcc-sub5-hp1 (129.79.5.253) 11.726 ms 0.627 ms 0.571 ms  2 iub-gw (129.79.8.10) 3.133 ms 0.717 ms 0.651 ms  3 156.56.249.22 (156.56.249.22) 2.544 ms 3.138 ms 2.538 ms  4 abilene-iupui.abilene.ucaid.edu (198.32.11.13) 5.245 ms 3.402 ms 3.493 ms  5 clev-ipls.abilene.ucaid.edu (198.32.8.26) 9.381 ms 9.586 ms 9.244 ms  6 nycm-clev.abilene.ucaid.edu (198.32.8.30) 23.198 ms 21.963 ms 21.775 ms  7 border-abilene-oc3.advanced.org (209.211.237.97) 23.448 ms 23.268 ms 23.052 ms  8 www.internet2.edu (209.211.239.208) 23.559 ms 23.478 ms 23.234 ms

17. 17 Traceroute  8 www.internet2.edu (209.211.239.208) 23.559 ms 23.478 ms 23.234 ms  Hop  Router/host name  Router/host address  Round-trip times of each of three probes

18. 18 Traceroute – Example Uses  Identify the path, and then perform pings along the path segments to isolate troublesome segments.  Insure that Internet2 is being used for a connection, rather than commercial Internet.

19. 19 Looking Glass  These network tools work from your vantage point out; what if you want to measure from some other vantage point back in? You want a looking glass:  See traceroute.org

20. 20 Ping Plotter  Shareware tool; $15 http://www.pingplotter.com  Performs a visual traceroute and ping tests along the entire path  Permits identification of bottlenecks along a path

21. 21 Ping Plotter

22. 22 MRTG.ORG  Multi Router Traffic Grapher  Open Source  Collect and graph scalar, time-based data, e.g., router and link performance data.

23. 23 MRTG Example MRTG graph showing traffic from SLAC to a physics lab at University of Wisconsin

24. 24 Iperf  Client/server application that Measures maximum TCP bandwidth Facilitates tuning of TCP and UDP parameters Reports bandwidth, jitter, and packet loss  http://dast.nlanr.net/Projects/Iperf/

25. 25 Iperf Example on Intercampus DS3  At server, invoke:  iperf -fk -i30 -u -s (f)ormat reports in kbps (i)nterval for reporting = 30 seconds (u)dp (s)erver mode

26. 26 Gnuplotping  Pings multiple hosts in parallel with graphical display (gnuplot) of the delay distribution.  Runs on Unix/X-Windows  http://www.gnuplot.info/

27. 27 Gnuplotping

28. 28 Network Traffic Analyzer  Reveals the traffic on a LAN  Protocol analysis  Reports such as utilization, protocols, conversations, nodes, etc.

29. 29 Network Traffic Analyzer: Software  Ethereal http://ethereal.com/  EtherPeek http://wildpackets.com/

30. 30 Network Traffic Analyzer: Commercial Hardware  Fluke Networks NetTool http://www.flukenetworks.com/us/LAN/Handheld+Testers/NetTool/Overview.htm This is a technician-oriented tool to perform diagnostics and testing. http://www.flukenetworks.com/us/LAN/Handheld+Testers/NetTool/Overview.htm  Fluke has an excellent white paper on auto-negotiation problems. Highly recommended.  Network Associates Sniffer http://nai.com/ This is an engineer-oriented tool to analyze packets and traffic- often needs interpretation. http://nai.com/

31. 31 Internet2 Detective  I2 Detective is a small application.  Very easy to use  Detects Internet2 connection.  Measures connection bandwidth (using Iperf).  Detects multicast connection. http://detective.internet2.edu/

32. 32 ANL Web100 based Network Diagnostic Tester (NDT)  Highly recommended  http://miranda.ctd.anl.gov:7123/ http://miranda.ctd.anl.gov:7123/ Located at Argonne National Laboratory; 100 Mbps FastEthernet network connection This java applet was developed to test the reliability and operational status of your desktop computer and network connection. It does this by sending data between your computer and this remote NDT server. These tests will determine: The slowest link in the end-to-end path (Dial-up modem to 10 Gbps Ethernet/OC-192) The Ethernet duplex setting (full or half); If congestion is limiting end-to-end throughput. It can also identify 2 serious error conditions: Duplex Mismatch Excessive packet loss due to faulty cables.

33. 33 H.323 Beacon  Open Source Software  Used to measure, monitor and qualify the performance of an H.323 Videoconference session.  Provides H.323-protocol specific evidence and other information necessary to troubleshoot H.323 application performance problems in the network and at the host (end-to-end) http://www.itecohio.org/beacon/

34. 34 Other Tools  CAIDA Internet Measurement Tool Taxonomy http://www.caida.org/tools/taxonomy/measurement/ http://www.caida.org/tools/taxonomy/measurement/  A comprehensive list of internet measurement tools

35. 35 ViDe Videoconferencing Cookbook  http://videnet.gatech.edu/cookbook/ http://videnet.gatech.edu/cookbook/  All about videoconferencing

36. 36 Network Performance Advisor  From National Laboratory for Applied Network Research (NLANR)  http://dast.nlanr.net/projects/advisor/ http://dast.nlanr.net/projects/advisor/  The NLANR Network Performance Advisor is an easily customizable framework that provides a single application to integrate the measurement, analysis, and display of network performance statistics.  Free, open-source software

37. 37 Chariot  Now IxChariot  http://www.ixiacom.com/products/performance_applications/pa_display.php?skey=pa_ixchari ot&section=perftest http://www.ixiacom.com/products/performance_applications/pa_display.php?skey=pa_ixchari ot&section=perftest  Computer-based, distributed measurement with central control  Commercial software