Network Monitoring: A Practical Approach Philip Smith/IT Services University of Windsor March 21, 2003

Agenda Campus Structure Benchmarking on Campus Tools on Campus Benchmarking off Campus Tools off Campus Questions and Answers

Campus Structure Core Router (Nortel Networks Passport 8610) 60+ Building Subnets (student + faculty) Computer Science and Engineering have their own networks Have two external connections Internet (Telus) at 15Mb/s + over subscription CAnet*4 (AT&T) at 155Mb/s Both connections use ATM

Campus Structure (Block Diagram)

Campus Structure (Graphical)

Benchmarking on Campus: Benchmarks FTP (TCP/IP download performance) TTCP (TCPIP upload performance)  Need to consider both upload and download because you could have a duplex problem. PERFORM3 (Novell performance)

Benchmarking on Campus: FTP FTP is a disk to disk transfer protocol  theoretically this could & does affect performance.  We drop the first FTP test to each server because the file is not cached. FTP benchmark is run against 3 servers at or near the network core. Key servers are:  Admin1 (administrative server/AIX-IBM UNIX)  Pdomain (campus FTP server/IRIX-SGI UNIX)  Zeus (Lotus Notes server/AIX)

Benchmarking on Campus: TTCP TTCP is a memory to memory transfer protocol  disk is NOT involved. TTCP benchmark is run against 4 servers at or near the network core. Key servers are:  Admin1 (administrative server/AIX-IBM UNIX)  Cronus (Lotus Notes server/NT)  Pdomain (campus FTP server/IRIX-SGI UNIX)  Zeus (Lotus Notes server/AIX)

Benchmarking on Campus: PERFORM3 PERFORM3 is Novell’s benchmark for networks that are 10Mb/s or more. While Novell is not used very frequently in Computer Science it is used a great deal elsewhere on campus. At one point (circa 2000) Novell traffic was 2/3 of our Network. Modified PERFORM3 to run faster; limit is to twelve operations at 16K intervals instead of at each 4K interval. Modified test takes 1-2 minutes compared to 5 minutes. Run PERFORM3 benchmark against all available Novell servers.

Benchmarks on Campus: Methodology Using Work Study labour, annually run all three benchmarks from each subnet in each building using a common laptop. Run 4 TTCP tests against each of the 4 TTCP server (4*4=16) Run 3 FTP tests against each of the 3 FTP servers (3*3=9); remember first test is discarded Run 2 PERFORM3 tests against each Novell server (2*~9=18)

Benchmarks on Campus: Summary Results of annual building tests available on line.  URL:  Click on Benchmark Database from left hand menu. Also contains benchmarks from some faculty and staff that have complained about their performance.

Tools on Campus Protocol Analyzer WhatsUp MRTG MRTG-UFFE NMS

Tools on Campus: Protocol Analyzer Device that lets you see packets on the wire Our tool is a Network Associates’ Sniffer Primarily a troubleshooting tool However, by capturing the data on a connection (e.g. uplink) over time you can collect key network statistics Flaw: It only does ONE connection at a time Protocol Analyzer measures packets

Tools on Campus: WhatsUp Monitors network devices (e.g. switches & routers) servers & server applications uses ICMP (ping) and TCP/IP ports If device responds server is deemed to be up Flaw: Just because the web server port opens on port 80 this does not necessarily mean the web server is working properly; it just means that the web server is up WhatsUp measures availability Uses drill down method (example to follow)

Tools on Campus: WhatsUp

Drilling down into Memorial Hall, there is something wrong with the UPS (top diagram) It looks like the UPS management is down (bottom diagram)

Tools on Campus: MRTG MRTG = Multi Router Traffic Grapher Monitors bits in and out of a network device (eg. Switch port, router port, NIC card) Using SNMP it queries the switch for port activity once every five minutes Keeps daily, weekly monthly and yearly statistics on that port Flaw 1: If there is a lot of usage then the device(s) attached to the port are running well. If usage is low then ???? Flaw 2: It monitors amount of bits not the number of packets. If you had a Denial of Service attack with a large number of small packets MRTG would not indicate a problem MRTG measures bandwidth Like WhatsUp, MRTG uses drill down method

MRTG example: Fully drilled down view of Passport to CS SSR Router

Tools on Campus: MRTG-UFFE MRTG-UFFE = MRTG’s User Friendly Front End Add on to MRTG Homegrown utility that documents the important (special, unusual, busy) connections on campus Hyperlinks to MRTG MRTG-UFFE measures connections

Tools on Campus: NMS NMS = Network Management System MRTG only measures bits in (received) and out (transmitted) Only 2 of 34 parameters on the switch port Future Project

Benchmarks off Campus Mostly a new area of focus Have been monitoring using Protocol Analyzer, WhatsUp & MRTG Size of Internet Pipe growing yearly by about 2Mb. Recently we have also been monitoring using BroadBandReports.com

Benchmarks off Campus: WhatsUp

Benchmarks off Campus: MRTG

Benchmarks off Campus: BroadBandReports.Com

Tools Off Campus Protocol Analyzer WhatsUp MRTG BroadBandReports.com Internet Monitors

Tools Off Campus: Internet Monitors Internet Health Report   Measures Latency (TCP Open) Between Major U.S. carriers. Internet Traffic Report   Measures Latency (ICMP Echo) & Packet loss between selected routers world wide. Internet Average   Measures Latency, Packet Loss, and Reachability between thousands of servers and routers around the world. (Most Comprehensive)

Question & Answers Thanks for your attendance Philip Smith’s Network Performance site: