GridNM Network Monitoring Architecture (and a bit about my phd) Yee-Ting Li, 1 st Year UCL, 17 th June 2002

17th June 2002GridNM - Yee-Ting Li 2 What the GRID is Distributed System Interconnected with networks Balancing processors, storage and network utilisation Like the SETI project on steriods Networking is important to make GRID work

17th June 2002GridNM - Yee-Ting Li 3 Networking Important! Only way two grid nodes can communicate with each other Need ways of determining how ‘efficiently’ they talk Focus on: The characterising how they talk The language they use to talk

17th June 2002GridNM - Yee-Ting Li 4 Part 1 Network Metrics and Measurement GridNM Case studies

17th June 2002GridNM - Yee-Ting Li 5 Network Metrics / Characteristics Metric: ‘several quantities related to the performance and reliability of the Internet that we'd like to know the value of. When such a quantity is carefully specified, we term the quantity a metric.’ Can be empirical or derived Singletons, Sample and Statistical Metrics

17th June 2002GridNM - Yee-Ting Li 6 Example Metrics Connectivity One-way delay Two-way delay Throughput / goodput Network path Loss Jitter

17th June 2002GridNM - Yee-Ting Li 7 Metrics Example Video Conferencing Needs predictable bit rate Doesn’t usually matter if bit rate changes too much Needs constant jitter Low one-way delay preferable FTP Needs reliable transport Throughput depends on urgency of data Jitter and delay don’t matter

17th June 2002GridNM - Yee-Ting Li 8 Measurement Methodology How to get the metrics Must be repeatable – need to define methodology carefully Direct measurement of a performance metric using injected test traffic. Projection of a metric from lower-level measurements. Estimation of a constituent metric from a set of aggregated measurements. Estimation of a given metric at one time from a set of related metrics at other times.

17th June 2002GridNM - Yee-Ting Li 9 Measurement Example ‘ping’ measures rtt – a direct measurement Sending a single ‘ping’ would give a singleton - empirical Sending 10 pings (a sample) out and getting the average is a statistical metric – derived Using a set of measurements over time, we can derive an Estimate of the rtt Projection would be if we had the owd for each router to the next – add all up together to get path owd.

17th June 2002GridNM - Yee-Ting Li 10 Network Monitoring Uses Monitoring is measuring over long periods of time Gives an indication of network performance over time – a baseline Allows comparison of different tools for analysis Allows analysis of how different protocols behave in different conditions – in real life Allows ‘tuning’ of existing protocols to make most out of network

17th June 2002GridNM - Yee-Ting Li 11 GridNM Architecture for monitoring the network Backend – collects data for presentation Logs metrics in ASCII log files on a single host Allows mesh measurements – all nodes performs measurements to al other nodes Uses standard UNIX infrastructure – ssh Should be easily adaptable to using Globus certifications once interactive processing is introduced in EDG.

17th June 2002GridNM - Yee-Ting Li 12 GridNM (cont…) Uses existing (and future tools) to collect metrics Modular - uses XML to describe available resources Hosts Tools Locks hosts if under measurement – prevents other tests affecting metrics Currently monitoring 6 sites around Europe using 5 tools

17th June 2002GridNM - Yee-Ting Li 13 GridNM ‘plot’

17th June 2002GridNM - Yee-Ting Li 14 Security As secure as SSH But requires automatic logon Denial of Service Attacks Certain Tools (eg iperf) require servers to be run. GridNM runs the server (unless otherwise told not to) before each tests on the remote host

17th June 2002GridNM - Yee-Ting Li 15 Tool Examples NameProtocolMetricsNotes IperfTCP/UDPGoodput Idea application level performance UDPMonUDPLoss, goodput Indication of network performance PingICMP/IPRTT, Jitter Response of network TracerouteICMP/IPPath, RTT PipecharUDP?Router utilisation Approximate BBCPTCPGoodput SCP Copy GridFTPTCP…Goodput Application

17th June 2002GridNM - Yee-Ting Li 16 UDP versus TCP

17th June 2002GridNM - Yee-Ting Li 17 Rtt – good network

17th June 2002GridNM - Yee-Ting Li 18 Rtt – periodicity

17th June 2002GridNM - Yee-Ting Li 19 Rtt – bad network

17th June 2002GridNM - Yee-Ting Li 20 Rtt – bad network, loss

17th June 2002GridNM - Yee-Ting Li 21 TCP / Iperf Throughput

17th June 2002GridNM - Yee-Ting Li 22 TCP Performance

17th June 2002GridNM - Yee-Ting Li 23 TCP Performance

17th June 2002GridNM - Yee-Ting Li 24 What does TCP do? Tap is independent of Tank size Tank filled by application Valve opening (data rate) determined by feedback from network Small tanks mean small data rate Large tanks mean larger data rate Even larger tank mean smaller data rate?!?! Socket buffer size TCP Protocol Network

17th June 2002GridNM - Yee-Ting Li 25 Investigation Possible explanation: Rate of tank filling < rate of water flow out i.e. application not fast enough to fill socket buffer past threshold BUT - needs further investigation Back to back lab tests with PCs and routers Comparison to other tcp based tools

17th June 2002GridNM - Yee-Ting Li 26 Part 2 Network Communication Languages Known as transport protocols - determines how applications put traffic into the network Sits on top of IP – common language of the internet

17th June 2002GridNM - Yee-Ting Li 27 Transport Level Protocols TCP (HTTP, FTP, GridFTP) used for file transfer Gives guarantee on delivery All data is copied precisely Performance can be poor Respects other internet users UDP (Real, H323) used for video conferencing Gives no guarantees on delivery Data may be incomplete Performance good Doesn’t respect other internet users

17th June 2002GridNM - Yee-Ting Li 28 UDP versus TCP performance at high speeds

17th June 2002GridNM - Yee-Ting Li 29 Measuring Performance of Transport Level Protocols Need to identify what we want to measure – the metrics. Dependant on the use of the transport protocol. Need to analyse application level usage For Grid: Movement of ‘transient’ data File Transfer and Replication process jobs or ‘sandboxes’ Movement of Real-Time Data Video Conferencing – Access Grid Real-Time applications

17th June 2002GridNM - Yee-Ting Li 30 Transport Protocols ‘NG’ NameTransportNotes UDP BlastUDP TsunamiUDP/TCP Uses TCP as ‘control’ channel High Speed TCP TCP For 10Gb/sec links PGM / CCModified UDP Multicast UDP – new transport protocol IBP Application ‘logistical networking’

17th June 2002GridNM - Yee-Ting Li 31 Tools to Measure Grid Traffic Eg TCP Can use web100 – allows analysis of TCP traffic via fundamental variables important to TCP/IP\ GridFTP allows logging of transfer information UDP (UDP Blast, Tsunami) Need either transport level recording (like web100) or application monitoring PGM / CC Need application to be built to use transport protocol General Solution Gather SNMP data from nodes along network.

17th June 2002GridNM - Yee-Ting Li 32 Future Directions (the phd bit) Provision Title in field of Providing Advanced Transport Protocols for Grid Applications Aim: Use GridNM infrastructure to analyse performance of different transport protocols Implement findings into Grid infrastructure, eg GridFTP, to improve grid processes (processing jobs, file transfer, file replication, Access Grid…)

17th June 2002GridNM - Yee-Ting Li 33 Conclusion Created a flexible infrastructure to monitor and analyse internet traffic Shown metrics for different scenarios Given performance overview of current transport protocols Identified future areas of research into Transport Protocols for the grid.