Muhammad Murtaza Yousaf, Michael Welzl A Reliable Network Measurement and Prediction Architecture for Grid Scheduling Muhammad Murtaza Yousaf, Michael Welzl NSG Team, Distributed and Parallel Systems Group, Institute of Informatics, University of Innsbruck, Austria.

Motivation High reliability = main goal for these predictions! Selection of optimum site is crucial for Grid Scheduler. “A scheduler always needs to know how long file transfers will take.” High reliability = main goal for these predictions! Traffic changes a lot; routing doesn’t thus, measure / predict path (not traffic) properties! => represent minimum transfer delay (ideal case) Grid Scheduler Grid Site – 1 Site – 2 Grid Site Source

Design Consideration We are looking for a measurement and prediction system which provides: Reliable estimates, with Small overhead, and it must be Scalable, with High availability.

Architecture Description Five basic components: Client Monitor Active Prober Information Base Information Manager Monitor Information Manager Client Active Prober Base High level architecture of the system

Architecture Description (Cont…) Information Manager Network Interface Sniffer Extractor Filter Internal architecture of the Monitor Monitor has sub components: Sniffer Monitors and captures traffic from a network interface. Filter Packets captured by the Sniffer may belong to many flows separates packets which belong to a particular stream. Information Extractor Receives a filtered stream generates reliable estimates by using our measurement methodology.

Architecture Description (Cont…) The Active Prober injects some traffic for the measurement purpose, generates estimates as required. The Information Base is to store measurements. The Information Manager stores and retrieves information from Information Base with a timestamp. Interacts with the client. Invokes Active Prober, if Information is too old Network path is lightly loaded Information is missing Client behavior requires more reliable information.

Detection of Shared Paths Detection of shared paths and shared bottlenecks will enhance the usefulness of our predictions A B C α β E D

Bandwidth Estimation Technique: Packet Pair Two packets sent back-to-back Looks Active, but possible in Passive Interspacing after bottleneck Inverse of bottleneck bandwidth Preserved at non-bottleneck routers. Pi-1 Direction of flow Pi ∆ = s b Packet Pair (Pi-1 and Pi), before and after bottleneck

Conclusion and Future Work A measurement system to estimate network path characteristics in a grid. Passive (mostly). Packet Pair based Measurement Technique (relationship of ACKs and corresponding data packets). RTT Estimation Currently we are working on a preliminary version of the Information Extractor component of the Monitor for a reliable estimate of the path capacity (real-life and simulation). After an accurate and reliable estimate of bottleneck bandwidth, our target is the detection of shared bottlenecks because this knowledge can enhance the usefulness of our predictions. For experimental results we plan to use our system in Planet Lab, and our ultimate target is to deploy it on Austrian Grid.