National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Web100 Roll Out I2 Members Meeting April 9-11, 2003

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Agenda Web100 History Web100 Diagnostic Tools Web100 Applications and User Experience

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications The Web100 Project When there is a problem, just ask TCP TCP has an ideal vantage point –Can identify the bottleneck subsystem –Already measures the network – Can measure the application – Can adjust itself

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Web100 Components Kernel Instrument Set (KIS) Diagnostic Tools Autotuning Widely distributed Open Source TCP ESTATS MIB Promote vendor adoption

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Diagnostic Tools User mode tools to prove core functionality Template for future tool developers KIS validation Portable library to hide OS details

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications TCP Autotuning Supercedes today’s controls for experts –Eliminate primary cause of the wizard gap. –New TCP buffer management model. –TCP just gets it right without controls. –Paper in draft for future publication.

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Software Distribution Open source –Linux kernel patches –User mode Tools –Contributed Software Active user/developer support by the Web100 team User group meetings

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications IETF Standards TCP extended statistics MIB Adds detailed per connection statistics Standard TCP instruments and controls Most recent draft submitted early March –TSV WG work item –Key vendors are already participating

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Kernel Instrument Set (KIS) TCP Instruments prototyped in Linux 2.4. Simple API via /proc. Instrument groups: –Options and State –IP Traffic and Throughput –Triage –Congestion Events –Network Path properties –TCP API Usage –TCP Parameters –WAD Parameters

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Web100 Userland 1.3 Library: allows application developers to easily integrate Web100 support Command-line tools: provide simple, scriptable access to Web100 variables GUI: a user-friendly front-end to the Web100 variables Includes a comprehensive set of man pages, a developer’s guide, and a user’s guide

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Userland Library Provides useful abstraction and common inter-application functionality Base abstraction is the agent (from SNMP terminology). Other abstractions include the group, variable, and connection. Includes taking snapshots and generating log files

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Userland Library Cont’d Will handle different operating systems’ methods for exposing Web100 variables (Linux’s /proc vs. BSD’s sysctl(3)) Userland 2.0’s library’s programming interface will change, but old applications will not break as 1.3 and 2.0 may coexist on the same computer

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Userland Library Users Web100 command-line and GUI tools Web100-based Iperf: didc.lbl.gov/~jason/net100/ didc.lbl.gov/~jason/net100/ Web100 Python interface: ed/web100py ed/web100py Others?

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Userland Command-line Tools readall: reads the current value of all Web100 variables from all connections readvar: read the current value of a single Web100 variable from one connection deltavar: continuously monitor a single Web100 variable in a single connection, displaying how it changes every second writevar: write a value to a single Web100 variable in one connection (for tunable variables only)

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Userland GUI Access, display, and control (where applicable) values of Web100 variables List connections and related process info Triage pie chart shows the source of congestion: Sender, Receiver, Path Uses GTK2/GTK1 (common to all standard Linux distributions)

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Userland GUI Cont’d Display: –continually list values of all variables –graph those of interest –triage analysis per connection Control: –toggle auto-tuning per connection –set tunable variables: LimCwnd, LimRwin

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Userland Wrap-up Feel free to with any questions or problems with the Feature and improvement suggestions welcome!

National Center for Atmospheric Research Pittsburgh Supercomputing Center National Center for Supercomputing Applications Applied Web100 – One Example Larry Dunn teaches a basic network class at U.Minnesota using Web100. Students study local- and wide-area TCP flows by inspecting variables exposed by Web100. Wide-area experiments done to a Web100 enabled machine at NCSA (UIUC). Class taught Spring 2002, and is being repeated now.