SLAC monitoring Web Services Paola Grosso SLAC Networking Services ESCC Jan 28-29 2004 ESCC Jan28-29 IEPM Web Services

Outline of presentation What are Web Services? SOAP XMLRPC NM-WG (Network Measurement Working Group) schema SLAC Web Services: IEPM implementation Present and future collaborations and project evolution ESCC Jan28-29 IEPM Web Services

Web Services “A web service is any service that is available over the Internet, uses a standardized XML messaging system, and is not tied to any one operating system or programming language” ESCC Jan28-29 IEPM Web Services

XML XML = eXtensible Markup Language: It’s designed to structure, store and send information; Tags and structures are used-definable; Conversion of data to XML allows exchange of data in previously incompatible formats; XML data is in flat files, so that it’s software and hardware independent. ESCC Jan28-29 IEPM Web Services

XML-RPC, SOAP and WSDL XML-RPC: Simple Object Access Protocol (SOAP): Implements remote procedure calls over HTTP; It’s simple to use (i.e does not have advanced features as SOAP). Simple Object Access Protocol (SOAP): It’s a communication protocol based on XML; It’s platform and language independent. Web Services Description Language (WSDL): It’s an XML document; Used to describe and locate Web Services. ESCC Jan28-29 IEPM Web Services

GGF Network Measurements Working Group The group worked on: defining standards for describing network measurements taken in Grid environments, so to allow exchange of measurements with a common “dictionary”. http://www-didc.lbl.gov/NMWG/docs/draft-ggf-nmwg- hierarchy-01.pdf Grid services need to know: 1 what and where are the best source/s to copy data from; 2 if parallel streams need to be used, and how many; 3 what are the ideal TCP window and buffer sizes. Enumarate the stated needs of the Grid in respect to the NM-WG ESCC Jan28-29 IEPM Web Services

NM-WG schemas A “request” schema, defining how the measurements can be requested to the server: http://www.hep.ucl.ac.uk/~pdm/nmwg/2004-01-08-Dante-NM-WG-RequestSchema2.xsd A “response” schema, defining how the network measurements have to handed back to the requester/client: http://www-didc.lbl.gov/NMWG/schemas/NM-WG_Schemas_for_OGSI.html ESCC Jan28-29 IEPM Web Services

NetworkMeasurementRequest A request NetworkMeasurementRequest Subject Time Characteristics Methodology The network entity for which a measurement is requested, i.e a router, a node, a path The time period for which we are retrieving a measurement. The name of the characteristic requested The methodology requested http://www.hep.ucl.ac.uk/~pdm/nmwg/2004-01-08-Dante-NM-WG-RequestSchema2-examples/Dante-NM-WG-RequestSchema2-demo-NMWG1.xml ESCC Jan28-29 IEPM Web Services

A response NetworkMeasurement Characteristics Subject Methodology Results The name of the network property that has been measured. The host and The networks That are being tested. What test was run, and how it was run, in order to collect the measurement. The actual value of the measurement ESCC Jan28-29 IEPM Web Services

A response result A response can contain results of type: BandwidthAvailableType BadwidthAchievableType BandwidthType PathReorderingType PathDelayJitterType PathLossType DelayType Information on the network bandwidth Information on the path characteristics http://www-didc.lbl.gov/NMWG/schemas/response_Jan2004/sample.xml ESCC Jan28-29 IEPM Web Services

SLAC WSDL file The description of the available service is in the SLAC IEPM WSDL file: http://www-iepm.slac.stanford.edu/tools/soap/wsdl/IEPM_profile.wsdl The document provides: Operation that can be performed GetBandwidthAchievableTCP Binding of the operation to an action IEPM#GetBandwidthAchievableTCP Definition of input and output parameters Target and time BandwidthAchievableType Insert the WSDl relevant code in here!! ESCC Jan28-29 IEPM Web Services

IEPM group at SLAC “The IEPM group at SLAC is monitoring network connectivity and End-to-end performance for sites involved in High Energy Nuclear and Particle Physics” IEPM Bandwidth to the World monitors ~35 sites around the world and analyzes the data to determine variations and anomalies http://www-iepm.slac.stanford.edu/index.html ESCC Jan28-29 IEPM Web Services

IEPM BW tests The tests that are periodically running are: Ping and traceroutes Iperf = Single and multistream BBftp = Multistream file transfer software, used at SLAC to move physics data offisite ABwE = the Available Bandwidth Estimator (developed by J. Navratil) GridFTP = Data transfer protocol optimized for WANs ESCC Jan28-29 IEPM Web Services

IEPM BW data The data is available on the Web: http://www.slac.stanford.edu/comp/net/bandwidthtests/antonia/html/slac_wan_bw_tests.html For Web Services we started to store the data in an Oracle database: Easier retrieval than compared to flat files Easier data correlation in complex queries ESCC Jan28-29 IEPM Web Services

SLAC Web Services servers We have two servers running at SLAC: SOAP server XML-RPC server Implementation is fairly similar. Both servers dispatch incoming requests from clients to a Perl module (i.e IEPM): The Perl module connects to the database back-end where the data is stored. XML-RPC #!/usr/local/bin/perl use SOAP::Transport::HTTP; SOAP::Transport::HTTP::CGI -> dispatch_to('IEPM') -> handle; #!/usr/local/bin/perl use XMLRPC::Transport::HTTP; XMLRPC::Transport::HTTP::CGI -> dispatch_to('IEPM') -> handle; SOAP ESCC Jan28-29 IEPM Web Services

SOAP client The client can contact the server to extract data, providing: End node name Time period (start-end) Method name #!/usr/local/bin/perl -w use SOAP::Lite; my $node = "node1.cacr.caltech.edu"; my $timePeriod="20031201-20031205T143000"; my $measurement = SOAP::Lite ->service('http://www-iepm.slac.stanford.edu/tools/soap/wsdl/IEPM_profile.wsdl') ->GetBandwidthAchievableTCP("$node", "$timePeriod"); ESCC Jan28-29 IEPM Web Services

XML-RPC client Works in the same way as the SOAP client, with the same input parameters and few modication to the calling code: #!/usr/local/bin/perl -w use XMLRPC::Lite; my $node = "node1.cacr.caltech.edu"; my $timePeriod ="20031201-20031205T143000"; my $toolName = "iperf"; my $streams = 8; my $measurement = XMLRPC::Lite -> proxy('http://www-iepm.slac.stanford.edu/cgi-wrap/advisor.cgi’)-> call('IEPM.GetBandwidthAchievableTCPMultiStream',"$node", "$timePeriod","$toolName", "$streams") ->result; ESCC Jan28-29 IEPM Web Services

IEPM “canned” client To allow people to see the available data, without having to code their own client, we provide a public client interface to our Web Services: http://www-iepm.slac.stanford.edu/tools/soap/IEPM_client.html Data can be: Printed Plotted Time periods can be: User Specified Pre-defined: 24 hours 7 days 30 days ESCC Jan28-29 IEPM Web Services

Collaborating Need somebody else at the other end, a consumer of the data… So far: FNAL IEPM Web Services. They have implemented servers to serve the data collected there. (Maxim Grigoriev) <add url to FNAL IEPM data) To come: Monalisa interface to IEPM via web services. (Iosif LeGrand) <add URL to Monalisa> Write a script to get the data served by Maxim. ESCC Jan28-29 IEPM Web Services

Future developments Upgrade the service to use the most recent request/response schemas. Provide access via Web Services to PingER data. Integrate our measurements into the Advisor tool, for end-to-end performance debugging. Work with Grid developers at SLAC to allow to use the service for best path determination. ESCC Jan28-29 IEPM Web Services

Conclusion Questions and comments? The SLAC Web Services page: http://www-iepm.slac.stanford.edu/tools/web_services Our email address: iepm-l@slac.stanford.edu ESCC Jan28-29 IEPM Web Services