Information Services Discussion TeraGrid ‘08 NSF TeraGrid Review January 10, 2006 Information Services Discussion TeraGrid ‘08 John-Paul (JP) Navarro TeraGrid Grid Infrastructure Group (GIG) Area Co-Director for Software Integration University of Chicago, Argonne National Laboratory June 2008 Charlie Catlett (cec@uchicago.edu)

Vision The TeraGrid's Information Services vision is to: NSF TeraGrid Review Vision January 10, 2006 The TeraGrid's Information Services vision is to: define a coordinated way for TeraGrid participants to publish what they offer users, define a way for the TeraGrid to aggregate and index the information from all TeraGrid participants, and to publish this information to the public in a form that can easily be used by other software, users, and TeraGrid service providers themselves. Our motivating vision major improvements to how TeraGrid Service Providers communicate information about their service offerings to the User Community June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

High-Level Components NSF TeraGrid Review High-Level Components January 10, 2006 TeraGrid Wide Information Services Apache 2.0 WS/REST HTTP GET Clients Cache Tomcat WebMDS TeraGrid Wide Information WS/SOAP Clients WS MDS4 Service Provider Information Services WS/SOAP Service Provider Information WS MDS4 Clients June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

High-Availability Design NSF TeraGrid Review High-Availability Design January 10, 2006 TeraGrid Wide Information Services Clients info.teragrid.org Service Provider Information Services info.dyn.teragrid.org TeraGrid Dynamic DNS This is both a high-availability and high-throughput design Server failover propagates globally in 15 minutes … Static paths Dynamic paths June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

Information Services Design Goals NSF TeraGrid Review Information Services Design Goals January 10, 2006 Applies Grid concepts to information publishing Publishing is the responsibility of the information owner Publishing is done using standard (content) schemas Publishing thru standard interfaces regardless of content and where the data comes from Publishing services should be available globally (subject to authentication/authorization) Information owners publish to EVERYONE, not just the TeraGrid Publishing is a grid service Applies Grid concepts to aggregating information Publishing aggregated information is done exactly like original information publishing Aggregation uses standard information services interfaces to retrieve information This is how a collaboration, such as the TeraGrid, aggregates participant information Applies Grid concepts to querying information Querying can use standard interfaces regardless on content Our motivating vision major improvements to how TeraGrid Service Providers communicate information about their service offerings to the User Community June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

Service Provider vs TG Wide Services NSF TeraGrid Review Service Provider vs TG Wide Services January 10, 2006 Services Provider Information Services Content: Locally owned and maintained information Originates anywhere the service provider wishes Services: 1 general purpose MDS service 1 scheduling MDS service TeraGrid Wide Information Services Aggregate/index service provider information Additional central information (TGCDB, GIG operated services, …) Cached (service providers services can be down) Authenticated registrations Several redundant servers (99.5% plus availability) Information caching (persistence) Several MDS4 services (WS/SOAP) WebMDS/Tomcat, Apache 2.0, … services (WS/REST) Content published in: HTML, XML, CVS, … June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

Tooling WS/* (Tomcat 5.0, Apache 2.0) WebMDS (Globus 4.0.x/VDT 1.7.1) NSF TeraGrid Review Tooling January 10, 2006 WS/* (Tomcat 5.0, Apache 2.0) Benefits Very common web services platform Supports several web service interfaces (including simple) Supports multiple styles like REST, Web 2.0 Can be highly scalable Content Many formats: HTML, XHTML/XML, XML, RSS/Atom, … WebMDS (Globus 4.0.x/VDT 1.7.1) Live MDS4 content access XPath support XSLT transforms Many formats: HTML, XHTML/XML, XML, RSS/Atom WS/SOAP (Globus 4.0.x/VDT 1.x.y MDS4) Indexing, Trigger Registration, Publish, Subscribe Security/Authorization Robust WSRF interface XML June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

Current Content CTSS v4 Capability Kits NSF TeraGrid Review Current Content January 10, 2006 CTSS v4 Capability Kits Services Software Site & Resource Cross-Reference Information Service identifiers TGCDB identifiers and descriptions Science Gateways Descriptive Information Local HPC Software Prototype Coordinated software and services -> CTSS Local (uncoordinated) HPC software June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

CTSS 4 Capability Kits For each capability kit on each resource NSF TeraGrid Review CTSS 4 Capability Kits January 10, 2006 For each capability kit on each resource Current support level, and target support level Development, Testing, Production Support organization and contact Inca status URL Multiple version of a kit with different support levels June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

Site & Resource Cross Reference NSF TeraGrid Review Site & Resource Cross Reference January 10, 2006 June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

Science Gateways NSF TeraGrid Review January 10, 2006 June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

Content in development NSF TeraGrid Review Content in development January 10, 2006 HPC Local Software information Extended GridFTP information (Bryon Gill and SI) TGCP configuration information Core 2.0 Resource Description Repository "RDR" (Ed Hanna) Co/Meta-scheduling information (Warren Smith) SPRUCE On-Demand information (Suman Nadella) Science Gateway software and services (Jason Reilly, John McGee) Coordinated software and services -> CTSS Local (uncoordinated) HPC software June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

Soft versus Hard State Aggregation layer Leaf layer NSF TeraGrid Review Soft versus Hard State January 10, 2006 Aggregation layer MDS default is Soft state TeraGrid customized Hard state Leaf layer Determined by the provider design Coordinated software and services -> CTSS Local (uncoordinated) HPC software June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

Open Discussion Content delivery formats NSF TeraGrid Review Open Discussion January 10, 2006 Content delivery formats XML CSV JSON Perl text tginfo universal command line tool ….. Coordinated software and services -> CTSS Local (uncoordinated) HPC software June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

Publishing New Content NSF TeraGrid Review Publishing New Content January 10, 2006 Service Provider Information Services TeraGrid Wide Information Services Requirements gathering Identify content Information ownership Information (system) sources Aggregation/refresh/caching Access requirements Content integration Is (some) content in information services How is the content indexed/mapped with other content Development Choose existing schema and/or develop new schema Use existing or develop information providers Use existing or develop aggregation/refresh/caching Use existing or develop access views/applications This is both a high-availability and high-throughput design June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

New Content Access Views NSF TeraGrid Review New Content Access Views January 10, 2006 Service Provider Information Services TeraGrid Wide Information Services Requirements gathering Identify content Query protocols Query aggregation scope Query reliability Query frequency/performance For users and/or software Development Choose existing access protocols and views Develop new access views Develop new access protocols This is both a high-availability and high-throughput design June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

New Content Aggregation/Storage NSF TeraGrid Review New Content Aggregation/Storage January 10, 2006 Service Provider Information Services TeraGrid Wide Information Services Requirements gathering Persistence Versioning Etc Development Extend existing aggregation/storage methods Develop new aggregation/storage methods This is both a high-availability and high-throughput design June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)

More Information Find out more: Request content: NSF TeraGrid Review More Information January 10, 2006 Find out more: http://info.teragrid.org/ (links to content and documentation) Request content: mailto: help@teragrid.org or navarro@mcs.anl.gov Discuss Information Services content, requirements, and design: E-mail list tg-cat@teragrid.org View current Information Services content User Portal (scheduler load & queue contents): https://portal.teragrid.org:443/gridsphere/gridsphere?cid=resources User Documentation (CTSS 4 kits, software, services): http://www.teragrid.org/userinfo/software/ctss.php Information Service Main Page: http://info.teragrid.org/ June 2008 TeraGrid '08 Charlie Catlett (cec@uchicago.edu)