1. Information Services Discussion TeraGrid ‘08
NSF TeraGrid Review
January 10, 2006
Information Services Discussion TeraGrid ‘08
John-Paul (JP) Navarro
TeraGrid Grid Infrastructure Group (GIG)
Area Co-Director for Software Integration
University of Chicago, Argonne National Laboratory
June 2008
Charlie Catlett

2. Vision The TeraGrid's Information Services vision is to:
NSF TeraGrid Review
Vision
January 10, 2006
The TeraGrid's Information Services vision is to:
define a coordinated way for TeraGrid participants to publish what they offer users,
define a way for the TeraGrid to aggregate and index the information from all TeraGrid participants, and
to publish this information to the public in a form that can easily be used by other software, users, and TeraGrid service providers themselves.
Our motivating vision major improvements to how TeraGrid Service Providers communicate information about their service offerings to the User Community
June 2008
TeraGrid '08
Charlie Catlett

3. High-Level Components
NSF TeraGrid Review
High-Level Components
January 10, 2006
TeraGrid Wide
Information Services
Apache 2.0
WS/REST HTTP GET
Clients
Cache
Tomcat WebMDS
TeraGrid Wide
Information
WS/SOAP
Clients
WS MDS4
Service Provider Information Services
WS/SOAP
Service Provider
Information
WS MDS4
Clients
June 2008
TeraGrid '08
Charlie Catlett

4. High-Availability Design
NSF TeraGrid Review
High-Availability Design
January 10, 2006
TeraGrid Wide Information Services
Clients
info.teragrid.org
Service Provider
Information Services
info.dyn.teragrid.org
TeraGrid Dynamic DNS
This is both a high-availability and high-throughput design
Server failover propagates globally in 15 minutes …
Static paths
Dynamic paths
June 2008
TeraGrid '08
Charlie Catlett

5. Information Services Design Goals
NSF TeraGrid Review
Information Services Design Goals
January 10, 2006
Applies Grid concepts to information publishing
Publishing is the responsibility of the information owner
Publishing is done using standard (content) schemas
Publishing thru standard interfaces regardless of content and where the data comes from
Publishing services should be available globally (subject to authentication/authorization)
Information owners publish to EVERYONE, not just the TeraGrid
Publishing is a grid service
Applies Grid concepts to aggregating information
Publishing aggregated information is done exactly like original information publishing
Aggregation uses standard information services interfaces to retrieve information
This is how a collaboration, such as the TeraGrid, aggregates participant information
Applies Grid concepts to querying information
Querying can use standard interfaces regardless on content
Our motivating vision major improvements to how TeraGrid Service Providers communicate information about their service offerings to the User Community
June 2008
TeraGrid '08
Charlie Catlett

6. Service Provider vs TG Wide Services
NSF TeraGrid Review
Service Provider vs TG Wide Services
January 10, 2006
Services Provider Information Services
Content:
Locally owned and maintained information
Originates anywhere the service provider wishes
Services:
1 general purpose MDS service
1 scheduling MDS service
TeraGrid Wide Information Services
Aggregate/index service provider information
Additional central information (TGCDB, GIG operated services, …)
Cached (service providers services can be down)
Authenticated registrations
Several redundant servers (99.5% plus availability)
Information caching (persistence)
Several MDS4 services (WS/SOAP)
WebMDS/Tomcat, Apache 2.0, … services (WS/REST)
Content published in: HTML, XML, CVS, …
June 2008
TeraGrid '08
Charlie Catlett

7. Tooling WS/* (Tomcat 5.0, Apache 2.0) WebMDS (Globus 4.0.x/VDT 1.7.1)
NSF TeraGrid Review
Tooling
January 10, 2006
WS/* (Tomcat 5.0, Apache 2.0)
Benefits
Very common web services platform
Supports several web service interfaces (including simple)
Supports multiple styles like REST, Web 2.0
Can be highly scalable
Content
Many formats: HTML, XHTML/XML, XML, RSS/Atom, …
WebMDS (Globus 4.0.x/VDT 1.7.1)
Live MDS4 content access
XPath support
XSLT transforms
Many formats: HTML, XHTML/XML, XML, RSS/Atom
WS/SOAP (Globus 4.0.x/VDT 1.x.y MDS4)
Indexing, Trigger
Registration, Publish, Subscribe
Security/Authorization
Robust WSRF interface
XML
June 2008
TeraGrid '08
Charlie Catlett

8. Current Content CTSS v4 Capability Kits
NSF TeraGrid Review
Current Content
January 10, 2006
CTSS v4 Capability Kits
Services
Software
Site & Resource Cross-Reference
Information Service identifiers
TGCDB identifiers and descriptions
Science Gateways
Descriptive Information
Local HPC Software Prototype
Coordinated software and services -> CTSS
Local (uncoordinated) HPC software
June 2008
TeraGrid '08
Charlie Catlett

9. CTSS 4 Capability Kits For each capability kit on each resource
NSF TeraGrid Review
CTSS 4 Capability Kits
January 10, 2006
For each capability kit on each resource
Current support level, and target support level
Development, Testing, Production
Support organization and contact
Inca status URL
Multiple version of a kit with different support levels
June 2008
TeraGrid '08
Charlie Catlett

10. Site & Resource Cross Reference
NSF TeraGrid Review
Site & Resource Cross Reference
January 10, 2006
June 2008
TeraGrid '08
Charlie Catlett

11. Science Gateways NSF TeraGrid Review January 10, 2006
June 2008
TeraGrid '08
Charlie Catlett

12. Content in development
NSF TeraGrid Review
Content in development
January 10, 2006
HPC Local Software information
Extended GridFTP information (Bryon Gill and SI)
TGCP configuration information
Core 2.0 Resource Description Repository "RDR" (Ed Hanna)
Co/Meta-scheduling information (Warren Smith)
SPRUCE On-Demand information (Suman Nadella)
Science Gateway software and services (Jason Reilly, John McGee)
Coordinated software and services -> CTSS
Local (uncoordinated) HPC software
June 2008
TeraGrid '08
Charlie Catlett

13. Soft versus Hard State Aggregation layer Leaf layer
NSF TeraGrid Review
Soft versus Hard State
January 10, 2006
Aggregation layer
MDS default is Soft state
TeraGrid customized Hard state
Leaf layer
Determined by the provider design
Coordinated software and services -> CTSS
Local (uncoordinated) HPC software
June 2008
TeraGrid '08
Charlie Catlett

14. Open Discussion Content delivery formats
NSF TeraGrid Review
Open Discussion
January 10, 2006
Content delivery formats
XML
CSV
JSON
Perl text
tginfo universal command line tool
…..
Coordinated software and services -> CTSS
Local (uncoordinated) HPC software
June 2008
TeraGrid '08
Charlie Catlett

15. Publishing New Content
NSF TeraGrid Review
Publishing New Content
January 10, 2006
Service Provider
Information Services
TeraGrid Wide Information Services
Requirements gathering
Identify content
Information ownership
Information (system) sources
Aggregation/refresh/caching
Access requirements
Content integration
Is (some) content in information services
How is the content indexed/mapped with other content
Development
Choose existing schema and/or develop new schema
Use existing or develop information providers
Use existing or develop aggregation/refresh/caching
Use existing or develop access views/applications
This is both a high-availability and high-throughput design
June 2008
TeraGrid '08
Charlie Catlett

16. New Content Access Views
NSF TeraGrid Review
New Content Access Views
January 10, 2006
Service Provider
Information Services
TeraGrid Wide Information Services
Requirements gathering
Identify content
Query protocols
Query aggregation scope
Query reliability
Query frequency/performance
For users and/or software
Development
Choose existing access protocols and views
Develop new access views
Develop new access protocols
This is both a high-availability and high-throughput design
June 2008
TeraGrid '08
Charlie Catlett

17. New Content Aggregation/Storage
NSF TeraGrid Review
New Content Aggregation/Storage
January 10, 2006
Service Provider
Information Services
TeraGrid Wide Information Services
Requirements gathering
Persistence
Versioning
Etc
Development
Extend existing aggregation/storage methods
Develop new aggregation/storage methods
This is both a high-availability and high-throughput design
June 2008
TeraGrid '08
Charlie Catlett

18. More Information Find out more: Request content:
NSF TeraGrid Review
More Information
January 10, 2006
Find out more:
(links to content and documentation)
Request content:
mailto: or
Discuss Information Services content, requirements, and design:
list
View current Information Services content
User Portal (scheduler load & queue contents):
User Documentation (CTSS 4 kits, software, services):
Information Service Main Page:
June 2008
TeraGrid '08
Charlie Catlett