1. NSF TeraGrid Review
January 10, 2006
TeraGrid’s Integrated Information Service “IIS” Grid Computing Environments 2009
Lee Liming, JP Navarro, Eric Blau, Jason Brechin, Charlie Catlett, Maytal Dahan, Diana Diehl, Rion Dooley, Michael Dwyer, Kate Ericson, Ian Foster, Ed Hanna, David L. Hart, Chris Jordan, Rob Light, Stuart Martin, John McGee, Laura Pearlman, Jason Reilly, Tom Scavo, Michael Shapiro,
Shava Smallen, Warren Smith, Nancy Wilkins-Diehr
TeraGrid Grid Infrastructure Group (GIG)
University of Chicago, Argonne National Laboratory
November 2009
Charlie Catlett

2. Outline Introduction: 1st IIS System Architecture
NSF TeraGrid Review
Outline
January 10, 2006
Introduction:
Conceived in 2006; Production in 2007; Presented at GCE’07.
IIS Vision
1st IIS System Architecture
Distributed CI provider operated local information services
Centralized federation wide information services
Registries -> XML document entries
2nd IIS Information Architecture
Registry architecture and data format
The Capability Kit meta-registry
Current information registries
Leveraging IIS Examples – Providers and Consumers
Conclusion and Future Work
We are presenting the many changes over the last 2 years
November 20, 2009
GCE09
Charlie Catlett

3. Vision Human discovery of cyber-infrastructure
NSF TeraGrid Review
Vision
January 10, 2006
Provide an Authoritative Integrated Information Service enabling:
Human discovery of cyber-infrastructure
Science Gateways, Portals, Documentation, CLIs
Software discovery of cyber-infrastructure
For automated resource, service, and software selection and access
For auto-configuration (applications, gateways, workflow engines)
Providers to advertise their cyber-infrastructure offerings
Advertise any information about any CI capability
Providers own data, and independently control publishing
Streamlined operations
Change integration and management
Automated testing, and monitoring
Enabling Discovery: BOTH by humans and by software, these combined require a formal information model and repository driving both
To be authoritative it needs to be the same information, human information based on software information
Of course: to make it as easy as possible for everyone to discover and/or advertise
NOTE we don’t consider it in IIS scope to develop human interfaces,
or to manage the source information (we’re not a database)
IIS is the infrastructure information pipeline (Grid)
November 20, 2009
GCE09
Charlie Catlett

4. Easy to Advertise and Discover
NSF TeraGrid Review
Vision
January 10, 2006
Provide an Authoritative Integrated Information Service enabling:
Human discovery of cyber-infrastructure
Science Gateways, Portals, Documentation, CLIs
Software discovery of cyber-infrastructure
For automated resource, service, and software selection and access
For auto-configuration (applications, gateways, workflow engines)
Providers to advertise their cyber-infrastructure offerings
Advertise any information about any CI capability
Providers own data, and independently control publishing
Streamlined operations
Change integration and management
Automated testing, and monitoring
Easy to Advertise and Discover
This directly impacted our data format, tool selection, and interface design
November 20, 2009
GCE09
Charlie Catlett

5. Distributed Architecture Components
NSF TeraGrid Review
Distributed Architecture Components
January 10, 2006
WS/REST
Federation Wide Integrated Information Service
WS/SOAP
Apache 2.0
XML
Repository
Tomcat WebMDS
Clients
TeraGrid Wide
Databases
WS MDS4
Distributed design
Aggregation flexible by information type
Top level implementation requirement was to make it easy to discover and advertise
Service Provider Local Information Service
WS MDS4
Clients
HTTPD
November 20, 2009
GCE09
Charlie Catlett

6. High-Availability Architecture
NSF TeraGrid Review
High-Availability Architecture
January 10, 2006
High-Level
Aggregation
Service Provider
Publishing
Clients
info.teragrid.org
Dynamic DNS
This is both a high-availability and high-throughput design
Dynamic DNS provides a 15 minute ttl in failover situations
Automated difference detection, manual synchronization …
November 20, 2009
GCE09
Charlie Catlett

7. Information Architecture
NSF TeraGrid Review
Information Architecture
January 10, 2006
Registry Architecture
Named Registries, with schema compliant
Registry Entries, which are each an
XML Document
The Capability Deployment Meta-Registry
Universal Identifiers
Site and Resource Identifiers
Capability Identifier
Registry entry cross-references
Extensibility
Meta-Registry Extensions
New Registries
XML
Registry C
Entry 1
Entry 2
Entry 3 …
Registry B
Entry 1
Entry 2
Entry 3 …
Registry A
Entry 1
Entry 2
Entry 3 …
Universal identifiers enable the “Integrated” vision of IIS
Note that the Meta-Registry is part of the information architecture, nothing in the system architecture requires it’s existence
<Reg1.Entry>
<id>entry1</id>
<foo>bar</foo>
</Reg1.Entry>
November 20, 2009
GCE09
Charlie Catlett

8. TeraGrid Capability Meta-Registry
NSF TeraGrid Review
TeraGrid Capability Meta-Registry
January 10, 2006
Each Capability Deployment
Where (site and resource)
What (name, class, and description)
Support information
Status information
Software and services component information
Extensions
November 20, 2009
GCE09
Charlie Catlett

9. Capabilities Kit Registry by Class
NSF TeraGrid Review
Capabilities Kit Registry by Class
January 10, 2006
CTSS
Gateways
Renci Portal …
Application Development & Runtime
TeraGrid Core Integration (local info service)
Co-scheduling, meta-scheduling
Common Client
Computation & Scheduling Clients
Data Collections
Data Management
Data Movement servers, Clients
Distributed Parallel Application Support
Distributed Programming Systems
Local Compute
Login
Nimbus/Cloud Computing
Parallel Application Support
Remote Computation
Science Gateway Support
Visualization Software (VTSS)
WAN GPFS, WAN Lustre file-systems
Workflow Support
Local
Local HPC Software
Central
A single capability meta-registry enables a universal capability discovery interface
CTSS are the capabilities coordinated across HPC and Storage resources
Credential Server (MyProxy)
Integrated Information Services
User Portal
November 20, 2009
GCE09
Charlie Catlett

10. Other Registries Gateways Registries Local RP Registries
NSF TeraGrid Review
Other Registries
January 10, 2006
Gateways Registries
Science Gateway Web Services Application Registry
Local RP Registries
Local HPC Software Catalog
CTSS Extension Registries
Batch System Load (%)
Batch Queue Contents (requires authorization)
OGF GLUE2
A single capability meta-registry enables a universal capability discovery interface
CTSS are the capabilities coordinated across HPC and Storage resources
TeraGrid Central Database Registries
Site/Organization and Resource identifiers (IDs) and descriptions
Project/Allocation to Resource authorization list
TeraGrid Science Gateway Catalog
TeraGrid System Outages
November 20, 2009
GCE09
Charlie Catlett

11. Leveraging IIS Examples
NSF TeraGrid Review
Leveraging IIS Examples
January 10, 2006
Resource Description Repository Publishing
TeraGrid User Portal Batch Load & Queue Data
TeraGrid User Documentation
Software Discovery
CTSS Software
Local HPC Software
Science Gateway Software
Science Gateways Web Services “WS” Application Registry
Advanced Scheduling Information
Inca Verification & Validation
User Profile Service
Discovery CLI Interface
Significant progress over last 2 years, and our first paper
In GCE’07 I had an opportunity to present TeraGrid Information Services
November 20, 2009
GCE09
Charlie Catlett

12. Resource Description Repository “RDR”
TeraGrid Core Integration
Local Compute
Data Collections
(Storage)
TeraGrid Core Services uses RDR to collect and store validated, current and historical resource description information:
Common Resource Information
Compute Resource Information
Data Collections Information
Storage Information

13. TGUP Batch Load & Queue Data
Remote Computation -> Local Compute
IIS provides queue & batch load information from all RP sites for TGUP to use in system monitor
<LoadRP xmlns="">
<ComputeResourceLoad xmlns="">
<ResourceID>pople.psc.teragrid.org</ResourceID>
<SiteID>psc.teragrid.org</SiteID>
<LoadInfo hostname="tg-login1.pople.psc.teragrid.org" timestamp=" T13:46:19Z">
<Load>
<Type>queue</Type>
<Value>98</Value>
</Load>
November 20, 2009
GCE09

14. TeraGrid User Documentation
NSF TeraGrid Review
TeraGrid User Documentation
January 10, 2006
We our eating our own dogfood, publishing for internal consumption
Users and developers like yourself are our also our target
November 20, 2009
GCE09
Charlie Catlett

15. Software Discovery TeraGrid context: Problems:
>650 CTSS software package deployments
>1600 Local HPC software package deployments
>40 Science Gateways offering software packages
Problems:
How can users discover what software is available, and how to access it?
How can Science Gateways or Web Applications discover what software is available thru web service interfaces and invoke it?
November 20, 2009
GCE09

16. Software Discovery Solutions: Which enables, for example:
Single IIS interface to multiple software repositories including 3rd party HPC software and Science Gateway software.
A custom Gateway web services registry.
Which enables, for example:
Scientists to discover that Gaussian is available both from the command line and through a full service gateway such as GridChem (
Science Gateways and Applications to discover and invoke Gaussian web services automatically.
November 20, 2009
GCE09

17. Software Discovery Design
NSF TeraGrid Review
Software Discovery Design
January 10, 2006
Kit Registry
CTSS Kit Software
Comprehensive Software Discovery
Gateways Kit Software
Local HPC Kit Software
WS Enabled
Software Discovery
A single capability meta-registry enables a universal capability discovery interface
CTSS are the capabilities coordinated across HPC and Storage resources
Local HPC Software Registry
Gateway Web Services Registry
November 20, 2009
GCE09
Charlie Catlett

18. Gateway WS Application Registry
Each Gateway hosts a service (RESTful or otherwise) that publishes local web service metadata.
Information Services aggregates all configured Gateway hosted GAWSR metadata, creating a central registry.
Content of GAWSR metadata is rich enough to dynamically launch jobs via web services. (ie, the registry has enough metadata to allow a user/client to dynamically launch jobs)
Following slides demonstrate two clients using the GAWSR. The first & the latter is a.
November 20, 2009
GCE09

19. Dynamic execution of web services written in Java
November 20, 2009
GCE09

20. RIA Flex application showing the available metadata
November 20, 2009
GCE09

21. Local HPC Software Local HPC Software NSF TeraGrid Review
January 10, 2006
Local HPC Software
What value add does IIS provide an infrastructure
November 20, 2009
GCE09
Charlie Catlett

22. Advanced Scheduling Information
NSF TeraGrid Review
Advanced Scheduling Information
January 10, 2006
CTSS
Co-scheduling
Meta-scheduling
Computation & Scheduling Clients
Local Compute
Remote Computation
Science Gateway Support
Workflow Support
GLUE2 Registry
What value add does IIS provide an infrastructure
November 20, 2009
GCE09
Charlie Catlett

23. Inca Verification & Validation
Running on TeraGrid since 2003
Verifies IIS published information through automated, user-level testing
Total of ~2200 tests running on 18 login nodes, 2 grid nodes, and 3 servers
notifications for critical services
Status views from detailed test information to summary and historical reports
Data published as XML, HTML, or graphed
IIS compatible REST interface:
info.teragrid.org
XSL
XML
CTSS kit
registrations
November 20, 2009
GCE09

24. NSF TeraGrid Review
User Profile Service
January 10, 2006
Provide authenticated users with user-centric information
HTTPS with Basic Authentication
In html, csv, json, perl, and xml formats
What value add does IIS provide an infrastructure
November 20, 2009
GCE09
Charlie Catlett

25. Discovery CLI Interface
NSF TeraGrid Review
Discovery CLI Interface
January 10, 2006
The tginfo CLI:
What value add does IIS provide an infrastructure
November 20, 2009
GCE09
Charlie Catlett

26. Conclusion Federation Wide Standards
NSF TeraGrid Review
Conclusion
January 10, 2006
Federation Wide Standards
Information Integration Identifiers
Information Discovery REST APIs
Standard Capability Naming and Description Schemas
Federation Wide Information Discovery
Using a Central Federation Wide Index
Using a DNS/WWW model
Central Discovery  Distributed Information Access
Enable User Interfaces
Web 2.0, Science Gateways, and traditional Web servers
** IIS does not develop those interface
Our motivating vision major improvements to how TeraGrid Service Providers communicate information about their service offerings to the User Community
November 20, 2009
GCE09
Charlie Catlett

27. Conclusion & Future Work
NSF TeraGrid Review
Conclusion & Future Work
January 10, 2006
Information Architecture
Capability Definition Meta-Registry (BioMedical Informatics -- BIRN)
Capability Implementation Registry
More Capabilities and Capability Classes
Clouds/IaaS, SaaS, Distributed Programming Environments (SAGA) , Data Collections
Science Gateway Security Configuration Information (SAML)
System Architecture
Fully REST based registration services (Apache CXF, Globus CRUX)
Fully REST based aggregation services
More REST based discovery interfaces (with XPATH, XSLT support)
More custom REST service, some providing custom user services
Separate IIS project
Packaged, documented, and distributed for other projects
Our motivating vision major improvements to how TeraGrid Service Providers communicate information about their service offerings to the User Community
November 20, 2009
GCE09
Charlie Catlett

28. More Information Web Sites People http://info.teragrid.org/
NSF TeraGrid Review
More Information
January 10, 2006
Web Sites
(REST APIs)
People
JP Navarro, Lee Liming (IIS Architecture and Coordination)
Nancy Wilkins-Diehr (Gateway Information)
Warren Smith (Execution and Scheduling Information)
Ed Hannah (Resource Description Information)
Kate Ericson (Monitoring and Validation Information)
Rion Dooley (Authenticated User Custom Information)
What value add does IIS provide an infrastructure
November 20, 2009
GCE09
Charlie Catlett