1. The gLite middleware distribution
Oliver Keeble
CERN

2. Overview Why is gLite? What is gLite? Services and status Roadmap
Release process
The future
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

3. Middleware What’s so great about being in the middle anyway?
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

4. Why is gLite?
So we need software to manage the infrastructure, but why does EGEE produce a distribution?
Simplify availability of services and updates
Provide integrated 'services‘ from components
In the absence of standards, promote interoperability
Identify endorsed projects at certified versions
Provide an entry point for the uninitiated
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

5. Where does gLite begin... and end?
gLite is not just somewhere we stick stuff which might be useful for grids
Adding something to gLite has a cost for the project
Core services
Relationship with
Applications
RESPECT
Operational Tools
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

6. How is gLite put together?
gLite is…
A distribution
With its own eponymous development project
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

7. gLite 3.1 The current release is gLite 3.1
It is updated almost every week
Its purpose is to provide a stable platform for production grid usage
Some highlights
Data Management
Workload Management
Information System
Security and Authorisation
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

8. What’s in there?
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

9. What’s in there… continued
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

10. Running jobs
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

11. Workload Management
Just getting a job started involves a lot of services
VOMS
BDII
WMS UI
MyProxy WN
TORQUE CE
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

12. Data Management
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

13. The future The EGEE/gLite development activity is JRA1
Most of their effort is earmarked for maintenance
Logging, monitoring, docs
Address the bugs as found on the EGEE Production and Pre-Production Systems
Provide support
Address the short/medium term requests of the applications as decided by the TMB
Improve the manageability of the middleware in terms of deployability, reliability and usability.
Provide the internal unit testing of new or modified software
Participate in the definition of the gLite releases together with the SA3 activity
Maintain a web page with the relevant information needed by middleware users
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

14. Authorisation service
The single biggest chunk of development anticipated for EGEE-III
What will this bring?
Consistency across services
Distributed Policy Management
Auditing and Logging
Most components will have to be adapted
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

15. Authorisation Service
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

16. CREAM The infrastructure currently runs with the lcg-CE
CREAM is the scheduled replacement
Computing Resource Execution And Management service
Scalability
Parameter passing
Direct submission WS interface
job management (inc proxy renewal)
Improved service administration
A BES/JSDL interface is planned
A first release is ready now for production
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

17. WMS/ICE
WMS
Submission
through the WMS
CREAM
Direct Job
Submission
Job management tasks are delegated to the gLite Workload Management Service
Can now sustain 30k jobs per day under normal operations
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

18. Support for pilot jobs
A pilot job is a shell which pulls in the ‘real’ job when its execution starts
This is now a common way for VOs to manage their workloads
As it runs something on behalf of another user, the mapping to local accounts has to be redone; glexec
This mapping must be consistent across a site to prevent two different users sharing an account; SCAS
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

19. SAML2-XACML2 Query Protocol
glexec and SCAS
Site
Query for Authorisation decisions on behalf of users
gLExec-on-WN
LCMAPS
SAML2-XACML2 Query Protocol
GUMS & SAZ
SCAS
LCAS & LCMAPS
Site Central AuthZ Service
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

20. Encrypted Data Storage
gLite provides solutions for managing sensitive data
One application is medical data management, where DICOM is a standard for image acquisition and analysis
gLite will release new services for this
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

21. The great thing about standards…
Nordugrid
CREAM
ARC
EGEE
PBS/Torque
LSF
Condor
Load Leveler
Sun Grid Engine
GRAM v2
Unicore
OSG
DEISA
GRAM v4
NAREGI
Teragrid
Naregi
There are as many Computing Interfaces as Batch Systems!
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

22. Some standards relevant to gLite
Glue2
Encapsulates the ‘accumulated wisdom’ of a number of years of grid operations
Fixes problems
Better model
e.g. relationships between main entities in compute model have been improved
Buy-in
BES/JSDL
Proposed as solutions by OGF
Viability has not been established, investigations are ongoing
SRM2
The Storage Resource Manager (SRM) is a middleware component whose function is to provide dynamic space allocation and file management on shared storage components on the Grid.
The DPM supports SRM2.2
IGTF / x509
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

23. Glue 2
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

24. How gLite is put together
We allow independent things to remain independent
We make a gLite release almost every week!
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

25. Patch States [ ] Closed States In Configuration Configured [yes]
Acceptance
Criteria
Config
Required
With Provider
Ready for
Integration
Ready For
Certification
[yes]
[no]
Open
[no]
[yes]
Pass
[yes]
Pass
Pass In
Pre Production In
PPS-PDT
Certified
In Certification
[yes]
[no]
[no]
[no]
Other
Update
Available
SA1
SA3
JRA1
[no]
[yes] [
In Production
Rejected
Obsolete
] Closed States
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

26. Testing infrastructure
Framework
SAM, Nagios, Manual Testing
Regression Tests
Extensive use of virtualisation
ETICS
Move testing as far upstream as possible
Deployment tests
Regression tests
Unit tests
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

27. A gLite 3.1 update announcement
This indicates to an administrator which services they should update
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

28. Multiplatform Support
gLite currently runs on SL3 and SL4 ‘generations’, including selected services on 64bit
A release of the WN on SL5 is nearly ready
More to come
Work continues on Debian 4
Now profiting from earlier work to make the codebase more portable
Our build programme is being overhauled to be more multiplatform friendly
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

29. Challenges for gLite Release mechanisms IPv6
What is the best way to deliver the software?
IPv6
Response to virtualisation
Now the resource can adapt to the application, not the other way around
Distribution of VMs for services or applications
Support for running jobs in VMs
Retirement and end-of-life for services and platforms
Fight the Complexity
Partially a result of heterogeneity
Improve Agility
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

30. gLite Consortium
EGI as currently proposed does not directly cover middleware development
The EGEE PMB recommended the construction of a new organisation capable of guaranteeing the support and evolution of gLite in the EGI timeframe
Would allow gLite to be properly represented within EGI or a UMD project
A proposal for the gLite consortium is currently under discussion
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN

31. Summary
gLite is critical in the realisation of one of the principal goals of EGEE; Maintain, enhance and simplify the use of the production quality computing infrastructure for an increasing range of researchers in diverse scientific fields
gLite is enabling grids for e-science in Europe, across a multitude of scientific disciplines
While there are plenty of new ideas in the pipeline, the overarching goal is that stated above, and the primary focus is now on consolidation
The gLite middleware - EGEE 08 Istanbul - Oliver Keeble, CERN