1. Status & Strategy for gLite multi-platform Support
Author: Eamonn Kenny
Meeting: PMB – All Activity Meeting
Location: Sofia, Bulgaria
Date: 9-12pm, 22nd January 2008

2. EGEE Oct’07 Conference
PMB Meeting, Sofia, Bulgaria 2

3. SA3 All Hands Meeting (Dec ’07)
ETICS is now platform independent!
PMB Meeting, Sofia, Bulgaria

4. Current Status (Jan ’08) ETICS is now platform independent!
PMB Meeting, Sofia, Bulgaria

5. Yum/Apt Repositories for WNs
Two working repositories (ignore sl3):
Also available Torque clients:
glite/R3.1/glite-TORQUE_client/suse9
glite/R3.1/glite-TORQUE_client/centos4
These are constantly upgraded, currently version:
PMB Meeting, Sofia, Bulgaria 5

6. Revised SuSE differences
Package Name and Version Reason for RPM changes
boost| Boost not available in OS
CGSI_gSOAP_2.6| dm-util requires this, whereas srm/transfer require 2.7
glite-data-catalog-api-c| glite packager: Missing ext.axis.version
glite-data-catalog-cli| glite packager: Missing ext.axis.version
glite-data-srm-api-c| glite packager: Missing ext.axis.version
glite-data-srm-api-perl| glite packager: Missing ext.axis.version
glite-data-srm-cli| glite packager: Missing ext.axis.version
glite-data-transfer-api-c| glite packager: Missing ext.axis.version
glite-data-transfer-api-perl| glite packager: Missing ext.axis.version
glite-data-transfer-cli| glite packager: Missing ext.axis.version
glite-data-delegation-cli| Needed by newer glite-data-transfer-cli
glite-data-delegation-api-c| Needed by newer glite-data-transfer-cli
glite-data-util-c| glite packager: Missing ext.axis.version
glite-lb-client| Needs glite-lb-common (newer version)
glite-lb-common| Missing a -p in mkdir ${REPORTS}
glite-wms-common| configuration is missing
perl-SOAP-Lite| Use /usr/bin/env for portability
EGEE Oct’07
EGEE Oct’07 + Dec’07
PMB Meeting, Sofia, Bulgaria 6

7. Debian Porting Debian x86_64 (PSNC, Poland)
Dependency are being generated but not debs packages.
Building tarballs only for the moment.
ETICS Client version must be used to obtain --ignorelocking with --merge and --nocache.
Ubuntu 7 (HPCN, Sweden)
Åke (Ubuntu port) has built 85% of the gLite middleware but has not tested it.
Åke wants to see personalised views of the web interface which we suggested at EGEE ’07.
Work will continue on this in March 2008.
PMB Meeting, Sofia, Bulgaria 7

8. Porting Guide Use the porting guide:
10 step guide covering building of:
ETICS, globus, dependencies, Base Modules, RGMA, VOMS, DM/GFAL and the WN meta-package
PMB Meeting, Sofia, Bulgaria

9. Looking Towards EGEE III
Multi-platform Building Infrastructure
Controlled distributed structure
Scalable (with Condor?)
Sustainable (many platforms)
Multi-platform Patching Lifecycle
Fast
Efficient
Reproducable
Multi-platform Certification Infrastructure
Sustainable (Yaim + package management issues)
Reliable (who does the certification?)
PMB Meeting, Sofia, Bulgaria

10. Simplified Possible Lifecycle of a Build
Start
SA3 Local Build
with local config
Success
SA3 CERN
NMI Build
Failed
download
Failed
Create a new
software patch
Create a full set
of artefacts
CVS
Report patch in Savannah
CERN certify
New Platform
JRA1 accept or
reject the patch
Success
Accept
Add Platform
to Distribution
JRA1 developers implement
the patch
PMB Meeting, Sofia, Bulgaria

11. Current TCD Local Build Lifecycle
Start
SA3 Local Build
Success
SA3 CERN
NMI Build
Local
Patches
Failed
Create a new
Software patch
Failed
Create a full set
of artefacts
download
CVS
Report patch in Savannah
Informally certify
New Platform
JRA1 accept or
reject the patch
Success
Accept
Posted to
Webserver
TCD
specific
differences
JRA1 developers implement
the patch
PMB Meeting, Sofia, Bulgaria

12. Questions to Consider How do we speed up the patching process?
How do local sites allow experimental local builds and CERN NMI builds on the one machine?
Who will certify the new platform? CERN or the local site?
Who does the Yaim development?
If the patching process and certification process are long, how do you ever get the new platform into step with the supported JRA1 platforms?
PMB Meeting, Sofia, Bulgaria

13. Distributed Building Infrastructure
https & DN
Another Site
Web Server
NMI Submit
NMI Webserver
NMI Server
NMI archiver
NMI/ETICS dB host
etics.cern.ch
Condor CM
NMI Submit
Execute node
NMI Webserver
ETICS client
NMI archiver
NMI Server
Condor execute
NMI/ETICS dB host
NMI Hawkeye
Condor CM
PMB Meeting, Sofia, Bulgaria

14. TCD NMI Attempt https & DN Developer Builds Experimental Local Builds
Web Server
Submit times:
6am to 6pm
daily.
nmi.cs.tcd.ie
Condor Configuration
condor_config
condor_config.local
NMI Server
etics.cern.ch
Condor Configuration
6am to 6pm
6pm to 6am
condor_config
condor_config.local
NMI Server
Execute node1
Condor Configuration
condor_config
condor_config.local
condor_config.cern
Cronjob run at
6am and 6pm
PMB Meeting, Sofia, Bulgaria

15. Distributed Build Infrastructure
Multiple Condor Managers could be an issue.
Resources controlled by CERN (CERN-centric view), all logging recorded in CERNs central manager.
Condor was designed for local open networks.
Complicated firewall rules and NATed sites should be provided while retaining robustness and scalability.
Solving the above issues is solving a problem already solved by the Grid!
PMB Meeting, Sofia, Bulgaria

16. Infrastructure Considerations
Eventually with the introduction of NGIs we need open-source code that is easy to build by anyone.
We need to respect the security restrictions of a site.
Condor allows for heterogeneous job submit. NMI provides the hooks for Condor. Putting the Grid in the middle could be a good move.
PMB Meeting, Sofia, Bulgaria

17.   test Future of Patching on Multiple Platforms
Each system introduces more configuration & packaging for the developers.
Yaim needs to be partially rewritten for each OS (possibly).
Patches from other sites needs to be automatically reported to the developers and be reproducible.
A good reporting mechanism needs to be in place for managing the inclusion of these patches. Savannah isn’t great for tracking such issues.
Site Build Report
CERN Build 
Patch1
auto-propagation
Patch1 
OSi
Patch2
OSi
Patch2
test
Patch3
Patch3
PMB Meeting, Sofia, Bulgaria

18. Possible Certification on Multiple Platforms
etics-test
etics.cern.ch
Relocatable
glite-WN tarball
NMI Webserver
SAM tests
NMI archiver
NMI Server
NMI/ETICS dB host
Condor CM
CERN or Local Site
Execute node
ETICS client
Etics report
Condor execute
Install status
NMI Hawkeye
SAM results
PMB Meeting, Sofia, Bulgaria

19. Conclusions and Future Work
Debian is the top priority at the moment.
ETICS is stabilising, going towards user porting views.
Helping PSNC with Debian x86_64 will automatically help Debian x86 at CERN.
NMI testing starting at TCD.
TCD will have CentOS 5.1 gLite-WN built soon. Will be deployed at one site in Ireland (most likely).
(SA3 list  JRA1 subset build) generator is being designed by TCD for gLite-WNs. Needs some formal reasoning.
A gLite-WN diff build will also be defined by TCD. This will analyse existing repositories to deduce the minimum subset to build.
A stable non-intrusive ETICS/NMI infrastructure with automatic patch reporting, per platform user views and automatic certification processes will greatly help the effort in EGEE III and well on into the future.
Eventually ETICS submit should be set up at local sites with a local ETICS dB/repository and some form of mirroring.
PMB Meeting, Sofia, Bulgaria