1. www.eu-etics.org INFSOM-RI-026753 Elisabetta Ronchieri INFN CNAF ETICS 2 nd EU Review (CERN) 15 February 2008 WP3 - Software Configuration Tools and Methodologies

2. INFSOM-RI-026753 EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February 2008 2 GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

3. INFSOM-RI-026753 EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February 2008 3 GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

4. INFSOM-RI-026753 EGEE and gLite are registered trademarks Goals ETICS 2nd EU Review - CERN - 15 February 2008 4 G1. Provide a configuration, build and integration system able to describe software tasks to be executed by the ETICS facility. G2. Provide a configuration information schema able to capture the configuration information of software developed by software project and a coherent storage of such information in the ETICS database

5. INFSOM-RI-026753 EGEE and gLite are registered trademarks Goals ETICS 2nd EU Review - CERN - 15 February 2008 5 G3. Web Server: the heart of the ETICS system –Provide a logic for the entire ETICS system –Handle access to the ETICS system in secure mode –Handle metadata, artefacts, and user requests –Perform complex queries on behalf of the clients –Handle build/test remotely –Support build/test locally G4. Command Line Interfaces (CLIs): one of the ETICS clients –Allow users to register their software, specify configuration information (i.e., edit metadata) –Handle access to Web Service in secure mode –Checkout code and metadata –Build/Test code locally and remotely –Tag code and metadata –Produce reports and different format of artefacts

6. INFSOM-RI-026753 EGEE and gLite are registered trademarks ETICS CLIs, Build/Test Web Serivce and metadata ETICS Portal (myETICS, Build/Test, Repository, Administration) Report DB Metronome Execution Engine Metronome Client Wrapper WNs ETICS Infrastructure Repository Service Build/Test Artefacts Project DB Clients Web Service Via browser

7. INFSOM-RI-026753 EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February 2008 7 GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

8. INFSOM-RI-026753 EGEE and gLite are registered trademarks Achievements 8 ETICS 2nd EU Review - CERN - 15 February 2008 A1. Authentication and Authorization Performed authentication using standard x.509 certificates in Web Service and CLIs Registered information of users (e.g., name and distinguish name) in the ETICS data model Based authorization on role definition –i.e., defined roles to descriminate the operations users can perform –e.g., clients connecting without certificate are mapped as Guest Allowed or denied operations to the users by Web Service RoleDescription AdministratorSuper user Module AdministratorHandles individual projects Developerruns remote builds Integratorruns remote builds, registers artefacts in the repository Testerruns remote tests, stores results in the report repository Release ManagerDefines configurations at project level, and project releases GuestRead-only access

9. INFSOM-RI-026753 EGEE and gLite are registered trademarks Achievements 9 ETICS 2nd EU Review - CERN - 15 February 2008 A2. Software Metadata definition Based schema version on Common Information Model (CIM) Defined formal entities to express project structure, build/test/checkout configurations, security information, dependencies, environment variables, properties, users, roles –E.g., software project structure and configuration structure –Project, Subsystem and Component referred as Modules –More than one Configurations is allowed for each Module –More than one Platforms is allowed for each Configuration –Set Hierarchy in Project and Subsystem Configurations Project Subsystem Component... Subsystem... Component..... Configuration Platform 1 VCS Command Build Command Test Command Dependency1 Property1 Environment1 Platform n VCS Command Hierarchy1

10. INFSOM-RI-026753 EGEE and gLite are registered trademarks Achievements 10 ETICS 2nd EU Review - CERN - 15 February 2008 A3. Build/Test Handled the downloading of code and metadata information Supported the build and test locally and remotely Generated platform specific artefacts (e.g., RPMS, MSI, Debian, tarballs) during the build of software Generated build and test reports Supported the browsing of software information Supported the editing of software metadata –e.g., the possibility to create metadata, modify/remove them and copy existing metadata Supported co-scheduling by the editing functionality –DT  {node1, node2, …, nodeN} –Set hierarchy in DT configuration –Node  {service1, service2, …, serviceN} –Set services in Node configuration Handled code and metadata tag Project DestrebutedTest Node... Subsystem... Component.....

11. INFSOM-RI-026753 EGEE and gLite are registered trademarks Achievements 11 ETICS 2nd EU Review - CERN - 15 February 2008 A4. Interoperability and Compliance to Standards Metrics –Identified four interoperability metrics 1. Number of 3rd Party Components (No3C) 2. Number of Generic Components (NoGC) 3. Number of Architectural Revisions (NoAR) 4. Number of Versions (NoV) –Used configuration information stored in the ETICS database Collectors –Provided information collectors for the reporting engine –Compliance to standards: Ipv6 WS-compliance –Interoperability Metrics

12. INFSOM-RI-026753 EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February 2008 12 GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

13. INFSOM-RI-026753 EGEE and gLite are registered trademarks Challenges ETICS 2nd EU Review - CERN - 15 February 2008 13 C1. Improve the data model in order to guarantee a better response time from the backend –CIM does not scale to large software applications –Started a revision of the data model C2. Improve the CLI performances –Current version based on CIM and SOAP –As the size of a project increases, the performance of the client decreases to handle large XML documents –Substantially performance improvements by using newer version of the Python SOAP engine (called ZSI) in handling large XML documents –Started tests in the ETICS client C3. Improve the calculation of the configuration properties –Substantially performance improvements by making better use of caching

14. INFSOM-RI-026753 EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February 2008 14 GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

15. INFSOM-RI-026753 EGEE and gLite are registered trademarks WP3 in Numbers 15 ETICS 2nd EU Review - CERN - 15 February 2008 Deliverables: Milestones: Del. N.Del. NameDelivery Date D3.1Configuration, build and integration specificationsPM03 D3.2Interoperability reports specifications, configuration, build and integration systemPM12 D3.3Final evaluation report and configuration, build and integration system (final release)PM20 Mil. N.Mil. DescriptionDelivery Date M3.1Configuration, build and integration system deployed in the ETICS resource pools (prototype)PM12 M3.2Configuration, build and integration system deployed in the ETICS resource pools (final release)PM22

16. INFSOM-RI-026753 EGEE and gLite are registered trademarks WP3 in Numbers 16 ETICS 2nd EU Review - CERN - 15 February 2008 OSVersionArchitectureCompiler Fedora Core3ia32gcc 3.4.4 5ia32gcc 4.1.1 6ia32, x86_64gcc 4.1.1 Scientific Linux3ia32, x86_64gcc 3.2.3 5Ia32gcc 4.1.1 Scientific Linux Cern3ia32, x86_64gcc 3.2.3, gcc 3.4.5 4ia32, ia64, x86_64gcc 3.4.6 Red Hat Enterprise Linux3ia32gcc 3.2.3 4ia32, x86_64gcc [3.4.3, 3.4.6], gcc [3.4.5, 3.4.6] SUSE9ia32, x86_64gcc [3.3.1, 3.3.5], gcc 3.3.3 10ia32gcc [4.0.2, 4.1.0] Solaris10ia32gcc 3.4.3 Debian3ia32, x86_64gcc 3.3.5 4ia32, x86_64gcc 4.1.2 centOS4ia32, x86_64gcc [3.4.4, 3.4.6], gcc 3.4.6 5ia32, x86_64gcc 4.1.2, gcc 4.1.1 Darwin/Mac OS772powerpcgcc 3.3.3 880powerpcgcc 4.0.1 891ia32gcc 4.0.1 8100powerpcgcc 4.0.0 ubuntu5Ia32gcc 4.0.2 6Ia32gcc 4.0.3 7ia32gcc 4.1.2 Win3271 aix52powerpcgcc 3.3.3 Measurements Supported Platforms 41Supported Platforms 41 –Fedora Core 4 –Scientific Linux 3 –Scientific Linux Cern 5 –Red Had Enterprise Linux 5 –SUSE 5 –Solaris 1 –Debian 4 –centOS 5 –Darwin/Mac OS 4 –win32 1 –aix 1 –ubuntu 3

17. INFSOM-RI-026753 EGEE and gLite are registered trademarks WP3 in Numbers 17 ETICS 2nd EU Review - CERN - 15 February 2008 Measurements: –Supported languages 7 –C, C++, Python, Java, Perl, sh, csh –Implemented Requirements 97.56% D3.1 D3.1 –Total number of open and closed bugs over time –Ratio of closed/open bug over time RatioYear 0.902006 0.672007

18. INFSOM-RI-026753 EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February 2008 18 GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

19. INFSOM-RI-026753 EGEE and gLite are registered trademarks Conclusions 19 ETICS 2nd EU Review - CERN - 15 February 2008 Most important achievements A1. Authorization and Authentication A2. Software Metadata Definition A3. Build/Test A4. Interoperability and compliance to standards Three outstanding challenges C1. Data Model Structure C2. CLI Performances C3. CLI Property Calculation Planned in ETICS II activities to address the challenges

20. INFSOM-RI-026753 EGEE and gLite are registered trademarks ETICS 2nd EU Review - CERN - 15 February 2008 20 Thanks http://www.eu-etics.org