1. www.eu-etics.org INFSOM-RI-026753 Elisabetta Ronchieri INFN CNAF ETICS 2 nd EU Review (CERN) 15 February 2008 WP3 - Software Configuration Tools and Methodologies

2. INFSOM-RI-026753 EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February 2008 2 WP3 in NumbersWP3 in Numbers GoalsGoals ChallengesChallenges Major AchievementsMajor Achievements Future PlansFuture Plans

3. INFSOM-RI-026753 EGEE and gLite are registered trademarks WP3 in Numbers 3 ETICS 2nd EU Review - CERN - 15 February 2008 Team:Team: Deliverables:Deliverables: AffiliationNamePeriod INFN CNAFE. RonchieriFrom 04/06 INFNE. FerroFrom 01/06 to 09/07 INFN CNAFM. SelmiFrom 04/06 to 11/07 INFNS. Da RoncoFrom 10/07 to 09/07 CERNG. Diez-Andino SanchoEntire project CERNA. Di MeglioEntire project CERNM.-E. BéginEntire project Del. N.Del. NameDelivery Date D3.1Configuration, build and integration specificationsPM03 D3.2Interoperability reports specifications, configuration, build and integration systemPM12 D3.3Final evaluation report and configuration, build and integration system (final release)PM20

4. INFSOM-RI-026753 EGEE and gLite are registered trademarks WP3 in Numbers 4 ETICS 2nd EU Review - CERN - 15 February 2008 Metrics:Metrics:

5. INFSOM-RI-026753 EGEE and gLite are registered trademarks Goals 5 ETICS 2nd EU Review - CERN - 15 February 2008 To identify and deploy the software configuration, build and packaging tools and methodologies – To collect user and application requirements about software configuration, build and integration – To provide a metadata schema able to capture the configuration information of software developed by research and non-research projects To identify software configuration items that can be used in interoperability analysis To develop a configuration, build and integration system based on existing or adapted tools To capture the process of building and integrating software so that it can be reproduced by the ETICS facility To adapt the tools in order to provide adequate functionality on all the platforms supported by the projects To develop a general packaging information tool able to use metadata-software information for the generation of platform specific packages (e.g., RPMS, MSI, Debian, tarballs)

6. INFSOM-RI-026753 EGEE and gLite are registered trademarks Goals 6 ETICS 2nd EU Review - CERN - 15 February 2008 To assist WP2 in the deployment of the tools on the computing resources To collaborate with WP4 in the definition and integration of tests in the build process To collaborate with WP4 in the definition and generation of interoperability and quality reports To store and maintain the configuration, build and integration tools in the project software repository To satisfy the requirements coming from past and present software projects To bring past and present users to adopt new tools for the building, configuration, testing and integration of their software

7. INFSOM-RI-026753 EGEE and gLite are registered trademarks Challenges 7 ETICS 2nd EU Review - CERN - 15 February 2008

8. INFSOM-RI-026753 EGEE and gLite are registered trademarks Major Achievements 8 ETICS 2nd EU Review - CERN - 15 February 2008 The developed tools and services are in production. They have been running for more than 18 months. – Currently almost 40000 remote build/tests have been executed in a total of 31 projects. Two different clients have been developed (Command Line Interfaces (CLIs) by WP3 and Web Application (WA) by WP5) and the service providing all the metadata information Both clients, the CLIs and the WA (accessible via a browser), provide the same functionalities in terms of: – Editing – Submission – Security

9. INFSOM-RI-026753 EGEE and gLite are registered trademarks Major Achievements 9 ETICS 2nd EU Review - CERN - 15 February 2008 CLIs, Web Service (WS) and Database are under the WP3 responsability. Database has been designed to organize a software project. – It explicitly describes the objects and the relationships between them. WS is the core service of the ETICS system, – managing the user requests – contacting the execution engine to commit build and test jobs – performing security checks CLIs implement similar functionalities as the WA does by using the same WS interface. – It also provides functions to create and manage build and test configurations in the local workspace without using the information in the central database. Web Application Web Service Via command-line tools Via browser Remote build/test infrastructure Database WP3

10. INFSOM-RI-026753 EGEE and gLite are registered trademarks Major Achievements 10 ETICS 2nd EU Review - CERN - 15 February 2008 By using CLIs it is possible: – To get metadata information – To checkout metadata software information, source code and binary packages – To perform local/remote build and test – Different package formats are generated according to the platforms used during build and test – To edit metadata with roll-back in case of failures (thanks to the transaction mechanism developed in the WS) This functionality also contributes to make a software configuration self-consistent, supporting the lock mechanism. It guarantees the reproducibility of configurations across the time. – To add new software configurations from existing ones and optionally tag the corresponding code in the underlying Version Control System (VCS) – ETICS is able to support different type of VCSs. – To extend ETICS tools by using the Plugin Framework (WP4), which allows adding extra functionalities and execute quality validation tools called plugins. – Plugins related to the compliance to the IPv6 and WS-I standards, the interoperability evaluation, and to the number of code lines have been developed.

11. INFSOM-RI-026753 EGEE and gLite are registered trademarks Major Achievements 11 ETICS 2nd EU Review - CERN - 15 February 2008 CLIs were evaluated by the ETICS user who filled in a questionnaire. Range of answers: 1: Never2: Rarely3: Sometimes 4: Often5: Very Often6: Always Questions:Score: Building software4,05 Testing software2,03 Editing software metadata2,61 Editing software metadata locally2,13 Building software remotely2,10 Testing software remotely1,39 Listing software metadata2,63 Handling of user permissions1,32 Reliability Range of answers: 1: Very Low2: Modest3: Fairly Good 4: Good5: Very Good6: Excellent Questions:Score: Building software4,05 Editing software metadata 2,61 Building software remotely 2,10 USAGE Frequency

12. INFSOM-RI-026753 EGEE and gLite are registered trademarks Major Achievements 12 ETICS 2nd EU Review - CERN - 15 February 2008 CLIs were evaluated by the ETICS user who filled in a questionnaire. Range of answers: 1: Never2: Rarely3: Sometimes 4: Often5: Very Often6: Always Questions:Score: Building software4,05 Testing software2,03 Editing software metadata2,61 Editing software metadata locally2,13 Building software remotely2,10 Testing software remotely1,39 Listing software metadata2,63 Handling of user permissions1,32 Reliability Range of answers: 1: Very Low2: Modest3: Fairly Good 4: Good5: Very Good6: Excellent Questions:Score: Building software4,05 Editing software metadata 2,61 Building software remotely 2,10 USAGE Frequency CLIs are mainly used for building software locally, whilst it is rarely used for testing software both locally and remotely.

13. INFSOM-RI-026753 EGEE and gLite are registered trademarks Major Achievements 13 ETICS 2nd EU Review - CERN - 15 February 2008 CLIs were evaluated by the ETICS user who filled in a questionnaire. Range of answers: 1: Never2: Rarely3: Sometimes 4: Often5: Very Often6: Always Questions:Score: Building software4,05 Testing software2,03 Editing software metadata2,61 Editing software metadata locally2,13 Building software remotely2,10 Testing software remotely1,39 Listing software metadata2,63 Handling of user permissions1,32 Reliability Range of answers: 1: Very Low2: Modest3: Fairly Good 4: Good5: Very Good6: Excellent Questions:Score: Building software4,05 Editing software metadata 2,61 Building software remotely 2,10 USAGE Frequency The reliability of the CLIs are between fairly good and good for local building, whilst it is fairly good for remote building and editing functionalities.

14. INFSOM-RI-026753 EGEE and gLite are registered trademarks Future Plans 14 ETICS 2nd EU Review - CERN - 15 February 2008 To port the client to other common platforms (e.g. Windows, where some work has already been done) To improve the performance and the scalability of the system – We have the more and more projects and users using the system To enrich and optimize the metadata schema – Requirements change and evolve over the time – The two years of experience with the system has taught us where and how to improve it Co-scheduling testing – This work is partially done – This is an innovative feature

15. INFSOM-RI-026753 EGEE and gLite are registered trademarks Future Plans 15 ETICS 2nd EU Review - CERN - 15 February 2008 Some new features – during the first two years had a lower priority but are becoming more and more demanded by users – Scheduling functionality (users want to program when their builds/tests should be executed) – Enriching the number and variety of plugins – Introducing more flexibility in the way users edit information

16. INFSOM-RI-026753 EGEE and gLite are registered trademarks ETICS 2nd EU Review - CERN - 15 February 2008 16 Thanks http://www.eu-etics.org