INFSOM-RI Elisabetta Ronchieri INFN CNAF ETICS 2 nd EU Review (CERN) 15 February 2008 WP3 - Software Configuration Tools and Methodologies

INFSOM-RI EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

INFSOM-RI EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

INFSOM-RI EGEE and gLite are registered trademarks Goals ETICS 2nd EU Review - CERN - 15 February G1. Provide a configuration, build and integration system able to describe software tasks to be executed by the ETICS facility. G2. Provide a configuration information schema able to capture the configuration information of software developed by software project and a coherent storage of such information in the ETICS database

INFSOM-RI EGEE and gLite are registered trademarks Goals ETICS 2nd EU Review - CERN - 15 February G3. Web Server: the heart of the ETICS system –Provide a logic for the entire ETICS system –Handle access to the ETICS system in secure mode –Handle metadata, artefacts, and user requests –Perform complex queries on behalf of the clients –Handle build/test remotely –Support build/test locally G4. Command Line Interfaces (CLIs): one of the ETICS clients –Allow users to register their software, specify configuration information (i.e., edit metadata) –Handle access to Web Service in secure mode –Checkout code and metadata –Build/Test code locally and remotely –Tag code and metadata –Produce reports and different format of artefacts

INFSOM-RI EGEE and gLite are registered trademarks ETICS CLIs, Build/Test Web Serivce and metadata ETICS Portal (myETICS, Build/Test, Repository, Administration) Report DB Metronome Execution Engine Metronome Client Wrapper WNs ETICS Infrastructure Repository Service Build/Test Artefacts Project DB Clients Web Service Via browser

INFSOM-RI EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

INFSOM-RI EGEE and gLite are registered trademarks Achievements 8 ETICS 2nd EU Review - CERN - 15 February 2008 A1. Authentication and Authorization Performed authentication using standard x.509 certificates in Web Service and CLIs Registered information of users (e.g., name and distinguish name) in the ETICS data model Based authorization on role definition –i.e., defined roles to descriminate the operations users can perform –e.g., clients connecting without certificate are mapped as Guest Allowed or denied operations to the users by Web Service RoleDescription AdministratorSuper user Module AdministratorHandles individual projects Developerruns remote builds Integratorruns remote builds, registers artefacts in the repository Testerruns remote tests, stores results in the report repository Release ManagerDefines configurations at project level, and project releases GuestRead-only access

INFSOM-RI EGEE and gLite are registered trademarks Achievements 9 ETICS 2nd EU Review - CERN - 15 February 2008 A2. Software Metadata definition Based schema version on Common Information Model (CIM) Defined formal entities to express project structure, build/test/checkout configurations, security information, dependencies, environment variables, properties, users, roles –E.g., software project structure and configuration structure –Project, Subsystem and Component referred as Modules –More than one Configurations is allowed for each Module –More than one Platforms is allowed for each Configuration –Set Hierarchy in Project and Subsystem Configurations Project Subsystem Component... Subsystem... Component..... Configuration Platform 1 VCS Command Build Command Test Command Dependency1 Property1 Environment1 Platform n VCS Command Hierarchy1

INFSOM-RI EGEE and gLite are registered trademarks Achievements 10 ETICS 2nd EU Review - CERN - 15 February 2008 A3. Build/Test Handled the downloading of code and metadata information Supported the build and test locally and remotely Generated platform specific artefacts (e.g., RPMS, MSI, Debian, tarballs) during the build of software Generated build and test reports Supported the browsing of software information Supported the editing of software metadata –e.g., the possibility to create metadata, modify/remove them and copy existing metadata Supported co-scheduling by the editing functionality –DT  {node1, node2, …, nodeN} –Set hierarchy in DT configuration –Node  {service1, service2, …, serviceN} –Set services in Node configuration Handled code and metadata tag Project DestrebutedTest Node... Subsystem... Component.....

INFSOM-RI EGEE and gLite are registered trademarks Achievements 11 ETICS 2nd EU Review - CERN - 15 February 2008 A4. Interoperability and Compliance to Standards Metrics –Identified four interoperability metrics 1. Number of 3rd Party Components (No3C) 2. Number of Generic Components (NoGC) 3. Number of Architectural Revisions (NoAR) 4. Number of Versions (NoV) –Used configuration information stored in the ETICS database Collectors –Provided information collectors for the reporting engine –Compliance to standards: Ipv6 WS-compliance –Interoperability Metrics

INFSOM-RI EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

INFSOM-RI EGEE and gLite are registered trademarks Challenges ETICS 2nd EU Review - CERN - 15 February C1. Improve the data model in order to guarantee a better response time from the backend –CIM does not scale to large software applications –Started a revision of the data model C2. Improve the CLI performances –Current version based on CIM and SOAP –As the size of a project increases, the performance of the client decreases to handle large XML documents –Substantially performance improvements by using newer version of the Python SOAP engine (called ZSI) in handling large XML documents –Started tests in the ETICS client C3. Improve the calculation of the configuration properties –Substantially performance improvements by making better use of caching

INFSOM-RI EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

INFSOM-RI EGEE and gLite are registered trademarks WP3 in Numbers 15 ETICS 2nd EU Review - CERN - 15 February 2008 Deliverables: Milestones: Del. N.Del. NameDelivery Date D3.1Configuration, build and integration specificationsPM03 D3.2Interoperability reports specifications, configuration, build and integration systemPM12 D3.3Final evaluation report and configuration, build and integration system (final release)PM20 Mil. N.Mil. DescriptionDelivery Date M3.1Configuration, build and integration system deployed in the ETICS resource pools (prototype)PM12 M3.2Configuration, build and integration system deployed in the ETICS resource pools (final release)PM22

INFSOM-RI EGEE and gLite are registered trademarks WP3 in Numbers 16 ETICS 2nd EU Review - CERN - 15 February 2008 OSVersionArchitectureCompiler Fedora Core3ia32gcc ia32gcc ia32, x86_64gcc Scientific Linux3ia32, x86_64gcc Ia32gcc Scientific Linux Cern3ia32, x86_64gcc 3.2.3, gcc ia32, ia64, x86_64gcc Red Hat Enterprise Linux3ia32gcc ia32, x86_64gcc [3.4.3, 3.4.6], gcc [3.4.5, 3.4.6] SUSE9ia32, x86_64gcc [3.3.1, 3.3.5], gcc ia32gcc [4.0.2, 4.1.0] Solaris10ia32gcc Debian3ia32, x86_64gcc ia32, x86_64gcc centOS4ia32, x86_64gcc [3.4.4, 3.4.6], gcc ia32, x86_64gcc 4.1.2, gcc Darwin/Mac OS772powerpcgcc powerpcgcc ia32gcc powerpcgcc ubuntu5Ia32gcc Ia32gcc ia32gcc Win3271 aix52powerpcgcc Measurements Supported Platforms 41Supported Platforms 41 –Fedora Core 4 –Scientific Linux 3 –Scientific Linux Cern 5 –Red Had Enterprise Linux 5 –SUSE 5 –Solaris 1 –Debian 4 –centOS 5 –Darwin/Mac OS 4 –win32 1 –aix 1 –ubuntu 3

INFSOM-RI EGEE and gLite are registered trademarks WP3 in Numbers 17 ETICS 2nd EU Review - CERN - 15 February 2008 Measurements: –Supported languages 7 –C, C++, Python, Java, Perl, sh, csh –Implemented Requirements 97.56% D3.1 D3.1 –Total number of open and closed bugs over time –Ratio of closed/open bug over time RatioYear

INFSOM-RI EGEE and gLite are registered trademarks Contents ETICS 2nd EU Review - CERN - 15 February GoalsGoals AchievementsAchievements ChallengesChallenges WP3 in NumbersWP3 in Numbers ConclusionsConclusions

INFSOM-RI EGEE and gLite are registered trademarks Conclusions 19 ETICS 2nd EU Review - CERN - 15 February 2008 Most important achievements A1. Authorization and Authentication A2. Software Metadata Definition A3. Build/Test A4. Interoperability and compliance to standards Three outstanding challenges C1. Data Model Structure C2. CLI Performances C3. CLI Property Calculation Planned in ETICS II activities to address the challenges

INFSOM-RI EGEE and gLite are registered trademarks ETICS 2nd EU Review - CERN - 15 February Thanks