INFSOM-RI WP3 Software Configuration Tools and Methodologies Elisabetta Ronchieri (INFN CNAF) ETICS Final Review (CERN) 15 February 2008

INFSOM-RI Contents GoalsGoals AchievementsAchievements ChallengesChallenges Metrics and StatisticsMetrics and Statistics ConclusionsConclusions ETICS Final Review - WP3 - CERN, 15 February

INFSOM-RI Goals ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Goals To provide a configuration, build and integration system based on existing or adapted tools able to describe build/integration tasks to be executed by the ETICS facilityTo provide a configuration, build and integration system based on existing or adapted tools able to describe build/integration tasks to be executed by the ETICS facility To provide a configuration information schema able to capture the configuration information of software developed by research projects and allowing coherent storage of such information in the ETICS databaseTo provide a configuration information schema able to capture the configuration information of software developed by research projects and allowing coherent storage of such information in the ETICS database To collaborate with other WP in the project in the deployment of tools, definition and integration of tests in the build process and definition and generation of interoperability and quality reportsTo collaborate with other WP in the project in the deployment of tools, definition and integration of tests in the build process and definition and generation of interoperability and quality reports 4 ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Achievements ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI ETICS CLIs, Build/Test Web Service and metadata ETICS Portal (myETICS, Build/Test, Repository, Administration) Report DB Metronome Execution Engine Metronome Client Wrapper WNs ETICS Infrastructure Repository Service Build/Test Artefacts Project DB Clients Web Service Via browser ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Software Metadata Definition Several software projects that use different languages, build and test procedures, libraries and work on different platforms, are registered in the ETICS system adopting the same procedure Several software projects that use different languages, build and test procedures, libraries and work on different platforms, are registered in the ETICS system adopting the same procedure WP3 has designed a software metadata model based on Common Information Model (CIM), characterized by formal entities able to express software project structure, build/test/checkout configurations, security information, dependencies, environment variables, properties, users, rolesWP3 has designed a software metadata model based on Common Information Model (CIM), characterized by formal entities able to express software project structure, build/test/checkout configurations, security information, dependencies, environment variables, properties, users, roles –E.g., software project structure and configuration structure The use of the same data model allows software managers to easily collect software information and instrument the Software Quality Assurance processThe use of the same data model allows software managers to easily collect software information and instrument the Software Quality Assurance process 7 ETICS Final Review - WP3 - CERN, 15 February 2008 Project Subsystem Component... Subsystem... Component..... Configuration Platform 1 VCS Command Build Command Test Command Dependency1 Property1 Environment1 Platform n VCS Command Sub-Configuration1 Project, Subsystem and ComponentProject, Subsystem and Component More than one Configurations is allowed for each ModuleMore than one Configurations is allowed for each Module More than one Platforms is allowed for each ConfigurationMore than one Platforms is allowed for each Configuration Set Sub-Configuration in Project and Subsystem ConfigurationsSet Sub-Configuration in Project and Subsystem Configurations

INFSOM-RI Build/Test Process (1) Several software projects are built/tested in the ETICS system by using APIs able to support different Version Control Systems and compilation tools, to detect dependencies automatically, to integrate software maintained on different repositories, and to handle reproducible builds and tests creating reports and artefacts (e.g., RPMS, MSI, Debian, tarballs ) Several software projects are built/tested in the ETICS system by using APIs able to support different Version Control Systems and compilation tools, to detect dependencies automatically, to integrate software maintained on different repositories, and to handle reproducible builds and tests creating reports and artefacts (e.g., RPMS, MSI, Debian, tarballs ) WP3 has provided tools allowing users toWP3 has provided tools allowing users to –download source code, binary code and metadata information –browse software information of different projects –edit software metadata in order to provide the integration of the ETICS mechanisms together with users’ internal procedures adopted e.g. during the building/testing of code –freeze a particular version of code and metadata Users and the ETICS execution engine use the same tools for building/testing softwareUsers and the ETICS execution engine use the same tools for building/testing software 8 ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Build/Test Process (2) Users with different backgound (e.g., Windows and Linux) may use one of the ETICS clients to execute local and remote operations, and multi-node tests using the same framework Users with different backgound (e.g., Windows and Linux) may use one of the ETICS clients to execute local and remote operations, and multi-node tests using the same framework For multi-node tests, users can include in the software projects entities related to tests thanks to the flexibility of the software metadata:For multi-node tests, users can include in the software projects entities related to tests thanks to the flexibility of the software metadata: –DT  {node1, node2, …, nodeN} –Set Sub-configurations in DT Configuration –Node  {service1, service2, …, serviceN} –Set services in Node Configuration 9 Project DestrebutedTest Node... Subsystem... Component..... ETICS Final EU Review - CERN - 15 February ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Secure Access Users access the ETICS system in secure mode. WP3 has provided:WP3 has provided: –a service and tools able to authenticate users by adopting standard x.509 certificates and to authorize users based on role definition –i.e., defined roles to descriminate the operations users can perform –e.g., clients connecting without certificate are mapped as Guest –allowed or denied operations to the users by Web Service –a tool to register information of users (e.g., name and distinguish name) in the ETICS data model 10 RoleDescription Administrator Super user Module Administrator Handles individual projects Developer runs remote builds Integrator runs remote builds, registers artefacts in the repository Tester runs remote tests, stores results in the report repository Release Manager Defines configurations at project level, and project releases Guest Read-only access ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Interoperability and Compliance to Standards The configuration information stored in the ETICS database is used to provide comparative interoperability information using data such as standards versions, libraries versions, supported platforms, packaging formats and deployment properties. The configuration information stored in the ETICS database is used to provide comparative interoperability information using data such as standards versions, libraries versions, supported platforms, packaging formats and deployment properties. WP3 has identified four architecture interoperability metrics [1] easily applicable to software registered in the ETICS system for Quality Assurance measurementsWP3 has identified four architecture interoperability metrics [1] easily applicable to software registered in the ETICS system for Quality Assurance measurements –Number of 3rd Party Components (No3C) –Number of Generic Components (NoGC) –Number of Architectural Revisions (NoAR) –Number of Versions (NoV) In addition, work on BES interoperability testing has been done in collaboration with OMII-Europe In addition, work on BES interoperability testing has been done in collaboration with OMII-Europe WP3 has provided information collectors for the reporting engineWP3 has provided information collectors for the reporting engine –Compliance to standards (I Pv6, WSI-compliance) –Interoperability ( Architecture interoperabilty) [1] [1] S. Kalyanasundaram, K. Ponnambaiam, A. Singh, ‘Metrics for Software Architecture: A Case Study in the Telecommunication Domain’, 1998 IEEE 11 ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Challenges ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Performance and Scalability Due to the large number of ETICS users and the complexity of projects (e.g. gLite and gCube), keeping an adequate response time is a challengeDue to the large number of ETICS users and the complexity of projects (e.g. gLite and gCube), keeping an adequate response time is a challenge Work keeps being done to make sure ETICS provides an efficient service: Server-side performanceServer-side performance –Improve the data model in order to guarantee a better response time from the backend –CIM seems not to scale to very large software applications –Started a revision of the data model Client-side performanceClient-side performance –Current version based on CIM and SOAP –As the size of a project increases, the performance of the client decreases to handle large XML documents –Substantially performance improvements by using newer version of the Python SOAP engine (called ZSI) in handling large XML documents –Started tests in the ETICS client Improve the calculation of the configuration propertiesImprove the calculation of the configuration properties –Substantially performance improvements by making better use of caching 13 ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Feature Requests Vs. Implementation With more users, larger number of requirementsWith more users, larger number of requirements –Difficult to strike the right balance between available resources and user requests for new features –Developers –Integrators –Managers –Software porters (to a wide range of resources) –Deployment and system testers We have coped with a much larger number of releases than expected, in order to provide the requested functionality, while protecting qualityWe have coped with a much larger number of releases than expected, in order to provide the requested functionality, while protecting quality ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Metrics and Statistics ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Deliverables Deliverables: Milestones: 16 ETICS Final Review - WP3 - CERN, 15 February 2008 Del. N. Del. Name Delivery Date D3.1 Configuration, build and integration specifications Configuration, build and integration specifications PM03 D3.2 Interoperability reports specifications, configuration, build and integration system Interoperability reports specifications, configuration, build and integration system PM12 D3.3 Final evaluation report and configuration, build and integration system (final release) PM20 Mil. N. Mil. Description Delivery Date M3.1 Configuration, build and integration system deployed in the ETICS resource pools (prototype) PM12 M3.2 Configuration, build and integration system deployed in the ETICS resource pools (final release) PM22

INFSOM-RI Dissemination Collaborations in papers: 17 Year Papers and Posters 2008 [1] S. Da Ronco, G. Diez-Andino Sancho, L. Dini, E. Ronchieri, and M. Selmi, “Integrated Solutions For Information Sharing In Health Care Applications”, In Proceedings of 41° Hawaii International Information Sharing In Health Care Applications”, In Proceedings of 41° Hawaii International Conference on System Science (HICSS-41), 7-10 January, Waikoloa, Big Island, HI, USA, IEEE Conference on System Science (HICSS-41), 7-10 January, Waikoloa, Big Island, HI, USA, IEEE Computer Society. Computer Society [2] A. Di Meglio, M.-E. Bégin, P. Couvares, E. Ronchieri, E. Takacs, “ETICS: the international software engineering service for the Grid”, In Proceedings of CHEP’07, 2-7 September 2007, Victoria, British Columbia, Canada. [3] M.-E. Bégin, et al., “Analysis of Requirements for Automated Interoperability Testing”, In Proceedings of the Integrated Design And Processing Tehcnology, IDPT 2007, 3–8 June 2007, Antalya, Turkey. [4] M.-E. Bégin, et al., “Build, Configuration, Integration and Testing Tools for Large Software Projects: ETICS”, In N. Guelfi and D. Buchs (Eds.), Proceedings of the 3rd International Workshop on Rapid Integration of Software Engineering techniques, RISE 2006, LNCS 4401, © Copyright Springer-Verlag 2007, [4] M.-E. Bégin, et al., “Build, Configuration, Integration and Testing Tools for Large Software Projects: ETICS”, In N. Guelfi and D. Buchs (Eds.), Proceedings of the 3rd International Workshop on Rapid Integration of Software Engineering techniques, RISE 2006, LNCS 4401, © Copyright Springer-Verlag 2007, [5] S. Da Ronco, G. Diez-Andino Sancho, M. Gentilini, E. Ronchieri, and M. Selmi, “ETICS Meta-Data Software Editing - From Check Out To Commit Operations”, In Proceedings of CHEP’07, 2-7 September 2007, Victoria, British Columbia, Canada. [5] S. Da Ronco, G. Diez-Andino Sancho, M. Gentilini, E. Ronchieri, and M. Selmi, “ETICS Meta-Data Software Editing - From Check Out To Commit Operations”, In Proceedings of CHEP’07, 2-7 September 2007, Victoria, British Columbia, Canada [6] M.-E. Bégin, et al., “Build, Configuration, Integration and Testing Tools for Large Software Projects: ETICS”, In Proceedings of the 3rd International Workshop on RISE 2006, Projects: ETICS”, In Proceedings of the 3rd International Workshop on RISE 2006, September 2006, Geneva, Switzerland, Univerity of Geneva. September 2006, Geneva, Switzerland, Univerity of Geneva. [7] “ETICS – eInfrastructure for Testing, Integration and Configuration of Software”, at V INFN Grid Workshop, December 18-20, 2006, Padua, Italy Workshop, December 18-20, 2006, Padua, Italy 17 ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Measurements Supported Platforms 41 –Fedora Core 4 –Scientific Linux 3 –Scientific Linux Cern 5 –Red Had Enterprise Linux 5 –SUSE 5 –Solaris 1 –Debian 4 –centOS 5 –Darwin/Mac OS 4 –win32 1 –aix 1 –ubuntu 3 Supported platforms 18 OSVersionArchitectureCompiler Fedora Core 3ia32 gcc ia32 gcc ia32, x86_64 gcc Scientific Linux 3 ia32, x86_64 gcc Ia32 gcc Scientific Linux Cern 3 ia32, x86_64 gcc 3.2.3, gcc ia32, ia64, x86_64 gcc Red Hat Enterprise Linux 3ia32 gcc ia32, x86_64 gcc [3.4.3, 3.4.6], gcc [3.4.5, 3.4.6] SUSE9 ia32, x86_64 gcc [3.3.1, 3.3.5], gcc ia32 gcc [4.0.2, 4.1.0] Solaris10ia32 gcc Debian3 ia32, x86_64 gcc ia32, x86_64 gcc centOS4 ia32, x86_64 gcc [3.4.4, 3.4.6], gcc ia32, x86_64 gcc 4.1.2, gcc Darwin/Mac OS 772powerpc gcc powerpc gcc ia powerpc gcc ubuntu5Ia32 gcc Ia32 gcc ia32 gcc Win3271 aix52powerpc gcc ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI User support and service bugs Metrics: –Supported languages 7 –C, C++, Python, Java, Perl, sh, csh –Supported Requirements 97.56% D3.1 D3.1 –Total number of open and closed bugs over time –Ratio of closed/open bug over time RatioYear ETICS Final Review - WP3 - CERN, 15 February 2008 Users, projects and requirements have greatly increased with time, but not the size of the team! Bugs reported for 2006 started in April

INFSOM-RI Conclusions ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI Conclusions Write once, build/test everywhere!Write once, build/test everywhere! –providing a unique information framework for an heterogeneous pool of developers with their own dedicated building/testing procedures WP3 has achieved the goal of providing consistent builds and tests on multiple platforms in trasparent way for the end-usersWP3 has achieved the goal of providing consistent builds and tests on multiple platforms in trasparent way for the end-users –adopting specific service and tools –adopting intuitive instruments –guaranteeing the reproducibility of builds and tests 21 ETICS Final Review - WP3 - CERN, 15 February 2008

INFSOM-RI ETICS Final Review - WP3 - CERN, 15 February Thanks