1. EMI SA2: Quality Assurance (EMI-SA2 Work Package) Alberto Aimar (CERN) WP Leader

2. Outline EMI SA2 Summary Objectives and Tasks Deliverables and Milestones SA2 Setup Organization SA2 and the other EMI WPs Services and Tools Next Steps Issues and concerns Plans for Q1st quarter, deliverables, milestones, events/meetings to organise/ participate 27/05/2010 EMI SA2: Quality Assurance 2

3. SA2 Partners CERN - European Organization Nuclear Research CESNET - Czech Republic NREN CINECA - Consortium Italian Univ, CNR, Min.Research JUELICH - FZJ Supercomputing Centre, Jülich GRNET - Greek Research and Technology Network INFN - Istituto Nazionale di Fisica Nucleare TCD - Trinity College, Dublin UPJS - Pavol Jozef Safarik University, Kosice 27/05/2010 EMI SA2: Quality Assurance 5

4. EMI SA2 Objectives 1.Define and establish a common software quality assurance process and metrics for all engineering activities 2.Allow the EMI middleware to consistently pass the customer acceptance criteria and continually improve the software quality and the process itself 3.Monitoring metrics value trends, reviewing quality control activities and related tests, providing support and consultancy in QA matters 4.Enable a continuous integration and testing process by selecting and maintaining tools and resources for building and testing software either within the project or in collaboration with external resource providers 27/05/2010 EMI SA2: Quality Assurance 4

5. SA2 Tasks SA2.1 – Work Package coordination (Task leader: CERN, all partners participate to the meetings and reviews) This task deals with the regular coordination of the Work Package, reporting and review of milestones and deliverables. SA2.2 – Quality Assurance Process Definition and Monitoring (Task leader: CERN. Participants: CINECA, INFN, UPJS) Definition of a standards-compliant software engineering process Continual activity of monitoring its correct application within the activities of the EMI project. Success criteria of this task are the availability of an agreed, documented and regularly updated process and the minimization of deviation in its application by the project members 27/05/2010 EMI SA2: Quality Assurance 6

6. SA2 Tasks SA2.3 – Metrics and KPIs Definition and Reporting (Task leader: CERN. Participants: TCD) Definition and continual collection and reporting of software quality metrics according to suitable QA models. Provides information to the PEB and other project bodies on status of software as an instrument for corrective actions. Success criteria are the regular production of reports and their use to identify improvement areas. SA2.4 – Tools and Repositories Selection, Maintenance and Integration (Leader: CERN. Participants: CINECA, GRNET, INFN, UPJS) Definition and when necessary maintenance of tools required to support the QA process. Support software providers to integrate required information to and from tools maintained outside the EMI project. Setup and maintenance of repositories for storing EMI software packages, tests, build and test reports and metrics generated during all software development activities within EMI. 27/05/2010 EMI SA2: Quality Assurance 7

7. SA2 Tasks SA2.5 – QA Implementation Review and Support (Task leader: UPJS. Participants: CERN, INFN, TCD, UPJS) Review activities of QA, test and certification implementations done by the Product Teams Sample review of test plans and tests, compliance with packaging, porting guidelines, validation of documentation, etc. Support Product Teams in effective design and implementation of tests to be used with testing tools such as ETICS. Success criterion for is the usage of tools and procedures by all project members to be measured by surveys and verifications SA2.6 – Testbeds Setup, Maintenance and Coordination (Task leader: INFN. Participants: CERN, CESNET, FZJ) Setup and maintenance of distributed testbeds for the project continuous integration and testing operations Coordination and provision of larger-scale testbeds from collaborating resource providers. Success criteria for this task are the availability and reliability metrics of the execution nodes. 27/05/20108

8. SA2 Deliverables DSA2.1 - Quality Assurance Plan: Definition of the global software QA processes, procedures, roles and responsibility and the related metrics and measurement methodologies. [M1] DSA2.2.1 - QA Tools Documentation: Describes the software engineering tools and the repository management systems provided by SA2 to EMI and third-party users. This document is updated and revised regularly [M2] DSA2.4 - Continuous Integration and Certification Testbeds: Describes the distributed certification testsbeds for internal and acceptance certification and its access and usage requirements [M3] DSA2.2.2 - QA Tools Documentation: Describes the software engineering tools and the repository management systems provided by SA2 to EMI and third-party users. Updated and revised regularly [M10] DSA2.2.3 - QA Tools Documentation: Describes the software engineering tools and the repository management systems provided by SA2 to EMI and third-party users. [M22] 27/05/2010 EMI SA2: Quality Assurance 9

9. SA2 Deliverables Periodic QA Report DSA2.3.1 - Periodic QA Reports: A report on the compliance with and results of the quality assurance process is produced monthly by the activity manager and is reported every twelve months to the EC [M3] DSA2.3.2 - Periodic QA Reports: A report on the compliance with and results of the quality assurance process is produced monthly by the activity manager and is reported every twelve months to the EC [M12] DSA2.3.3 - Periodic QA Reports: A report on the compliance with and results of the quality assurance process is produced monthly by the activity manager and is reported every twelve months to the EC [M24] DSA2.3.4 - Periodic QA Reports: A report on the compliance with and results of the quality assurance process is produced monthly by the activity manager and is reported every twelve months to the EC [M36] 27/05/2010 EMI SA2: Quality Assurance 10

10. SA2 Milestones 27/05/2010 EMI SA2: Quality Assurance 11

11. 27/05/2010 EMI SA2: Quality Assurance 12 SA2 KPIs 99%

12. 27/05/2010 EMI SA2: Quality Assurance 13 SA2 KPIs

13. 27/05/2010 EMI SA2: Quality Assurance 14 SA2 KPIs

14. SA2 Internal SA2 Internal Wiki - Repository of the WP Nothing secret, but is not linked from outside, could be made public Internal documents, work in progress, meeting minutes, etc Mailing lists (all and by service contacts) Bug tracker, project management tool ITIL for Services, SCRUM for development, ISO for QA Weekly Meetings (Wednesday 10:30?) Prepared in advance with a short summary of progress and issue Minutes available on the meeting page CERN audio conf. Everybody should participate will be a phone meeting Action List specific to EMI SA2 Maintained in the Wiki? Indico category for EMI SA2 meetings ( or what will be set by NAx) 27/05/2010 EMI SA2: Quality Assurance 17

15. 27/05/2010 EMI SA2: Quality Assurance 19 SA2 and Other WPs

16. Many Topics FOR THE PRODUCT TEAMS Repository Platforms, virtual images Testbed(s) and worker nodes Build tools Tests tools Development tools and metrics Bug tracking, tickets systems Documentation tools QA metrics (ISO 9128) QA Tools QA processes Other Infrastructure (?) Processes and procedures 27/05/2010 EMI SA2: Quality Assurance 20 SLA Agreements, acceptance criteria Reports and Reviews Monitoring Releases IN ADDITION FOR SA2 Planning, mgmt Organisation Wiki internal Public WIki Users Support User feedback EGI contact, other customers Contacts, discussion with PTs Deliverables and Milestones

17. Q1 Plans Topics FOR THE PRODUCT TEAMS Repository Platforms, virtual images Testbed(s) and worker nodes Build tools Tests tools Development tools and metrics Bug tracking, tickets systems Documentation tools QA metrics QA Tools QA processes Infrastructure Processes and procedures 27/05/2010 EMI SA2: Quality Assurance 20 SLA Agreements, acceptance criteria Reports and Reviews Monitoring Releases IN ADDITION FOR SA2 Planning, mgmt Organization Wiki internal Public WIki Users Support User feedback EGI contact, other customers Contacts, discussion with PTs Deliverables and Milestones DSA2.1, DSA2.2, MSA2.1, MSA2.2

18. Q1 Plans Deliverables and Milestones DSA2.1 - Quality Assurance Plan: Definition of the global software QA processes, procedures, roles and responsibility and the related metrics and measurement methodologies. [M1] DSA2.2.1 - QA Tools Documentation: Describes the software engineering tools and the repository management systems provided by SA2 to EMI and third-party users. This document is updated and revised regularly [M2] 27/05/2010 EMI SA2: Quality Assurance 20

19. Q1 Plans Next Steps Discuss priorities (topic by topic) Setup infrastructure (wiki, tools) Assign responsibilities within SA2 Prepare plans for each SA2 Services Prepare a Service Catalogue Provide initial tools and services ASAP Provide initial documentation ASAP Constant interaction with PTs for tools and services  Need contacts with the PTs (via area QA representatives)  Via the SA2 members near the PTs (ARC, gLite, UNICORE)  Start ASAP, phone conf on specific topics (SA2, then Area reps) 27/05/2010 EMI SA2: Quality Assurance 20

20. Thank you 27/05/201021 EMI SA2: Quality Assurance