EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI WP5 Review Michel Drescher EGI.eu SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Activity Overview This slide will be provided by the PO It will summarise the activity in tables by: –The # partners, # people, # countries –The # PM and #FTE per country It will summarise the activity in graphics by: –The % effort of the activity within the project –The geographical spread across Europe SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI WP5 Objectives SA2 – Michel Drescher - EGI-InSPIRE EC Review #Objective O1Establish agreements with key software providers O2Maintain the UMD Roadmap O3Define general & component-specific quality criteria O4Verify the software against these criteria O5Provide a repository for the software components within UMD, and the related support tools O6Provide a distributed support unit within the EGI Helpdesk infrastructure with expertise on the deployed middleware in production use Taken from the Description of Work

EGI-InSPIRE RI Tasks SA2 – Michel Drescher - EGI-InSPIRE EC Review TaskObjectiveDescription TSA2.1O1, O2 Manage and coordinate the activity. Manage the relationships with external Technology Providers. Collate and distribute prioritised requirements Maintain UMD Roadmap and release schedules Coordinate and chair meetings with Technology Providers for resolution of issues and conflicts. TSA2.2O3 Develop, maintain and publicise: -Generic Acceptance Criteria -Component-specific Acceptance Criteria TSA2.3O4 Validate contributed software against Acceptance Criteria Provide component acceptance reports summarising the verification TSA2.4O5 Provide the infrastructure for EGI.eu Provide the infrastructure for the Software Provisioning process TSA2.5O6 Establish and maintain a second level support unit (the “DMSU”) Provide consultancy to EGI Operations and Technology Providers

EGI-InSPIRE RI EGI Software Supply Chain EGI Software Supply Chain We set out to 1.Drive and steer the evolution of middleware –Based on installed software –Prioritise collected requirements –Provide UMD Roadmap updates –Publish EGI Software Quality Criteria SA2 – Michel Drescher - EGI-InSPIRE EC Review TCB OMB UCB Production Infrastructure Production Infrastructure D5.1, D5.2 MS505 D5.1, D5.2 MS505

EGI-InSPIRE RI We set out to 2.Establish a software supply chain model –Technology Providers as suppliers –Verify deliveries, assemble a unified middleware (UMD) –Resource centres as customers SA2 – Michel Drescher - EGI-InSPIRE EC Review Technology Providers Resource Centres Repository Acceptance Testing Acceptance Testing Staged Rollout EGI Software Supply Chain MS503 MS501, MS504

EGI-InSPIRE RI We set out to 3.Provide operational expertise on deployed middleware –Integrated as 2 nd line support in EGI Helpdesk –Develop best practices on service configuration –Recommendations on strategic middleware development SA2 – Michel Drescher - EGI-InSPIRE EC Review Fancy diagram missing MS502

EGI-InSPIRE RI Defining business Processes –What are the triggers and terminal conditions? –Who is responsible for which step? Interactions –Who/what are the triggering actors processes? –Which information is passed-in, and out? Artefacts –What needs documentation? –Where is it available? SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Processes Middleware evolution –Requirements capturing and processing –UMD Roadmap maintenance and publication –Technology Provider release schedule coordination –Establishing relationships with Technology Providers EGI Software supply chain –Quality Criteria maintenance –Acceptance testing (Criteria verification) –Middleware helpdesk processes –Software Provisioning process and infrastructure SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Interactions With the User communities: –In the TCB through prioritised requirements With the Operations community –In the TCB through prioritised requirements –Via the SVG/RAT for security vulnerabilities –Via the DMSU for software incidents and problems –With Resource Centres using the Software Repository With Technology Providers –In the TCB for new requirements –In Task Forces for Software Quality, and Repositories SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Artefacts Middleware evolution –UMD Roadmap –Standards evolution and roadmap –MoUs and SLAs with Technology Providers Software Supply Chain –UMD Release Schedule –Quality Criteria documentation –Verification reports, and guidelines –StagedRollout reports –Software Provisioning tracking artefacts SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Doing business Implementing processes, artefacts and interactions Validate feasibility with small, controllable scope –With internal Technology Providers EGI Trust Anchors, SAM (JRA1) Establish and maintain appliance delivery model –With external Technology Providers (EMI) Re-scope and implement distribution based supply chain Verify principal supply chain with two dry runs Scale out to full enactment of the supply chain Review regularly, and implement adjustments SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Reviewing Objective O1 Establish agreements with key software providers MoUs negotiated in collaboration with EGI Policy team SLA negotiation, goal, and template defined in MS505 –Defines processes, Interactions and Artefacts –Scope of the service (Delivery, Quality Assurance, Incident Management, Vulnerability Management) –Provider performance (Metrics & Objectives) –Problem management & Termination of Agreement MoUs agreed and signed with: –EMI, IGE, SAGA, StratusLab SLAs negotiated and signed with: –EMI, IGE, SAGA SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Reviewing Objective O2 Maintain the UMD Roadmap UMD Roadmap available in two revisions: D5.1, D5.2 –Also outlines Requirements engineering process Key Capabilities covering nine functional areas –Security, Information, Storage, Data, Compute, Operations, Virtualisation, Remote Instrumentation, Clients –Capability dependencies define functional composition of a Grid SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Reviewing Objective O3 Define general & component-specific quality criteria General Criteria apply to all delivered software Specific Criteria structured along UMD Roadmap Capabilities –Applies to all software that implements that Capability –Promotes fair, and impartial verification Process follows UMD Roadmap publication schedule –One, versioned, document set is in force –Many historical version provided for reference –Drafts publicised for review before put into force Release information available at: SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Reviewing Objective O4 Verify delivered software against the EGI Quality Criteria Uses the Software Provisioning infrastructure Scope of verification effort and focus defined in MS503 –Provides specific instructions and templates for verifiers Produces verification reports for each verification –Targets suppliers and customers: Technology Providers and Resource Centres –Targets software provisioning supply chain participants: Quality Criteria definition task (TSA2.2), DMSU (TSA2.5), StagedRollout sites (see TSA1.3) SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Reviewing Objective O5 Provide a repository for the software components within UMD, and the related support tools Software Provisioning infrastructure and technical process defined in MS501, MS504 Provide the UMD Repository and EGI.eu IT infrastructure SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Reviewing Objective O6 Provide a distributed support unit within the EGI Helpdesk infrastructure with expertise on the deployed middleware in production use Scope of the DMSU processes, interactions and main artefacts (tickets) defined in MS502 –User-generated tickets are assigned via the TPM (TSA1.???) –May assign tickets to Technology Provider 3 rd level support units Provides Consultancy –EGI Operations –Technology Providers SA2 – Michel Drescher - EGI-InSPIRE EC Review Ticket statistics graph for DMSU missing

EGI-InSPIRE RI WP5 impact and value Provide open, impartial software evolution plans Provision software for a large community –Avoid duplicated, per-site software assessment –Reduced overall effort for increased sustainability Provide a unified middleware repository –One-stop shop solution in a multi-provider, multi-use case infrastructure –“Off-the-shelf” experience for site administrators Quality in delivery depends on stakeholders SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Issues in PY1 1.Requirements reported as support requests –Rerouted to TCB’s Requirements capturing process 2.Low ratio of ticket resolution in DMSU –Further analysis of tickets on-going 3.Staffing –Target staffing reached in January 2011 SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Issues in PY1 4.Criteria Verification not well understood –Task Force Quality Assurance set up –Introducing concepts of formal Acceptance Testing –Continuous education of Technology providers 5.Lack of interaction and pro-activeness of EMI –Task Force Repositories and Automation –Discuss technical issues around software delivery –Large discrepancy between software supply models –On-going, time consuming activity SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Use of Resources SA2 – Michel Drescher - EGI-InSPIRE EC Review TaskTask PY1PartnerPY1Q1Q2Q3Q4 TSA %1 – EGI.eu90%20%116%110%116%Delayed employment, compensating TSA %12A – CSIC76%44%84%99%79% 29 – LIP85%0%36%173%133%Delayed employment, compensating TSA %12A – CSIC148%50%196%192%153%Need to understand this before review! 12B – FCTSG92%0%13%112%245%Compensating late employment, Test-bed setup 29 – LIP77%0%4%173%133%Delayed employment, compensating TSA % 9 – CESNET91%69%96%102%97% 16A – GRNET47%0%60%48%79% 16B – AUTH42%0% 40%127%Provisioning Workflow maintenance PY2+ 16E – IASA201%281%340%163%22%Implementation Provisioning Workflow in PY1 16F – ICCS33%0% 131%Provisioning Workflow maintenance PY2+ TSA % 9 – CESNET100%97%99%106%96% 10D – JUELICH83%15%82%124%111% 21A – INFN99%81%60%140%117% 36 – UPCH0% 38B – LIU72%0%90%101%97% 41 – NORDUNET33%0% 133%0%PY2: 9 – CESNET takes over task lead and effort

EGI-InSPIRE RI WP5 in a nutshell Processes, Interactions and Artefacts are defined Documentation is of varying quality –Some artefacts are well defined, some are fuzzy –Some processes are executed mostly by intuition Reproducibility of results needs improvement –E.g. impartial provisioning of delivered software –Reporting artefacts vary by producing individuals Technically, objectives are met… SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Plans for next year Continuous review and maintenance of: –Documentation, processes, and artefacts –E.g., requirements engineering in the TCB Enhance DMSU expertise, analysis toolset and knowledgebase Identify and implement missing processes –E.g. “routing slip” for new Technology Providers Investigate community orientated software repositories SA2 – Michel Drescher - EGI-InSPIRE EC Review

EGI-InSPIRE RI Summary Structural work is finished –Verified by production level use with internal providers Quality of service is itself a process –Establish and formalise regular reviews and improvements EMI as largest Technology Provider… –Shows similarities to monopolies in other ICT sectors –Not ready for multi-provider, multi-use case settings SA2 – Michel Drescher - EGI-InSPIRE EC Review