EGI Process Assessment and Improvement Plan – EGI core services – Tiziana Ferrari FedSM project 1EGI Process Assessment and Improvement Plan (Core Services)

EGI Core Services Technical tools that support EGI daily operations Customers: NGIs and end-users Providers: NGIs on behalf of EGI.eu – Accounting Portal and APEL central Repository – GOCDB – Incident Management Tool (EGI Helpdesk - GGUS) – Message Brokers – Metrics Portal – Service Availability Monitoring – Central SAM (operation tool monitoring) and Nagios instances (security monitoring, middleware versions, etc.) – Operations Portal – Security Monitoring Tools – Catch all Grid Services, operations support tools (DTEAM, OPS), site certification EGI Process Assessment and Improvement Plan (Core Services) 2

(I) Service Level Management of Core Services 3EGI Process Assessment and Improvement Plan (Core Services)

What is it about? Negotiation of the EGI.eu Operational Level Agreement – Definition of measurable targets per service – Documentation of EGI.eu OLA in Document DB, release notes – Periodic review of EGI.eu OLA – Periodic review of achieved service level targets – Improvement plan in case of underperformance No Service Level Agreements currently in place EGI.eu Operational Level Agreement currently including a subset of core services – – 4 EGI Process Assessment and Improvement Plan (Core Services)

Who is responsible? Service Level Manager – EGI Chief Operations Officer Customers – NGIs represented by the NGI Operations Managers – OMB – End-user communities represented by Virtual Research Communities – UCB, where applicable Grid Oversight team – Monthly performance analysis (to be established) 5 EGI Process Assessment and Improvement Plan (Core Services)

What are the requirements (according to FitSM)? Services to be delivered shall be agreed with customers. SLAs shall include agreed service targets. A service catalogue shall be maintained. Services and SLAs shall be reviewed at planned intervals. Service performance shall be monitored against service targets. For supporting services or service components provided by Federation members, OLAs shall be agreed. 6 EGI Process Assessment and Improvement Plan (Core Services)

What is the target capability level? Level 5Level 4Level 3Level 2Level 1Level 0 7 EGI Process Assessment and Improvement Plan (Core Services) Defined (standardized procedures to deliver defined outputs): Clearly defined and documented procedures, roles and responsibilities, but process effectiveness and efficiency not measured and reported

What is the target capability level? There is a defined and documented procedure for negotiating SLAs, which in particular specifies the required scope of the negotiated SLA, the output format in which the SLA must be stored, and how the SLA and service targets are defined. There is a defined and documented procedure for maintaining a service catalogue clearly specifying differentiated service offerings. There is a defined and documented procedure for reviewing services and SLAs at planned intervals. There is a defined and documented procedure for service performance monitoring and reporting the results to relevant parties. There is a defined and documented procedure for establishing OLAs, well-aligned to specific services offered to customers. 8 EGI Process Assessment and Improvement Plan (Core Services)

What is the baseline (current) capability level? Level 5Level 4Level 3Level 2Level 1Level 0 9 EGI Process Assessment and Improvement Plan (Core Services) Repeatable (intuitive activities to deliver results): Repeatable results/outputs most of the time, solid, "intuitive" understanding of activities to be performed, but poor and inconsistent documentation and recording

What is the baseline (current) capability level? There is a defined and documented procedure for negotiating SLAs, which in particular specifies the required scope of the negotiated SLA, the output format in which the SLA must be stored, and how the SLA and service targets are defined. There is a defined and documented procedure for maintaining a service catalogue clearly specifying differentiated service offerings. There is a defined and documented procedure for reviewing services and SLAs at planned intervals  only OLA There is a defined and documented procedure for service performance monitoring and reporting the results to relevant parties  only OLA There is a defined and documented procedure for establishing OLAs, well-aligned to specific services offered to customers. 10 EGI Process Assessment and Improvement Plan (Core Services)

Major gaps to be addressed Lack of a service catalogue  PQ12 PQ13end of Aug 2013 – Assign responsibility of performance review (need reports first) PQ14, end of Oct 2013 – OLA vs SLA  do we need SLAs? undefined relation between SLAs and OLAs, include all core services PQ15, end of Jan 2013 – Many quality procedures tied to EGI-Inspire quality assurance  define new procedures, start periodic review process + review targets 11 EGI Process Assessment and Improvement Plan (Core Services)

(II) Service Reporting 12EGI Process Assessment and Improvement Plan (Core Services)

Service What is it about? Establish infrastructure for gathering of measurements – SAM. SAM for core operations tools in place as of Oct 2012 (not covering all core services right now) Establish infrastructure for report generation – Raw data currently available per tool from MyEGI (central instance) – Operations portal developments due in May 2013 (PQ13) 13 EGI Process Assessment and Improvement Plan (Core Services)

Who is responsible? Service Reporting Manager – To be established  AUTH when reports will be available from the operations portal (PQ13, June 2013) Customers – NGIs represented by the NGI Operations Managers – OMB – End-user communities represented by Virtual Research Communities – UCB, for end-user facing tools Grid Oversight team – Monthly performance analysis, to be established (PQ13, June 2013) 14 EGI Process Assessment and Improvement Plan (Core Services)

SR: Requirements  Service reports shall be specified and agreed with interested parties.  The description of each service report shall include its identity, purpose, audience, frequency and content.  Service reports shall be produced. Service reporting shall include performance against service targets, relevant information about significant events, workload characteristics and detected nonconformities. EGI Process Assessment and Improvement Plan (Core Services) 15

Major gaps to be addressed [Average capability 1.33] PQ13, June 2013 – Customer view of report (need to combine some of the services into one report)? OLA report? SLA report? – Establish process for reporting and performance analysis on a monthly basis – Possibly automate the generation of tickets as already done with sites 16 EGI Process Assessment and Improvement Plan (Core Services)

(III) Incident and Service Request Management 17EGI Process Assessment and Improvement Plan (Core Services)

Incident/service Req. What is it about? Process for registering incidents affecting the tools Manage customer requests (e.g. access to tools, configuration changes, requirements) Manage major incidents which bypass some normal procedures Establish escalation procedures in case of incidents/service requests 18 EGI Process Assessment and Improvement Plan (Core Services)

Who is responsible? Incident and Service Request Manager – Incident Manager: R. Trompert (SA1.7) – Service Request: Chief EGI Operations Officer Incident and Service Request Case Manager – Specific team responsible of supporting and operating the service 19 EGI Process Assessment and Improvement Plan (Core Services)

ISRM Requirements All incidents shall be recorded according to a defined procedure Personnel involved in the incident and service request management process shall have access to relevant information including configuration and release information. Customers shall be kept informed of the progress of their reported incidents and service requests (GGUS and RT)  All incidents shall be classified according to a defined procedure.  All incidents shall be prioritized according to a defined procedure, taking into account impact and urgency of the incidents  Closure of incidents shall follow a defined procedure.  Escalation of incidents shall follow a defined procedure  no operations escalation, no alarms in the operations dashboard  There shall be a definition of a major incident. Major incidents shall be classified and managed according to a documented procedure. EGI Process Assessment and Improvement Plan (Core Services) 20

Major gaps to be addressed [Average capability 2.50 ??] PQ13 – Define escalation (turn tests into operations) – Make sure that all core services are supported in GGUS PQ14 – Lack of procedures for incident classification, prioritization and closure – No definition of major incident – No procedure to handle major incidents  GGUS notification mechanism being discussed for critical services (?) Unclear availability of configuration information 21 EGI Process Assessment and Improvement Plan (Core Services)