1. Regional Operations Centres Core infrastructure Centres
M. Cristina Vistoli
Cork Conference
Regional Operations Centres
Core infrastructure Centres
Role and Responsibilities
EGEE is proposed as a project funded by the
European Union under contract IST

2. ROC Objectives
Bring the middleware release to a set of Resource Centers
Maintain the Grid resource centers aligned with the release
Validate the set of grid services and components to form a functioning Grid to serve user communities
Produce and validate installation/upgrade procedures and documentation
Develop/Run test suites for components and Grid services to certify a site installation
Allow a Resource Center to join the EGEE production infrastructure
Develop expertise in the usage and troubleshooting of problems experienced by resource centers and users

3. ROC Objectives
Provide support to resource centers and users, interact with Application/VO specific support
Refer and escalate middleware problems to developers
Collaborate with CIC to have monitoring systems and performance evaluator running
Define the tools to measure the service level provided by resource centers ….

4. ROC –team 1 Certification , Release documentation and distribution
Responsibility to certify the middleware release for the region:
the Certification Activity is a collaborative activity among the ‘ROCs/Release groups’, in contact with Middleware developments
Use of a distributed Certification testbed to ‘validate’ the middleware and run Grid services in a ‘pre-production’ environment
Responsibility to customize the middleware release for the region
add specific region VO’s
defining Grid services configuration for VO’s
defining local configuration with Resource Centers
Responsibility to host a CVS packages repository
Cern central packages repository
ROC specific repository
Responsibility to provide configuration and automatic installation tools for the region’s production infrastructure sites
Responsibility to run local installation testbed to verify installation procedures

5. ROC –team 2: GRID Management Team
Responsibility to coordinate the upgrade or new middleware installation with the Resource Centers in the region defining plans and timetables
Responsibility to keep a repository of Resource Centers configurations
Responsibility to certify the Resource Centers in the region/federation willing to join the EGEE production infrastructure
in collaboration with Resource centers Grid managers
Development of a site certification suite
Responsibility to provide support during the deployment phase
Responsibility to interact with OMC and corresponding GRID teams in ROC’s to provide an agreed general EGEE infrastructure system architecture and deployment plan

6. ROC –team 2: GRID Management Team
Responsibility to provide and develop testing/certification procedure to verify and validate Resource Center and Grid services for VO applications
Development of Application specific certification suites for Grid Services and Resources, in collaboration with VO managers, VO grid and application experts

7. ROC –team 3: GRID Support Team
Responsibility to provide support:
For VO Applications, to VO users
Interface with VO Applications support systems
For VO Services: VOMS, RLS and VO specific Grid services, RB, GIS, Monitoring
To Resource centres
For Release/Installation/documentation
For Infrastructure, system wide Grid Services (Production GRID RB, RLS, Monitoring service, GIS)
Responsibility to develop specific procedures to recover or proactively avoid congestion or faulty situation in resource centers
Responsibility to provide a Knowledge Base to be shared among ROCs
Coordination of support procedures and ticket-database exchange
procedures to provide interoperability among ROC’s support systems and give users ‘transparent’ problem resolution.
Responsibility to escalate problems to CIC or middleware developers
How are ROC’s notified about bug solutions?

8. CIC Objectives Provide the basic GRID service infrastructure
operating the key services which connect users with resources.
Proactively monitor core services and provide support to the Regional Operations Centres
Provide operations and planning to operate and monitor grid-wide and Virtual Organization-specific grid services
Monitor infrastructure components and services
Verify that suppliers are accomplishing SLA
Tune and optimize performance
Publish usage figures and accounting

9. CIC- Team 1 Grid Services Management team
Responsibility to operate
Monitoring services
Resource brokering service (RB)
Allocation and scheduling services
Replica data catalogues (RLS)
Authorisation services (VOMS)
Accounting services
Information services (BDII)
Responsibility to validate a set of grid services and components to form a functioning Grid to serve user communities and Virtual Organization
Responsibility to collect and provide usage and availability figures about resources, jobs, VO, grid services, Service Level Agreement compliance, resources sharing, etc.

10. CIC- Team 2 Grid Services Support team
Responsibility to provide support and proactively monitoring all the grid services (Resource Brokers, RLS, GIS etc) in the federation to avoid congestion or faulty situations.
Responsibility to provide support to Roc’s, application and virtual organizations
Responsibility to interact with Virtual Organization to validate Grid services and middleware configuration for specific applications

11. CIC- Team 3 Monitoring team
Responsibility to:
Operate and run monitoring servers
Monitor VO usage/availability of resources
Provide Job Monitoring at VO level
Distribute monitoring ‘plug-ins’ to be integrated in the middleware relase
Develop and customize monitoring tools

14. Use case: GOC (ROC-CIC) experience and organization in Italian Grid
Production Grid : new issues
Set up a Management Organization
Coordination committee to analyze experiment requirements and define resource access policies.
Deployment planning according to CC decisions
New Management tools definition and implementation
Operations Services and support for Multi-VO users

15. Services and testbeds
Production service  important to define procedures for ROC to connect the Resource Centres to Production Infrastructure, interaction with National GRIDs is mandatory
In Italy 17 sites with LCG-2 middleware ready to join the prod services, certified by the Italian ROC
In other regions probably there is a similar situation
Development service  JRA1
Which sites, level of resources
Certification testbed  former EDG application TB?
Extended – which sites, resources
Multiple? Production, development
Deployment testbeds  ROC responsibility
Deployment teams ROC’s – to test deployment procedure
Application integration testbed  NA4
Few machines, apps to build with grid m/w – no real service
Tutorial/training testbed  GILDA, NA4
On-demand, not permanent, where? Resources?