INFSO-RI Enabling Grids for E-sciencE GLOBAL GRID USER SUPPORT THE MODEL AND EXPERIENCE IN LCG/EGEE Gilles Mathieu(1), Torsten Antoni(2), Flavia Donno(3), Helmut Dres(2), Alistair Mills(3), Philippa Strange(4), David Spence(4), Min Tsai(5), Marco Verlato(6) (1) IN2P3/CNRS Computing Centre, Lyon – France (2) Forschungzentrum Karlsruhe, Karlsruhe – Germany (3) CERN, European Organization for Nuclear Research, Geneva – Switzerland (4) Rutherford Appleton Laboratory, CCLRC, Chilton – England (5) Academia Sinica, Tapei – Taiwan (6) Istituto Nazionale Fisica Nucleare (INFN), Padova - Italy

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February Outline Motivation GGUS: history and general description/architecture Services offered GGUS supporters: first and second line support Interface to the Regional Operations Centres (ROCs) GGUS for Grid Operations: CIC-on-duty Some performance number and statistics Conclusions and future work

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February Motivation A single access point for support A portal with a well structured information and updated documentation Knowledgeable experts Correct, complete and responsive support Tools to help resolve problems –search engines –monitoring applications –resources status Examples, templates, specific distributions for software of interest Interface with other Grid support systems Connection with developers, deployment, operation teams Assistance during production use of the grid infrastructure

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February Central Application (GGUS) Deployment Support Middleware Support Network Support Operations Support TPM ROC 1 ROC 10 ROC… VO Support Interface Webportal The Support Model “Regional Support with Central Coordination" The ROCs and VOs and the other project wide groups such as the Core Infrastructure Center (CIC), middleware groups (JRA), network groups (NA), service groups (SA) areCICJRANA connected via a central integration platform provided by GGUS. Regional Support units User Support units Technical Support units

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February The GGUS System

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February GGUS Portal: user services Browseable tickets Search through solved tickets Useful links (Wiki FAQ) Broadcast tools Latest News GGUS Search Engine Updated documentation (Wiki FAQ)

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February GGUS Portal: Search engine GGUSSearchEngine Ongoing work to make it faster and to search through a widerset of docs and DBs

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February GGUS Portal:… and more Hot line with supporters available to answer to problems VO specific supporting teams for middleware integration problems Links to specific useful tools and middleware Examples and templates Links to dedicated Virtual Organization portals that gather resources and monitoring information, contact mailing lists, etc. Modular system to accommodate further needs GGUS has shown to meet user needs

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February TPM Grid experts GGUS Supporters VO-TPM VO experts User First line support VO Support Units Middleware Support Units Deployment Support Units Operations Support ROC Support Units Network Support Second line support

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February Operations support Purpose/role –Detect problems by monitoring the grid –Report them by creating and assigning GGUS tickets –Provide help and follow-up on problems Operations Support teams : “CIC on duty” –Currently 6 teams (CERN, France, Italy, UK, Russia, Taiwan) –Weekly shift CIC/GGUS interface –Based on Web services at GGUS side –“CIC-on-duty dashboard”: graphic user interface for operators, hosted at IN2P3 Computing Centre (Lyon, France)

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February FZK, Karlsruhe, GermanyIN2P3-CC, Lyon, France CIC  ROC Basic Workflow CIC PORTAL CIC-on-duty dashboard UK FRGERIT … Regional Support Units Operator on duty - Create() - Set(ticket) SOAP -Get(ticket) - Get_all() Ticket Central Helpdesk CIC Helpdesk WSDL GGUS Problem detection & reporting Ticket follow-up GGUS Interface

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February GGUS  ROC Basic Workflow Web Portal GGUS System Operations support ROC-1 Helpdesk ROC-1 Interface Ticket solved Ticket assignment to ROC-1 SU-1 SU-2 SU-N ROC-X Helpdesk ROC-X Interface SU-1 SU-2 SU-N Ticket re-assigned TPM

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February ROCs Interface: how it works First Interface between ROC_Italy Helpdesk and GGUS – ready since November ’04 –in ‘production’ since March ‘05 Based on Web Services at GGUS side, several advantages: –sample code available for PHP/Perl/Python and other computing languages –very fast: service requests/sec on the GGUS Servers – easy to adapt Based on at local side (importing tool) XML exchange format Tickets fields mapping between the two systems

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February Performance statistics September October A peak of 80 tickets per day has been reached. The system can handle up to 1400 tickets a week. November 2005: 315 tickets

Enabling Grids for E-sciencE INFSO-RI G. Mathieu et al., CHEP06 Mumbai February Conclusions “Regional Support with Central Coordination" The GGUS model of “Regional Support with Central Coordination" has shown to be a workable and scalable solution for User support, given the feedback received from the users. GGUShas improved The functionality and usability of the GGUS system has improved in the last months, thanks to the help of the ROCs and the experience acquired. GGUS/CIC GGUS/CIC interface has made the system a central mechanism for Grid Operations and monitoring. interfaces with the ROCs The existent interfaces with the ROCs are quite practical and make the system function as one. do not knowrealistic figure The ticket traffic is increasing. We still do not know what a realistic figure would be for the number of ticket to be expected. However, the system can be dimensioned appropriately with more TPMs and support units. metrics established A lot of metrics established to measure the performance of the system. plan to offer resilience to system and network failures GGUS is working on a plan to offer resilience to system and network failures. We need more specialized supporters We need more specialized supporters and dedicated man power in order to guarantee good quality of service. Training sessions for supporters are organized by the GGUS Teams and recording made available off-line.