Łukasz Skitał 2, Renata Słota 1, Maciej Janusz 1 and Jacek Kitowski 1,2 1 Institute of Computer Science AGH University of Science and Technology, Mickiewicza.

Slides:



Advertisements
Similar presentations
Polish Infrastructure for Supporting Computational Science in the European Research Space EUROPEAN UNION Services and Operations in Polish NGI M. Radecki,
Advertisements

Optimizing of data access using replication technique Renata Słota 1, Darin Nikolow 1,Łukasz Skitał 2, Jacek Kitowski 1,2 1 Institute of Computer Science.
Sergey Belov, LIT JINR 15 September, NEC’2011, Varna, Bulgaria.
Client/Server Grid applications to manage complex workflows Filippo Spiga* on behalf of CRAB development team * INFN Milano Bicocca (IT)
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES News on monitoring for CMS distributed computing operations Andrea.
LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.
EUROPEAN UNION Polish Infrastructure for Supporting Computational Science in the European Research Space Cracow Grid Workshop’10 Kraków, October 11-13,
The SAM-Grid Fabric Services Gabriele Garzoglio (for the SAM-Grid team) Computing Division Fermilab.
Cracow - CYFRONET PACKAGING pack into portable format e.g. rpm PACKAGING pack into portable format e.g. rpm PACKAGING pack into portable format e.g. rpm.
Virtual Organization Approach for Running HEP Applications in Grid Environment Łukasz Skitał 1, Łukasz Dutka 1, Renata Słota 2, Krzysztof Korcyl 3, Maciej.
Polish Infrastructure for Supporting Computational Science in the European Research Space Policy Driven Data Management in PL-Grid Virtual Organizations.
Advanced Grid-Enabled System for Online Application Monitoring Main Service Manager is a central component, one per each.
CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.
CGW 2003 Institute of Computer Science AGH Proposal of Adaptation of Legacy C/C++ Software to Grid Services Bartosz Baliś, Marian Bubak, Michał Węgiel,
Polish Infrastructure for Supporting Computational Science in the European Research Space QoS provisioning for data-oriented applications in PL-Grid D.
OSG Operations and Interoperations Rob Quick Open Science Grid Operations Center - Indiana University EGEE Operations Meeting Stockholm, Sweden - 14 June.
HPDC 2007 / Grid Infrastructure Monitoring System Based on Nagios Grid Infrastructure Monitoring System Based on Nagios E. Imamagic, D. Dobrenic SRCE HPDC.
CERN, BalticGrid Project Rolandas Naujikas [rolandas nawyeekas] CERN IT/GD Vilnius University Lithuania.
A Proposal of Application Failure Detection and Recovery in the Grid Marian Bubak 1,2, Tomasz Szepieniec 2, Marcin Radecki 2 1 Institute of Computer Science,
CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Job Monitoring for the LHC experiments Irina Sidorova (CERN, JINR) on.
LCG Middleware Testing in 2005 and Future Plans E.Slabospitskaya, IHEP, Russia CERN-Russia Joint Working Group on LHC Computing March, 6, 2006.
1 1 Service Composition for LHC Computing Grid Monitoring Beob Kyun Kim e-Science Division, KISTI
Migrating Desktop The graphical framework for running grid applications Bartek Palak Poznan Supercomputing and Networking Center The.
Cracow Grid Workshop October 2009 Dipl.-Ing. (M.Sc.) Marcus Hilbrich Center for Information Services and High Performance.
High Level Architecture (HLA)  used for building interactive simulations  connects geographically distributed nodes  time management (for time- and.
Migrating Desktop Marcin Płóciennik Marcin Płóciennik Kick-off Meeting, Santander, Graphical.
And Tier 3 monitoring Tier 3 Ivan Kadochnikov LIT JINR
Enabling Grids for E-sciencE System Analysis Working Group and Experiment Dashboard Julia Andreeva CERN Grid Operations Workshop – June, Stockholm.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Nagios for Grid Services E. Imamagic, SRCE.
Cracow Grid Workshop ‘06 17 October 2006 Execution Management and SLA Enforcement in Akogrimo Antonios Litke Antonios Litke, Kleopatra Konstanteli, Vassiliki.
KUKDM’2011, Zakopane Semantic Based Storage QoS Management Methodology Renata Słota, Darin Nikolow, Jacek Kitowski Institute of Computer Science AGH-UST,
Interactive Workflows Branislav Šimo, Ondrej Habala, Ladislav Hluchý Institute of Informatics, Slovak Academy of Sciences.
Kraków Kick-off meeting Migrating Desktop General concept Intuitive Grid-user’s work environment independent of a hardware.
US LHC OSG Technology Roadmap May 4-5th, 2005 Welcome. Thank you to Deirdre for the arrangements.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks SA1: Grid Operations Maite Barroso (CERN)
Migrating Desktop Bartek Palak Bartek Palak Poznan Supercomputing and Networking Center The Graphical Framework.
Trusted Virtual Machine Images a step towards Cloud Computing for HEP? Tony Cass on behalf of the HEPiX Virtualisation Working Group October 19 th 2010.
EUROPEAN UNION Polish Infrastructure for Supporting Computational Science in the European Research Space Operational Architecture of PL-Grid project M.Radecki,
INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.
Interactive European Grid Environment for HEP Application with Real Time Requirements Lukasz Dutka 1, Krzysztof Korcyl 2, Krzysztof Zielinski 1,3, Jacek.
Cracow Grid Workshop, October 15-17, 2007 Polish Grid Polish NGI Contribution to EGI Resource Provisioning Function Automatized Direct Communication Tomasz.
High Level Architecture (HLA)  used for building interactive simulations  connects geographically distributed nodes  time management (for time- and.
Last update 31/01/ :41 LCG 1 Maria Dimou Procedures for introducing new Virtual Organisations to EGEE NA4 Open Meeting Catania.
Tool Integration with Data and Computation Grid “Grid Wizard 2”
Julia Andreeva on behalf of the MND section MND review.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Resource Allocation in EGEEIII Overview &
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Monitoring of the LHC Computing Activities Key Results from the Services.
Marian Bubak 1,2, Włodzimierz Funika 1,2, Roland Wismüller 3, Tomasz Arodź 1,2, Marcin Kurdziel 1,2 1 Institute of Computer Science, AGH, Kraków, Poland.
Global ADC Job Monitoring Laura Sargsyan (YerPhI).
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Grid Oversight in Service Level Agreement environment Małgorzata Krakowian,
Migrating Desktop Uniform Access to the Grid Marcin Płóciennik Poznan Supercomputing and Networking Center Poznan, Poland EGEE’07, Budapest, Oct.
Open Science Grid OSG Resource and Service Validation and WLCG SAM Interoperability Rob Quick With Content from Arvind Gopu, James Casey, Ian Neilson,
DataTAG is a project funded by the European Union International School on Grid Computing, 23 Jul 2003 – n o 1 GridICE The eyes of the grid PART I. Introduction.
Migrating Desktop Uniform Access to the Grid Marcin Płóciennik Poznan Supercomputing and Networking Center Poland EGEE’08 Conference, Istanbul, 24 Sep.
DataTAG is a project funded by the European Union CERN, 8 May 2003 – n o 1 / 10 Grid Monitoring A conceptual introduction to GridICE Sergio Andreozzi
II EGEE conference Den Haag November, ROC-CIC status in Italy
Claudio Grandi INFN Bologna Virtual Pools for Interactive Analysis and Software Development through an Integrated Cloud Environment Claudio Grandi (INFN.
SAM architecture EGEE 07 Service Availability Monitor for the LHC experiments Simone Campana, Alessandro Di Girolamo, Nicolò Magini, Patricia Mendez Lorenzo,
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Resource allocation Małgorzata Krakowian 1.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks The Dashboard for Operations Cyril L’Orphelin.
G. Russo, D. Del Prete, S. Pardi Kick Off Meeting - Isola d'Elba, 2011 May 29th–June 01th A proposal for distributed computing monitoring for SuperB G.
InSilicoLab – Grid Environment for Supporting Numerical Experiments in Chemistry Joanna Kocot, Daniel Harężlak, Klemens Noga, Mariusz Sterzel, Tomasz Szepieniec.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.
Honolulu - Oct 31st, 2007 Using Glideins to Maximize Scientific Output 1 IEEE NSS 2007 Making Science in the Grid World - Using Glideins to Maximize Scientific.
Daniele Bonacorsi Andrea Sciabà
Regional Operations Centres Core infrastructure Centres
E-GRANT: EGI Resource Allocation Tool Current status
Patricia Méndez Lorenzo ALICE Offline Week CERN, 13th July 2007
Interoperability & Standards
Management of Virtual Execution Environments 3 June 2008
Presentation transcript:

Łukasz Skitał 2, Renata Słota 1, Maciej Janusz 1 and Jacek Kitowski 1,2 1 Institute of Computer Science AGH University of Science and Technology, Mickiewicza 30, Krakow, Poland 2 Academic Computer Centre CYFRONET-AGH, Nawojki 11, Krakow, Poland Int.eu.grid project Grid environments evolve from being 'better batch systems' into full-featured systems supporting variety of application types. Int.eu.grid project focuses on creating grid environment for interactive or real-time applications. Due to it's strict requirements, real-time applications belongs to the most demanding group, which is very difficult to support in distributed infrastructure. To support one of this difficult applications we have created a specialized virtual organization (HEP VO) with web-based Management System. Management System for HEP VO (MS-HEPVO) provides control over HEP VO specific aspects without overlapping with existing Virtual Organization Management System (VOMS). MS-HEPVO covers following areas of VO management: site management (certification, SLA, monitoring and statistics) and application control (grid job submission, job pool support, diagnostics). The Application and VO HEP Application is based on ATLAS filtering framework for LHC experiment at CERN. Considerable effort has been put into migration of this application to the grid environment. Application uses pilot jobs idea and requires real-time communication between LHC experiment and grid worker nodes. HEP VO provides stable, dynamic environment using resources available from many grid sites. Management of this kind of VO is not trivial. To be up to this task, VO is based on contracts strengthened by Service Level Agreements (SLA) with sites. Management of the environment and monitoring of contracts fulfillment require some additional effort, which can be minimized with adequate tools, like Management System for HEP VO. User interface The system offers portal-based interface, which can be used by VO Manager, site administrators and application supervisors. This interface provides, among others, quick overview of application's run-time environment condition basing on SLAs fulfillment. The system is also integrated with Migrating Desktop (MD), offering interface (plug-in) for application supervisor. MD plug- in allows to adjust grid job submission to current experiment needs. Management System for HEP VO Management System for HEP VO supports VO management process. The system consists of two web- services: Grid job submission service is responsible for application run-time environment. It keeps number of grid jobs on level desired by the application. The service takes advantage of dynamic resource allocation offered by the grid. Site management and SLA Monitoring service is used to keep track of site operations, including site certification, monitoring and failure recovery. The service gathers SLA metrics values from different information sources (like JIMS, O-CMG, SAM or BDII) and checks, if SLAs are fulfilled. It can react on any SLA infringement by changing rules for resource allocation (eg. exclude failing resources) and sending warnings or alarms. Use cases Management System for HEP VO is used by three type of users: VO Manager supervise VO operation. He performs certification of sites, manages site state according to its operation, prepares job submission profiles (application parameters sets). VO Manager checks also site operation according to signed SLA and in case of any problem can change site status. Experiment operator supervise the application. He control job submission basing on current experiment needs and can retrieve job output for problem diagnostic. Site owner register site by filling a form. He provides all relevant information which are also used for preparation of site’s specific SLA References [1] The int.eu.grid project [2] Service Level Agreement metrics for real-time application on the Grid Skital L., Janusz M., Slota R., Kitowski J.,PPAM 2007, September 9-12, 2007, Poland Acknowledgement The work has been supported by EU IST int.eu.grid project. Certification procedure Each site before going into production has to pass certification procedure, which validates site configuration and quality. MS-HEPVO supports this procedure. It provides a form for site registration, what starts certification process, allows to change site status during the procedure, visualize monitoring data and support SLA which is signed after successful certification. SLA Enforcement MS-HEPVO monitors SLA fulfillment. It gather data for each metric for each site and creates a report for VO Manager. In case of problems a warning to VO Manager and an administrator is sent. If failing site disrupt application stability, site’s operation in HEP VO can be automatically suspended.