Overview of Monitoring and Information Systems in OSG MWGS08 - September 18, 2008 - Chicago Marco Mambelli - University of Chicago

Slides:

Advertisements

Similar presentations

Open Science Grid Discovering and understanding the site environment Or, yet another site test kit.

Advertisements

FP7-INFRA Enabling Grids for E-sciencE EGEE Induction Grid training for users, Institute of Physics Belgrade, Serbia Sep. 19, 2008.

DataGrid is a project funded by the European Union 22 September 2003 – n° 1 EDG WP4 Fabric Management: Fabric Monitoring and Fault Tolerance

Sergey Belov, LIT JINR 15 September, NEC’2011, Varna, Bulgaria.

MyOSG: A user-centric information resource for OSG infrastructure data sources Arvind Gopu, Soichi Hayashi, Rob Quick Open Science Grid Operations Center.

OSG End User Tools Overview OSG Grid school – March 19, 2009 Marco Mambelli - University of Chicago A brief summary about the system.

Rsv-control Marco Mambelli – Site Coordination meeting October 1, 2009.

CILogon OSG CA Mine Altunay Jim Basney TAGPMA Meeting Pittsburgh May 27, 2015.

OSG Operations and Interoperations Rob Quick Open Science Grid Operations Center - Indiana University EGEE Operations Meeting Stockholm, Sweden - 14 June.

OSG Services at Tier2 Centers Rob Gardner University of Chicago WLCG Tier2 Workshop CERN June 12-14, 2006.

OSG Middleware Roadmap Rob Gardner University of Chicago OSG / EGEE Operations Workshop CERN June 19-20, 2006.

Mark L. Green, Ph.D. Grid Computational Scientist Monitoring and Information Services Architecture and Deployment University at Buffalo The State University.

Publication and Protection of Site Sensitive Information in Grids Shreyas Cholia NERSC Division, Lawrence Berkeley Lab Open Source Grid.

G RID M IDDLEWARE AND S ECURITY Suchandra Thapa Computation Institute University of Chicago.

OSG S ITE INSTALLATION AND M AINTENANCE Suchandra Thapa Computation Institute University of Chicago.

DOSAR Workshop, Sao Paulo, Brazil, September 16-17, 2005 LCG Tier 2 and DOSAR Pat Skubic OU.

OSG Software and Operations Plans Rob Quick OSG Operations Coordinator Alain Roy OSG Software Coordinator.

Production Coordination Staff Retreat July 21, 2010 Dan Fraser – Production Coordinator.

Mar 28, 20071/18 The OSG Resource Selection Service (ReSS) Gabriele Garzoglio OSG Resource Selection Service (ReSS) Don Petravick for Gabriele Garzoglio.

São Paulo Regional Analysis Center SPRACE Status Report 22/Aug/2006 SPRACE Status Report 22/Aug/2006.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks GStat 2.0 Joanna Huang (ASGC) Laurence Field.

And Tier 3 monitoring Tier 3 Ivan Kadochnikov LIT JINR

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks WMSMonitor: a tool to monitor gLite WMS/LB.

Open Science Grid OSG CE Quick Install Guide Siddhartha E.S University of Florida.

OSG Production Report OSG Area Coordinator’s Meeting Aug 12, 2010 Dan Fraser.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Nagios for Grid Services E. Imamagic, SRCE.

Introduction to OSG Fundamentals Suchandra Thapa Computation Institute University of Chicago March 19, 20091GSAW 2009 Clemson.

OSG Tier 3 support Marco Mambelli - OSG Tier 3 Dan Fraser - OSG Tier 3 liaison Tanya Levshina - OSG.

July 25, 20071/21 OSG Information Services Gabriele Garzoglio, Rob Quick, Chris Green OSG Information Services, VO Monitoring Services and Resource Selection.

INFSO-RI Enabling Grids for E-sciencE OSG-LCG Interoperability Activity Author: Laurence Field (CERN)

Grid Operations Lessons Learned Rob Quick Open Science Grid Operations Center - Indiana University.

VO Privilege Activity. The VO Privilege Project develops and implements fine-grained authorization to grid- enabled resources and services Started Spring.

INFSO-RI Enabling Grids for E-sciencE GridICE: Grid and Fabric Monitoring Integrated for gLite-based Sites Sergio Fantinel INFN.

Site Validation Session Report Co-Chairs: Piotr Nyczyk, CERN IT/GD Leigh Grundhoefer, IU / OSG Notes from Judy Novak WLCG-OSG-EGEE Workshop CERN, June.

Production Coordination Area VO Meeting Feb 11, 2009 Dan Fraser – Production Coordinator.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Using GStat 2.0 for Information Validation.

The OSG and Grid Operations Center Rob Quick Open Science Grid Operations Center - Indiana University ATLAS Tier 2-Tier 3 Meeting Bloomington, Indiana.

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America gLite Information System Claudio Cherubino.

AstroGrid-D Meeting MPE Garching, M. Braun VO Management.

Auditing Project Architecture VERY HIGH LEVEL Tanya Levshina.

Eileen Berman. Condor in the Fermilab Grid FacilitiesApril 30, 2008  Fermi National Accelerator Laboratory is a high energy physics laboratory outside.

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI How to integrate portals with the EGI monitoring system Dusan Vudragovic.

Open Science Grid Build a Grid Session Siddhartha E.S University of Florida.

April 25, 2006Parag Mhashilkar, Fermilab1 Resource Selection in OSG & SAM-On-The-Fly Parag Mhashilkar Fermi National Accelerator Laboratory Condor Week.

Area Coordinator Report for Operations Rob Quick 4/10/2008.

CERN - IT Department CH-1211 Genève 23 Switzerland CASTOR F2F Monitoring at CERN Miguel Coelho dos Santos.

VOX Project Status T. Levshina. 5/7/2003LCG SEC meetings2 Goals, team and collaborators Purpose: To facilitate the remote participation of US based physicists.

Open Science Grid OSG Resource and Service Validation and WLCG SAM Interoperability Rob Quick With Content from Arvind Gopu, James Casey, Ian Neilson,

Site Authorization Service Local Resource Authorization Service (VOX Project) Vijay Sekhri Tanya Levshina Fermilab.

The Resource Selection Service (ReSS) Activity Gabriele Garzoglio Fermilab, Computing Division March 14, 2006.

Operations Area Coordinator Report. 31 Jan Overview Operations Current Initiatives  RSV Version 2  New Probes, Easier Configuration, Improved.

Tier 3 Support and the OSG US ATLAS Tier2/Tier3 Workshop at UChicago August 20, 2009 Marco Mambelli –

FESR Trinacria Grid Virtual Laboratory gLite Information System Muoio Annamaria INFN - Catania gLite 3.0 Tutorial Trigrid Catania,

DataTAG is a project funded by the European Union CERN, 8 May 2003 – n o 1 / 10 Grid Monitoring A conceptual introduction to GridICE Sergio Andreozzi

OSG Status and Rob Gardner University of Chicago US ATLAS Tier2 Meeting Harvard University, August 17-18, 2006.

E-science grid facility for Europe and Latin America Updates on Information System Annamaria Muoio - INFN Tutorials for trainers 01/07/2008.

Parag Mhashilkar Computing Division, Fermilab.  Status  Effort Spent  Operations & Support  Phase II: Reasons for Closing the Project  Phase II:

Certificate Security For Users Obtaining and Using Your Personal Certificate using the OSG PKI Kyle Gross – OSG Operations Support Lead Elizabeth Prout.

Open Science Grid Configuring RSV OSG Resource & Service Validation Thomas Wang Grid Operations Center (OSG-GOC) Indiana University.

1 Grid2003 Monitoring, Metrics, and Grid Cataloging System Leigh GRUNDHOEFER, Robert QUICK, John HICKS (Indiana University) Robert GARDNER, Marco MAMBELLI,

3/5/2007Chris Green, FNAL / OSG VO-Level Site Validation.

Grid Colombia Workshop with OSG Week 2 Startup Rob Gardner University of Chicago October 26, 2009.

OSG Operations – Lessons Learned CHEP 2010, 18 October 15:10 (Asia/Taipei) – Room 2, BHSS OSG Operations – Lessons Learned CHEP 2010, 18 October 15:10.

Open Science Grid Progress and Status

Monitoring and Information Services Technical Group Report

POW MND section.

Evolution of SAM in an enhanced model for monitoring the WLCG grid

Leigh Grundhoefer Indiana University

Monitoring and Information Services

Information Services Claudio Cherubino INFN Catania Bologna

Presentation transcript:

Overview of Monitoring and Information Systems in OSG MWGS08 - September 18, Chicago Marco Mambelli - University of Chicago

Outline  Monitoring principles and OSG Monitoring  Central monitoring at GOC  Systems synergy: CEMon and BDII  Systems evolution: VORS and RSV  Resource exploration 9/18/08MWGS08 - OSG MIS - Marco Mambelli2

Monitoring and IS  Producer, Consumer, Intermediaries  Schema, Presentation, Content  Monitoring at the VO level  Panda  Monitoring at the Resource level  Ganglia, Cactus, Nagios, Custom systems  Monitoring and IS in OSG: ion/WebHome ion/WebHome  Information scouting 9/18/08MWGS08 - OSG MIS - Marco Mambelli3

OSG Monitoring  Grid Site_Verify Scanner  Test, GOC  Virtual Organization Resource Selector (VORS)  IS display (from Site_Verify), GOC  Generic Information Provider (GIP) Validation Service  Test, GOC  LDAP (CEMon/BDII) information display utility  IS display (from GIP), GOC  Gratia Accounting  Accounting, 3rdP  Resource and Service Validation  Test, Local+GOC  Virtual Organization Membership System (VOMS) Monitor  Test (VOMS servers), GOC 9/18/08MWGS08 - OSG MIS - Marco Mambelli4

Information Systems  OSG Grid Operations Center (GOC) Alerts and RSS Feed  Info (T-Tickets), GOC  OSG GOC Ticket Metrics Reports  T-Tickets, GOC  OSG Maintenance Scheduling Tool  now OIM  OSG Registration DB  now OIM  OSG Information Management (OIM) System  Info, Sysadmins  OSG Pacman Software Caches  Software packages, OSG 9/18/08MWGS08 - OSG MIS - Marco Mambelli5

BDII/CEMon  Alternative sources (Provider)  CEMon Consumer (passive)  BDII Scanner (active)  Data collector, aggregator (Intermediary)  Daemon  Storage and Server (Consumer)  Served Data Storage  BDII Server 9/18/08MWGS08 - OSG MIS - Marco Mambelli6

ReSS, alternative Consumer of CEMon 9/18/08MWGS08 - OSG MIS - Marco Mambelli7 Condor Match Maker Info Gatherer classads Condor Scheduler job What Gate? Gate 3 job CEMon CE Gate1 job-managers jobsinfo CLUSTER GIP CEMon CE Gate2 job-managers jobsinfo CLUSTER GIP CEMon CE Gate3 job-managers jobsinfo CLUSTER GIP

VO Resource Selector 9/18/08MWGS08 - OSG MIS - Marco Mambelli8

From VORS to RSV  Involve more information consumers (resource admins, users, VOs)  If possible run test locally, allowing still central collection and centralized triggering  Reduce reaction loop, removing the need for GOC’s intervention.  Allow different information nd status checks for GOC, VOs, Users, Admins 9/18/08MWGS08 - OSG MIS - Marco Mambelli9

Status monitors VORS RSV  tests collected in single probe (site_verify)  GOC running the test (grid job)  local consumers query central display  multiple probes (everyone can add probes)  runs locally  local display  GOC collects information for central display 9/18/08MWGS08 - OSG MIS - Marco Mambelli10

Resource and Service Validation 9/18/08MWGS08 - OSG MIS - Marco Mambelli11

Information scouting: OSG CE  Basic tests with Globus clients  Resource exploration  Know the resource (=read OSG and Middleware documentation):  ewOfServicesInOSG ewOfServicesInOSG  eModels eModels  uteElementInstall uteElementInstall  Informed exploration  OSG, Globus, … 9/18/08MWGS08 - OSG MIS - Marco Mambelli12

Other Systems (deprecated) GridCat MonALISA 9/18/08MWGS08 - OSG MIS - Marco Mambelli13