WLCG Information System Use Cases Review WLCG Operations Coordination Meeting 18 th June 2015 Maria Alandes IT/SDC.

Slides:



Advertisements
Similar presentations
Applications Area Issues RWL Jones GridPP13 – 5 th June 2005.
Advertisements

HTCondor and the European Grid Andrew Lahiff STFC Rutherford Appleton Laboratory European HTCondor Site Admins Meeting 2014.
Jan 2010 Current OSG Efforts and Status, Grid Deployment Board, Jan 12 th 2010 OSG has weekly Operations and Production Meetings including US ATLAS and.
AGIS for CMS. Proposal (1) To evaluate AGIS as a long term strategy for CMS for describing CMS topology (sites and services both SEs and CEs, but not.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Status of Interoperability Markus Schulz.
LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.
Tier-1 Batch System Report Andrew Lahiff, Alastair Dewhurst, John Kelly, Ian Collier 5 June 2013, HEP SYSMAN.
Monitoring the Grid at local, national, and Global levels Pete Gronbech GridPP Project Manager ACAT - Brunel Sept 2011.
CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Job Monitoring for the LHC experiments Irina Sidorova (CERN, JINR) on.
Monitoring in EGEE EGEE/SEEGRID Summer School 2006, Budapest Judit Novak, CERN Piotr Nyczyk, CERN Valentin Vidic, CERN/RBI.
Marian Babik, Luca Magnoni SAM Test Framework. Outline  SAM Test Framework  Update on Job Submission Timeouts  Impact of Condor and direct CREAM tests.
Grid job submission using HTCondor Andrew Lahiff.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks State of Interoperability Laurence Field.
Multi-core jobs at the RAL Tier-1 Andrew Lahiff, Alastair Dewhurst, John Kelly February 25 th 2014.
Enabling Grids for E-sciencE System Analysis Working Group and Experiment Dashboard Julia Andreeva CERN Grid Operations Workshop – June, Stockholm.
OSG Production Report OSG Area Coordinator’s Meeting Aug 12, 2010 Dan Fraser.
Towards a Global Service Registry for the World-Wide LHC Computing Grid Maria ALANDES, Laurence FIELD, Alessandro DI GIROLAMO CERN IT Department CHEP 2013.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks State of Interoperability Laurence Field.
EGEE-III INFSO-RI Enabling Grids for E-sciencE Overview of STEP09 monitoring issues Julia Andreeva, IT/GS STEP09 Postmortem.
GLUE 2 Open Issues in Storage Information Providers 16 th May 2014.
1 User Analysis Workgroup Discussion  Understand and document analysis models  Best in a way that allows to compare them easily.
Julia Andreeva, CERN IT-ES GDB Every experiment does evaluation of the site status and experiment activities at the site As a rule the state.
1 Andrea Sciabà CERN Critical Services and Monitoring - CMS Andrea Sciabà WLCG Service Reliability Workshop 26 – 30 November, 2007.
Information System Status and Evolution Maria Alandes Pradillo, CERN CERN IT Department, Grid Technology Group GDB 13 th June 2012.
Monitoring for CCRC08, status and plans Julia Andreeva, CERN , F2F meeting, CERN.
EGEE-III INFSO-RI Enabling Grids for E-sciencE Ricardo Rocha CERN (IT/GS) EGEE’08, September 2008, Istanbul, TURKEY Experiment.
Site Validation Session Report Co-Chairs: Piotr Nyczyk, CERN IT/GD Leigh Grundhoefer, IU / OSG Notes from Judy Novak WLCG-OSG-EGEE Workshop CERN, June.
INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.
SAM Sensors & Tests Judit Novak CERN IT/GD SAM Review I. 21. May 2007, CERN.
RSV: OSG Grid Fabric Monitoring and Interoperation with WLCG Monitoring Systems Rob Quick, Arvind Gopu, and Soichi Hayashi Computing in High Energy and.
ATP Future Directions Availability of historical information for grid resources: It is necessary to store the history of grid resources as these resources.
Workload management, virtualisation, clouds & multicore Andrew Lahiff.
Julia Andreeva on behalf of the MND section MND review.
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES Andrea Sciabà Hammercloud and Nagios Dan Van Der Ster Nicolò Magini.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Monitoring of the LHC Computing Activities Key Results from the Services.
ATLAS intentions on Job Priorities Simone Campana.
MND review. Main directions of work  Development and support of the Experiment Dashboard Applications - Data management monitoring - Job processing monitoring.
Storage Interfaces and Access pre-GDB Wahid Bhimji University of Edinburgh On behalf of all those who participated.
Report on Installed Resource Capacity Flavia Donno CERN/IT-GS WLCG Management Board, CERN 25 November 2008.
1 Cloud Services Requirements and Challenges of Large International User Groups Laurence Field IT/SDC 2/12/2014.
Next Steps after WLCG workshop Information System Task Force 11 th February
New solutions for large scale functional tests in the WLCG infrastructure with SAM/Nagios: The experiments experience ES IT Department CERN J. Andreeva.
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES Andrea Sciabà Ideal information system - CMS Andrea Sciabà IS.
HTCondor-CE for USATLAS Bob Ball AGLT2/University of Michigan OSG AHM March, 2015 Bob Ball AGLT2/University of Michigan OSG AHM March, 2015.
Open Science Grid OSG Resource and Service Validation and WLCG SAM Interoperability Rob Quick With Content from Arvind Gopu, James Casey, Ian Neilson,
CMS: T1 Disk/Tape separation Nicolò Magini, CERN IT/SDC Oliver Gutsche, FNAL November 11 th 2013.
Grid Deployment Board 5 December 2007 GSSD Status Report Flavia Donno CERN/IT-GD.
News and Status WLCG Information System Task Force Maria Alandes IT/SDC.
The GridPP DIRAC project DIRAC for non-LHC communities.
WLCG Operations Coordination report Maria Alandes, Andrea Sciabà IT-SDC On behalf of the WLCG Operations Coordination team GDB 9 th April 2014.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE Operations: Evolution of the Role of.
SAM Status Update Piotr Nyczyk LCG Management Board CERN, 5 June 2007.
Status of gLite-3.0 deployment and uptake Ian Bird CERN IT LCG-LHCC Referees Meeting 29 th January 2007.
Accounting Review Summary from the pre-GDB related to CPU (wallclock) accounting Julia Andreeva CERN-IT GDB 13th April
CREAM CE: upgrades in the system  Migration of the ALICE production queue in the CREAM CE: DONE  From pps-cream-fzk.gridka.de:8443/cream-pbs-pps to.
GGUS summary (3 weeks) VOUserTeamAlarmTotal ALICE7029 ATLAS CMS LHCb Totals
SAM architecture EGEE 07 Service Availability Monitor for the LHC experiments Simone Campana, Alessandro Di Girolamo, Nicolò Magini, Patricia Mendez Lorenzo,
WLCG Accounting Task Force Update Julia Andreeva CERN GDB, 8 th of June,
WLCG Information System Status Maria Alandes Pradillo, CERN CERN IT Department, Support for Distributed Computing Group GDB 9 th September 2015.
Maria Alandes Pradillo, CERN Training on GLUE 2 information validation EGI Technical Forum September 2013.
The Grid Information System Maria Alandes Pradillo IT-SDC White Area Lecture, 4th June 2014.
WLCG IPv6 deployment strategy
ATLAS Grid Information System
POW MND section.
CREAM Status and Plans Massimo Sgaravatto – INFN Padova
lcg-infosites documentation (v2.1, LCG2.3.1) 10/03/05
Monitoring of the infrastructure from the VO perspective
WMS Options: DIRAC and GlideIN-WMS
Presentation transcript:

WLCG Information System Use Cases Review WLCG Operations Coordination Meeting 18 th June 2015 Maria Alandes IT/SDC

Scope of this presentation  These slides summarise the current use of the Information System by the following identified Information System users or consumers:  ALICE  ATLAS  CMS  LHCb  SAM  REBUS  Miscellanea: GFAL, WLCG Google Earth Dashboard, CERN-IT management C5 report  The list may not be complete!  I haven’t gathered feedback from sites  It’s difficult to identify all possible use cases !  Some interactions of the BDII may be hidden and only known to SW architects of a particular SW tool or framework  So please speak up during this meeting if you have an important use case to report! 18th June 2015 WLCG Information System Use Cases Review 2

ALICE use cases  ALICE relies on the following information published in the resource BDII of CREAM CEs and also the site BDII for ARC CEs:  Status of the CEs (Production vs anything else)  Number of waiting jobs in the VOView  Number of running jobs in the VOView 18th June 2015 WLCG Information System Use Cases Review 3

ATLAS use cases  ATLAS runs a BDII collector on AGIS to gather the following information:  List of CEs, submission queues and associated parameters  List of SEs, protocols, storage areas and paths  Site latitude and longitude  Batch system type and version  In many cases the information taken from the BDII is reviewed and updated by ATLAS 18th June 2015 WLCG Information System Use Cases Review 4

CMS use cases  CMS relies on the following information published in the Infosys:  List of CEs  This is used to generate  The Vofeed for the SAM tests  To bootstrap the glideinWMS Factory XML file  HTCondor CEs are not taken from the BDII  The list of HTCondor CEs is taken using OSG collectors 18th June 2015 WLCG Information System Use Cases Review 5

LHCb use cases  LHCb uses the information published in the Infosys to feed DIRAC:  List of CEs  CEs are discovered whenever they are deployed  The Max CPU Time and the CPU Scaling Reference attributes are also used  SW Tags published in the Infosys are also used  In particular the VO-lhcb-pilot SW Tag 18th June 2015 WLCG Information System Use Cases Review 6

SAM use cases  SAM relies on the following information published in the Infosys:  Queue name  Even if SAM now relies on the experiments Vofeed to get the list of resources, for CREAM CEs, ARC CEs and GRAMs still needs to get the queue name from the BDII  This is not provided in the VOfeed 18th June 2015 WLCG Information System Use Cases Review 7

REBUS use cases  Capacities are taken automatically from the information published in the Infosys  For OSG HEPSPEC is taken from MyOSG  Pledges are manually reported by sites  Pledges are used for the accounting reports  It’s not so clear who uses the capacity information (?) 18th June 2015 WLCG Information System Use Cases Review 8

Miscellanea  GFAL 2 has also a dependency on the Infosys:  BDII lookups can be disabled in all cases with DBDII:ENABLE=false, or modifying the corresponding configuration file (i.e. /etc/gfal2.d/bdii.conf)  However, if the BDII lookups are enabled and the SURL is not a full SURL, the BDII will be contacted  Weekly C5 report (used internally in CERN IT for the management) relies fully on the BDII  WLCG Google Earth Dashboard relies also on the BDII to obtain the site coordinates 18th June 2015 WLCG Information System Use Cases Review 9

GLUE 1 or GLUE 2?  All the use cases presented in these slides, query GLUE 1 information 18th June 2015 WLCG Information System Use Cases Review 10

A future without the Infosys  OSG is trying to get rid of the dependencies on the Infosys  Not all WLCG sites will be published in the BDII anymore  Incomplete picture!  Could WLCG also consider getting rid of the Infosys?  Is this the direction we want to go?  Could the presented use cases find alternative ways of getting the same information?  Maybe we can learn from the OSG experience!  One less service to maintain  But multi VO sites may still need to maintain the Infosys as it is used by other VOs  If we stop the Infosys, what happens with GLUE? how do we ensure interoperability to exchange information? 18th June 2015 WLCG Information System Use Cases Review 11