WLCG Transfers Dashboard

Slides:

Advertisements

Similar presentations

WLCG Monitoring Consolidation NEC`2013, Varna Julia Andreeva CERN IT-SDC.

Advertisements

O. Keeble, F. Furano, A. Manzi, A. Ayllon, I. Calvet, M. Hellmich DPM Workshop 2014 DPM Monitoring.

Sergey Belov, Tatiana Goloskokova, Vladimir Korenkov, Nikolay Kutovskiy, Danila Oleynik, Artem Petrosyan, Roman Semenov, Alexander Uzhinskiy LIT JINR The.

LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.

Elasticsearch in Dashboard Data Management Applications David Tuckett IT/SDC 30 August 2013 (Appendix 11 November 2013)

Experience of xrootd monitoring for ALICE at RDIG sites G.S. Shabratova JINR A.K. Zarochentsev SPbSU.

ATLAS Off-Grid sites (Tier-3) monitoring A. Petrosyan on behalf of the ATLAS collaboration GRID’2012, , JINR, Dubna.

ATLAS DQ2 Deletion Service D.A. Oleynik, A.S. Petrosyan, V. Garonne, S. Campana (on behalf of the ATLAS Collaboration)

Julia Andreeva. \ Monitoring of the job processing Analysis Production Real time and historical views Users Opera- tion teams Sites Data management monitoring.

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services GS group meeting Monitoring and Dashboards section Activity.

Enabling Grids for E-sciencE Overview of System Analysis Working Group Julia Andreeva CERN, WLCG Collaboration Workshop, Monitoring BOF session 23 January.

FAX UPDATE 26 TH AUGUST Running issues FAX failover Moving to new AMQ server Informing on endpoint status Monitoring developments Monitoring validation.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks The network monitoring in grid context Operations.

Xrootd Monitoring for the CMS Experiment Abstract: During spring and summer 2011 CMS deployed Xrootd front- end servers on all US T1 and T2 sites. This.

And Tier 3 monitoring Tier 3 Ivan Kadochnikov LIT JINR

EGEE-III INFSO-RI Enabling Grids for E-sciencE Overview of STEP09 monitoring issues Julia Andreeva, IT/GS STEP09 Postmortem.

Dashboard program of work Julia Andreeva GS Group meeting

Storage cleaner: deletes files on mass storage systems. It depends on the results of deletion, files can be set in states: deleted or to repeat deletion.

Julia Andreeva, CERN IT-ES GDB Every experiment does evaluation of the site status and experiment activities at the site As a rule the state.

XROOTD AND FEDERATED STORAGE MONITORING CURRENT STATUS AND ISSUES A.Petrosyan, D.Oleynik, J.Andreeva Creating federated data stores for the LHC CC-IN2P3,

Backing Up and Restoring Databases by Using the SQL Server 2000.

CERN IT Department CH-1211 Geneva 23 Switzerland t A proposal for improving Job Reliability Monitoring GDB 2 nd April 2008.

ATP Future Directions Availability of historical information for grid resources: It is necessary to store the history of grid resources as these resources.

Julia Andreeva on behalf of the MND section MND review.

Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES Andrea Sciabà Hammercloud and Nagios Dan Van Der Ster Nicolò Magini.

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI How to integrate portals with the EGI monitoring system Dusan Vudragovic.

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Monitoring of the LHC Computing Activities Key Results from the Services.

DDM FAX Dashboard status and future Luca Magnoni IT/SDC 2 nd June 2014.

MND review. Main directions of work  Development and support of the Experiment Dashboard Applications - Data management monitoring - Job processing monitoring.

Global ADC Job Monitoring Laura Sargsyan (YerPhI).

Import XRootD monitoring data from MonALISA Sergey Belov, JINR, Dubna DNG section meeting,

Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES Author etc Dashboard Latency monitoring (Update) Alexandre Beche.

Federated Data Stores Volume, Velocity & Variety Future of Big Data Management Workshop Imperial College London June 27-28, 2013 Andrew Hanushevsky, SLAC.

03/09/2007http://pcalimonitor.cern.ch/1 Monitoring in ALICE Costin Grigoras 03/09/2007 WLCG Meeting, CHEP.

Network integration with PanDA Artem Petrosyan PanDA UTA,

Enabling Grids for E-sciencE CMS/ARDA activity within the CMS distributed system Julia Andreeva, CERN On behalf of ARDA group CHEP06.

WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers. Alexandre Beche.

SUM like functionality with WLCG-MON Ivan Dzhunov.

ATLAS Off-Grid sites (Tier-3) monitoring A. Petrosyan on behalf of the ATLAS collaboration GRID’2012, , JINR, Dubna.

XRootD Monitoring Report A.Beche D.Giordano. Outlines  Talk 1: XRootD Monitoring Dashboard  Context  Dataflow and deployment model  Database: storage.

WLCG Accounting Task Force Update Julia Andreeva CERN GDB, 8 th of June,

Slide 1 © 2016, Lera Technologies. All Rights Reserved. Oracle Data Integrator By Lera Technologies.

WLCG Transfers monitoring EGI Technical Forum Madrid, 17 September 2013 Pablo Saiz on behalf of the Dashboard Team CERN IT/SDC.

The Grid Information System Maria Alandes Pradillo IT-SDC White Area Lecture, 4th June 2014.

Accounting Review Summary and action list from the (pre)GDB Julia Andreeva CERN-IT WLCG MB 19th April

Test Title Test Content.

WLCG Accounting Task Force Introduction Julia Andreeva CERN 9 th of June,

Transition to EGI PSC-06 Istanbul Ioannis Liabotis Greece GRNET

Daniele Bonacorsi Andrea Sciabà

WLCG Workshop 2017 [Manchester] Operations Session Summary

Update on CERN IT Unified Monitoring Architecture (UMA)

ATLAS Grid Information System

RDMS CMS Computing Activities: current status & participation in ARDA

WLCG Resources Reporting

Key Activities. MND sections

PanDA setup at ORNL Sergey Panitkin, Alexei Klimentov BNL

POW MND section.

Evolution of SAM in an enhanced model for monitoring the WLCG grid

FTS Monitoring Ricardo Rocha

Experiment Dashboard overviw of the applications

Artem Petrosyan (JINR), Danila Oleynik (JINR), Julia Andreeva (CERN)

The ADC Operations Story

Advancements in Availability and Reliability computation Introduction and current status of the Comp Reports mini project C. Kanellopoulos GRNET.

Monitoring Of XRootD Federation

New Xrootd Monitoring Stream

Discussions on group meeting

Monitoring of the infrastructure from the VO perspective

Red Sky Update “Watching the horizon for emerging health threats”

Presentation transcript:

WLCG Transfers Dashboard Integration of XRootD Julia Andreeva, Sergey Belov, Alexandre Beche, Daniel Dieguez Arias, Domenico Giordano, Danila Oleynik, Artem Petrosyan, Matevz Tadel, David Tucket, Sergey Mitsyn

Contents First Step Current Status Work done for XRootD integration Statistics architecture New filters in the WLCG UI New views in the WLCG UI XRootD topology Next steps (XRootD monitoring) WLCG Transfers Dashboard - 2

First step WLCG Transfers Dashboard Cross-VO, cross-technology view Implementation started with FTS monitoring Dashboard MSG Broker API UI FTS instance Xrootd etc VO Monitoring WLCG Transfers Dashboard - 3

Current status WLCG Transfers Dashboard Cross-VO, cross-technology view Now XRootD technology is monitored too. Dashboard MSG Broker API UI FTS instance Xrootd/dCache etc VO Monitoring XRootD WLCG Transfers Dashboard - 4

Work done for XRootD integration New collector to consume XRootD messages (based in the FTS collector) New table and statistic procedures for XRootD messages New features in the UI: New filters by (technology and access type) New views (server, technology and access type) Static topology for sites (Extracted from OIM/GOCDB) Currently data is recorded on Dashboard Integration DB Presentation Title - 5

Statistics Architecture FTS transfers XRootD transfers Stored procedures Statistics 10 m Bins of 10 minutes UI Stored procedures Statistics 24 h Bins of 24 hours Presentation Title - 6

New filters in the WLCG UI Left panel in the UI Filter by technologies Filter by acces type Presentation Title - 7

New views in the WLCG UI Technology view Presentation Title - 8

New views in the WLCG UI Access type view Presentation Title - 9

New views in the WLCG UI Server view Presentation Title - 10

XRootD topology Presentation Title - 11

Next steps (XRootD monitoring) Validate content of XRootD transfers (subscribe to UCSD ML repository or using Gratia reports). Extend functionality following the list of requirements from people running XRootD federation Find a way to recover automatically the mapping for creating the XRootD topology (like current curl data service for countries). Presentation Title - 12

Questions http://dashb-wlcg-transfers-dev.cern.ch/ui/ Presentation Title - 13