Dashboard program of work Julia Andreeva GS Group meeting 28.07.09.

Slides:

Advertisements

Similar presentations

1 User Analysis Workgroup Update  All four experiments gave input by mid December  ALICE by document and links  Very independent.

Advertisements

WLCG Monitoring Consolidation NEC`2013, Varna Julia Andreeva CERN IT-SDC.

A tool to enable CMS Distributed Analysis

Analysis demos from the experiments. Analysis demo session Introduction –General information and overview CMS demo (CRAB) –Georgia Karapostoli (Athens.

LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.

CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.

Input from CMS Nicolò Magini Andrea Sciabà IT/SDC 5 July 2013.

CERN IT Department CH-1211 Geneva 23 Switzerland t The Experiment Dashboard ISGC th April 2008 Pablo Saiz, Julia Andreeva, Benjamin.

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services GS group meeting Monitoring and Dashboards section Activity.

CERN IT Department CH-1211 Genève 23 Switzerland t EIS section review of recent activities Harry Renshall Andrea Sciabà IT-GS group meeting.

Enabling Grids for E-sciencE Overview of System Analysis Working Group Julia Andreeva CERN, WLCG Collaboration Workshop, Monitoring BOF session 23 January.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks VO-specific systems for the monitoring of.

EGEE-III INFSO-RI Enabling Grids for E-sciencE Julia Andreeva CERN (IT/GS) CHEP 2009, March 2009, Prague New job monitoring strategy.

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Job Monitoring for the LHC experiments Irina Sidorova (CERN, JINR) on.

Cracow Grid Workshop October 2009 Dipl.-Ing. (M.Sc.) Marcus Hilbrich Center for Information Services and High Performance.

David Adams ATLAS ADA, ARDA and PPDG David Adams BNL June 28, 2004 PPDG Collaboration Meeting Williams Bay, Wisconsin.

CERN IT Department CH-1211 Genève 23 Switzerland t Monitoring: Tracking your tasks with Task Monitoring PAT eLearning – Module 11 Edward.

Enabling Grids for E-sciencE System Analysis Working Group and Experiment Dashboard Julia Andreeva CERN Grid Operations Workshop – June, Stockholm.

EGEE-III INFSO-RI Enabling Grids for E-sciencE Overview of STEP09 monitoring issues Julia Andreeva, IT/GS STEP09 Postmortem.

DDM Monitoring David Cameron Pedro Salgado Ricardo Rocha.

Stefano Belforte INFN Trieste 1 Middleware February 14, 2007 Resource Broker, gLite etc. CMS vs. middleware.

EGI-InSPIRE EGI-InSPIRE RI DDM Site Services winter release Fernando H. Barreiro Megino (IT-ES-VOS) ATLAS SW&C Week November

MW Readiness WG Update Andrea Manzi Maria Dimou Lionel Cons 10/12/2014.

1 User Analysis Workgroup Discussion  Understand and document analysis models  Best in a way that allows to compare them easily.

Julia Andreeva, CERN IT-ES GDB Every experiment does evaluation of the site status and experiment activities at the site As a rule the state.

WLCG Monitoring Roadmap Julia Andreeva, CERN , WLCG workshop, CERN.

Network awareness and network as a resource (and its integration with WMS) Artem Petrosyan (University of Texas at Arlington) BigPanDA Workshop, CERN,

Monitoring for CCRC08, status and plans Julia Andreeva, CERN , F2F meeting, CERN.

EGEE-III INFSO-RI Enabling Grids for E-sciencE Ricardo Rocha CERN (IT/GS) EGEE’08, September 2008, Istanbul, TURKEY Experiment.

PanDA Status Report Kaushik De Univ. of Texas at Arlington ANSE Meeting, Nashville May 13, 2014.

INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.

XROOTD AND FEDERATED STORAGE MONITORING CURRENT STATUS AND ISSUES A.Petrosyan, D.Oleynik, J.Andreeva Creating federated data stores for the LHC CC-IN2P3,

Julia Andreeva on behalf of the Dashboard team. 45 machines (14 dedicated to cms) 9% still on SLC4 (two weeks ago 30%) Services.

Visualization Ideas for Management Dashboards

Korea Workshop May GAE CMS Analysis (Example) Michael Thomas (on behalf of the GAE group)

CERN IT Department CH-1211 Geneva 23 Switzerland t A proposal for improving Job Reliability Monitoring GDB 2 nd April 2008.

ATP Future Directions Availability of historical information for grid resources: It is necessary to store the history of grid resources as these resources.

Julia Andreeva on behalf of the MND section MND review.

K. Harrison CERN, 22nd September 2004 GANGA: ADA USER INTERFACE - Ganga release status - Job-Options Editor - Python support for AJDL - Job Builder - Python.

Service Availability Monitor tests for ATLAS Current Status Tests in development To Do Alessandro Di Girolamo CERN IT/PSS-ED.

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI User-centric monitoring of the analysis and production activities within.

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Monitoring of the LHC Computing Activities Key Results from the Services.

Historical view for CMS job monitoring CERN, Summer student 2009 Student: PAVEL ŠIKTOROV Supervisor: JULIA ANDREEVA.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Grid Monitoring Tools E. Imamagic, SRCE CE.

MND review. Main directions of work  Development and support of the Experiment Dashboard Applications - Data management monitoring - Job processing monitoring.

Global ADC Job Monitoring Laura Sargsyan (YerPhI).

Pavel Nevski DDM Workshop BNL, September 27, 2006 JOB DEFINITION as a part of Production.

Enabling Grids for E-sciencE Grid monitoring from the VO/User perspective. Dashboard for the LHC experiments Julia Andreeva CERN, IT/PSS.

Enabling Grids for E-sciencE CMS/ARDA activity within the CMS distributed system Julia Andreeva, CERN On behalf of ARDA group CHEP06.

New solutions for large scale functional tests in the WLCG infrastructure with SAM/Nagios: The experiments experience ES IT Department CERN J. Andreeva.

Distributed Analysis Tutorial Dietrich Liko. Overview  Three grid flavors in ATLAS EGEE OSG Nordugrid  Distributed Analysis Activities GANGA/LCG PANDA/OSG.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Response of the ATLAS Spanish Tier2 for.

CERN - IT Department CH-1211 Genève 23 Switzerland t Grid Reliability Pablo Saiz On behalf of the Dashboard team: J. Andreeva, C. Cirstoiu,

Prototype of new Site Usability interface Amol Wakankar March, /1/20111.

SAM Status Update Piotr Nyczyk LCG Management Board CERN, 5 June 2007.

MND section. Summary of activities Job monitoring In collaboration with GridView and LB teams enabled full chain from LB harvester via MSG to Dashboard.

Ian Bird LCG Project Leader Status of EGEE  EGI transition WLCG LHCC Referees’ meeting 21 st September 2009.

Monitoring the Readiness and Utilization of the Distributed CMS Computing Facilities XVIII International Conference on Computing in High Energy and Nuclear.

News from the HEPiX IPv6 Working Group David Kelsey (STFC-RAL) HEPIX, BNL 13 Oct 2015.

WLCG Accounting Task Force Update Julia Andreeva CERN GDB, 8 th of June,

Efi.uchicago.edu ci.uchicago.edu Sharing Network Resources Ilija Vukotic Computation and Enrico Fermi Institutes University of Chicago Federated Storage.

WLCG Transfers monitoring EGI Technical Forum Madrid, 17 September 2013 Pablo Saiz on behalf of the Dashboard Team CERN IT/SDC.

Availability of ALICE Grid resources in Germany Kilian Schwarz GSI Darmstadt ALICE Offline Week.

Daniele Bonacorsi Andrea Sciabà

Key Activities. MND sections

ALICE Monitoring

POW MND section.

Update on Plan for KISTI-GSDC

Experiment Dashboard overviw of the applications

Monitoring of the infrastructure from the VO perspective

Presentation transcript:

Dashboard program of work Julia Andreeva GS Group meeting

STEP09 follow up (ATLAS) ATLAS Data management: 1.Monitoring of broken subscriptions 2. Monitoring of queues of subscriptions 3. Monitoring of subscriptions not picked up 4. Information ordered by source 5. Development of drill down plots giving efficiency and bandwidth consumed in a given time period 6. Some work on the pre-stage monitoring, especially for staged files and datasets Ricardo is working on enabling of this functionality. Should be in place by mid October.

STEP09 follow up (ATLAS) Improvements are required for analysis job monitoring. Because of the differences in the CMS and ATLAS job monitoring schemas and lack of Dashboard reporting from the ATLAS analysis jobs steady progress done for the CMS analysis monitoring was not applied to the ATLAS Dashboard job monitoring instance. The plan is to upgrade ATLAS schema to make it consistent with the CMS one (Irina). Instrument Ganga and hopefully PANDA jobs for Dashboard reporting. Reporting is foreseen via MSG using library developed by Maciej. Create collectors from MSG to Dashboard DB (work already in progress, Irina).New fellow joining the group in the beginning of September should take care about instrumentation of the ATLAS analysis jobs and should adapt existing CMS applications for ATLAS (with the help of the primary authors of the CMS applications). (Time estimation is 2-3 months)

STEP09 follow up (CMS) No changes were introduces to the Dashboard program of work after analysis of STEP09. Currently the main goal is to provide reliable, complete and user-friendly monitoring for analysis support team and user community. New features foreseen – creating MonAlisa repostory for CPU, memory, wall clock consumption, IO, etc.. for analysis jobs. This runtime information should be linked from the Task Monitoring application. Close interaction with the CMS CRAB and ProdAgent development teams, Analysis support teams and MonAlisa developers.

Other directions of work Enabling historical distributions (quality and numerical metrics) for SiteView (Daniel) Enabling Dashboard monitoring of ALICE jobs submitted to CREAM CE (Timur) FTS monitoring (Ricardo, for 4 experiments, first prototype should be ready by the end of September for demo in Barcelona) CMS job monitoring with Google Earth for demo in Barcelona. Upgrade of the Site Availability application used by all four experiments to make it compatible with the new SAM DB schema.