Shifters Jamboree Kaushik De ADC Jamboree, CERN December 4, 2014.

Slides:



Advertisements
Similar presentations
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
Advertisements

S. Gadomski, "ATLAS computing in Geneva", journee de reflexion, 14 Sept ATLAS computing in Geneva Szymon Gadomski description of the hardware the.
Workload Management Workpackage Massimo Sgaravatto INFN Padova.
Workload Management Massimo Sgaravatto INFN Padova.
Biometric Daily Time Record
1 Bridging Clouds with CernVM: ATLAS/PanDA example Wenjing Wu
Slide 1 of 9 Presenting 24x7 Scheduler The art of computer automation Press PageDown key or click to advance.
1 Status of the ALICE CERN Analysis Facility Marco MEONI – CERN/ALICE Jan Fiete GROSSE-OETRINGHAUS - CERN /ALICE CHEP Prague.
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES News on monitoring for CMS distributed computing operations Andrea.
LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.
The Panda System Mark Sosebee (for K. De) University of Texas at Arlington dosar workshop March 30, 2006.
CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Job Monitoring for the LHC experiments Irina Sidorova (CERN, JINR) on.
PanDA Multi-User Pilot Jobs Maxim Potekhin Brookhaven National Laboratory Open Science Grid WLCG GDB Meeting CERN March 11, 2009.
PanDA A New Paradigm for Computing in HEP Kaushik De Univ. of Texas at Arlington NRC KI, Moscow January 29, 2015.
FAX UPDATE 1 ST JULY Discussion points: FAX failover summary and issues Mailing issues Panda re-brokering to sites using FAX cost and access Issue.
Use of Condor on the Open Science Grid Chris Green, OSG User Group / FNAL Condor Week, April
OFFLINE TRIGGER MONITORING TDAQ Training 5 th November 2010 Ricardo Gonçalo On behalf of the Trigger Offline Monitoring Experts team.
Review of Condor,SGE,LSF,PBS
Grid User Interface for ATLAS & LHCb A more recent UK mini production used input data stored on RAL’s tape server, the requirements in JDL and the IC Resource.
A PanDA Backend for the Ganga Analysis Interface J. Elmsheuser 1, D. Liko 2, T. Maeno 3, P. Nilsson 4, D.C. Vanderster 5, T. Wenaus 3, R. Walker 1 1: Ludwig-Maximilians-Universität.
6/23/2005 R. GARDNER OSG Baseline Services 1 OSG Baseline Services In my talk I’d like to discuss two questions:  What capabilities are we aiming for.
EGEE-III INFSO-RI Enabling Grids for E-sciencE Ricardo Rocha CERN (IT/GS) EGEE’08, September 2008, Istanbul, TURKEY Experiment.
PanDA Status Report Kaushik De Univ. of Texas at Arlington ANSE Meeting, Nashville May 13, 2014.
INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.
Data Management: US Focus Kaushik De, Armen Vartapetian Univ. of Texas at Arlington US ATLAS Facility, SLAC Apr 7, 2014.
PanDA & BigPanDA Kaushik De Univ. of Texas at Arlington BigPanDA Workshop, CERN October 21, 2013.
Computing Facilities CERN IT Department CH-1211 Geneva 23 Switzerland t CF Agile Infrastructure Monitoring HEPiX Spring th April.
Testing and integrating the WLCG/EGEE middleware in the LHC computing Simone Campana, Alessandro Di Girolamo, Elisa Lanciotti, Nicolò Magini, Patricia.
Tier3 monitoring. Initial issues. Danila Oleynik. Artem Petrosyan. JINR.
Julia Andreeva on behalf of the MND section MND review.
EMI INFSO-RI Argus The EMI Authorization Service Valery Tschopp (SWITCH) Argus Product Team.
DIRAC Pilot Jobs A. Casajus, R. Graciani, A. Tsaregorodtsev for the LHCb DIRAC team Pilot Framework and the DIRAC WMS DIRAC Workload Management System.
Conclusions on Monitoring CERN A. Read ADC Monitoring1.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Monitoring of the LHC Computing Activities Key Results from the Services.
OSG Area Coordinator’s Report: Workload Management Maxim Potekhin BNL May 8 th, 2008.
Global ADC Job Monitoring Laura Sargsyan (YerPhI).
Pavel Nevski DDM Workshop BNL, September 27, 2006 JOB DEFINITION as a part of Production.
1. 2 Overview Extremely short summary of the physical part of the conference (I am not a physicist, will try my best) Overview of the Grid session focused.
Distributed Physics Analysis Past, Present, and Future Kaushik De University of Texas at Arlington (ATLAS & D0 Collaborations) ICHEP’06, Moscow July 29,
Distributed Analysis Tutorial Dietrich Liko. Overview  Three grid flavors in ATLAS EGEE OSG Nordugrid  Distributed Analysis Activities GANGA/LCG PANDA/OSG.
Grid Activities in CMS Asad Samar (Caltech) PPDG meeting, Argonne July 13-14, 2000.
Future of Distributed Production in US Facilities Kaushik De Univ. of Texas at Arlington US ATLAS Distributed Facility Workshop, Santa Cruz November 13,
ATLAS Distributed Analysis DISTRIBUTED ANALYSIS JOBS WITH THE ATLAS PRODUCTION SYSTEM S. González D. Liko
INFSOM-RI ETICS: E-infrastructure for Testing, Integration and Configuration of Software Alberto Di Meglio Project Manager.
A Validation System for the Complex Event Processing Directives of the ATLAS Shifter Assistant Tool G. Anders (CERN), G. Avolio (CERN), A. Kazarov (PNPI),
ATLAS Distributed Computing ATLAS session WLCG pre-CHEP Workshop New York May 19-20, 2012 Alexei Klimentov Stephane Jezequel Ikuo Ueda For ATLAS Distributed.
Analysis tools: JEDI client and BigPandaMon Torre Wenaus, Tadashi Maeno BNL ADC TIM, Chicago Oct 27, 2014.
Meeting with University of Malta| CERN, May 18, 2015 | Predrag Buncic ALICE Computing in Run 2+ P. Buncic 1.
Danila Oleynik (On behalf of ATLAS collaboration)
Breaking the frontiers of the Grid R. Graciani EGI TF 2012.
CMS Experience with the Common Analysis Framework I. Fisk & M. Girone Experience in CMS with the Common Analysis Framework Ian Fisk & Maria Girone 1.
Geant4 GRID production Sangwan Kim, Vu Trong Hieu, AD At KISTI.
G. Russo, D. Del Prete, S. Pardi Kick Off Meeting - Isola d'Elba, 2011 May 29th–June 01th A proposal for distributed computing monitoring for SuperB G.
Seven things you should know about Ganga K. Harrison (University of Cambridge) Distributed Analysis Tutorial ATLAS Software & Computing Workshop, CERN,
ADC Operations Shifts J. Yu Guido Negri, Alexey Sedov, Armen Vartapetian and Alden Stradling coordination, ADCoS coordination and DAST coordination.
Production System 2 manpower and funding issues Alexei Klimentov Brookhaven National Laboratory Aug 19, 2013 Production System Technical Meeting CERN.
Job submission overview Marco Mambelli – August OSG Summer Workshop TTU - Lubbock, TX THE UNIVERSITY OF CHICAGO.
J. Templon Nikhef Amsterdam Physics Data Processing Group Large Scale Computing Jeff Templon Nikhef Jamboree, Utrecht, 10 december 2012.
Monitoring Evolution and IPv6
Computing Operations Roadmap
U.S. ATLAS Grid Production Experience
Progress on NA61/NA49 software virtualisation Dag Toppe Larsen Wrocław
ATLAS Cloud Operations
PanDA setup at ORNL Sergey Panitkin, Alexei Klimentov BNL
POW MND section.
David Cameron ATLAS Site Jamboree, 20 Jan 2017
New monitoring applications in the dashboard
Readiness of ATLAS Computing - A personal view
The ADC Operations Story
Cloud Computing R&D Proposal
Presentation transcript:

Shifters Jamboree Kaushik De ADC Jamboree, CERN December 4, 2014

Introduction  Why should you attend Shifter’s Jamboree?  Maybe answer is obvious, since you are already here!  Being a shifter provides very useful training to become site expert, cloud expert, analysis expert – in fact ATLAS data expert  Being a ADC shifter is a big contribution to the success of ATLAS  It also provides Class 2 OTP credit  It is fun and easy to do – as you will hear today and tomorrow Dec 4, 2014 Kaushik De 2

Overview of Tutorials  Dec 4 afternoon:  ADC tutorial on new tools, features, documentation  General information useful for shifters, facilities experts, others  Interactions between operations people and developers  Dec 5 afternoon:  Overview of shifter activities  Viewpoint from shifters, interactions with GDP  Facilities experts should attend – shifters interact with them daily  Acronyms  ADCoS – ATLAS Distributed Computing operations Shifts  DAST – Distributed Analysis Shift Team  GDP – Grid Data Processing group Dec 4, 2014 Kaushik De 3

ADC Shifts  Scope of ADC shifts  ADCoS – for smooth ADC operations  DAST – to help analysis users  Both teams are essential for first line of support between thousands of users, millions of jobs daily and the hundreds of sites where jobs are run, data is stored  Both teams – triage problems, quick solutions, not to fix problems (which is done by site and tool experts)  No more for Run 2  Interactions with experts and squads: clouds, sites, GDP, software validation, PanDA… Dec 4, 2014 Kaushik De 4

ADC Tools  This afternoon will be tutorial on ADC tools  ProdSys2  Two components: DEfT, JEDI  No talk on DEfT today – while DEfT is central for task management, it is mainly used (and under commissioning) by experts  Task monitoring covered in BigPandaMon talk (Michal)  Rucio  Two talks (Martin, Kai/Cedric)  Very busy with commissioning  PanDA  Summary talk, including JEDI (Tadashi)  Central Services, Site Testing  Two talks (Jarka, Alexey) Dec 4, 2014 Kaushik De 5

ProdSys2 Dec 4, 2014 Kaushik De 6 Production Manager DEfT PanDA User JEDI User Task Job

PanDA Dec 4, 2014 site A Panda/JEDI server site B pilot Worker Nodes condor-g Autopilot https submit pull End-user submit Task, job pilot task job logger http send log DEfT Rucio Kaushik De7

Dec 4, 2014 A Few Words on PanDA PanDA WMS design goals: PanDA WMS design goals: Achieve high level of automation to reduce operational effort Flexibility in adapting to changing hardware and network Support diverse and changing middleware Insulate user from hardware, middleware, and all other complexities of the underlying system Unified system for production and user analysis Key features of PanDA Key features of PanDA Central job queue Pilot based job execution system Fairshare or policy driven priorities for thousands of users at hundreds of resources Automatic error handling and recovery Extensive monitoring Modular design 8 Kaushik De

Reminder: Tonight dinner is at Ristorante Luigia Careful, there are 2 of them in Geneva (and from 2016 one in Peru, see We go at the one in Petit Saconnex at 19:45 (Reservation: Campana)