ECGI meeting on job priorities on May 15th 2006, CNAF Bologna How LHCb thinks to use/integrate g-PBox (or single components) and when Gianluca Castellani.

Slides:



Advertisements
Similar presentations
DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing AliEn and EDG 1/13 Stefano Bagnasco, INFN Torino Interfacing AliEn to EDG Stefano Bagnasco, INFN Torino.
Advertisements

29 June 2006 GridSite Andrew McNabwww.gridsite.org VOMS and VOs Andrew McNab University of Manchester.
1 User Analysis Workgroup Update  All four experiments gave input by mid December  ALICE by document and links  Very independent.
E-science grid facility for Europe and Latin America A Data Access Policy based on VOMS attributes in the Secure Storage Service Diego Scardaci.
Scheduling under LCG at RAL UK HEP Sysman, Manchester 11th November 2004 Steve Traylen
Stuart K. PatersonCHEP 2006 (13 th –17 th February 2006) Mumbai, India 1 from DIRAC.Client.Dirac import * dirac = Dirac() job = Job() job.setApplication('DaVinci',
DIRAC API DIRAC Project. Overview  DIRAC API  Why APIs are important?  Why advanced users prefer APIs?  How it is done?  What is local mode what.
INFSO-RI Enabling Grids for E-sciencE XACML and G-PBox update MWSG 14-15/09/2005 Presenter: Vincenzo Ciaschini.
CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.
LHCC Comprehensive Review – September WLCG Commissioning Schedule Still an ambitious programme ahead Still an ambitious programme ahead Timely testing.
INFSO-RI Enabling Grids for E-sciencE Logging and Bookkeeping and Job Provenance Services Ludek Matyska (CESNET) on behalf of the.
Belle MC Production on Grid 2 nd Open Meeting of the SuperKEKB Collaboration Soft/Comp session 17 March, 2009 Hideyuki Nakazawa National Central University.
Enabling Grids for E-sciencE System Analysis Working Group and Experiment Dashboard Julia Andreeva CERN Grid Operations Workshop – June, Stockholm.
Maarten Litmaath (CERN), GDB meeting, CERN, 2006/02/08 VOMS deployment Extent of VOMS usage in LCG-2 –Node types gLite 3.0 Issues Conclusions.
AN INTEGRATED FRAMEWORK FOR VO-ORIENTED AUTHORIZATION, POLICY-BASED MANAGEMENT AND ACCOUNTING Andrea Caltroni 3, Vincenzo Ciaschini 1, Andrea Ferraro 1,
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Next steps with EGEE EGEE training community.
Getting started DIRAC Project. Outline  DIRAC information system  Documentation sources  DIRAC users and groups  Registration with DIRAC  Getting.
Glexec, SCAS & CREAM. Milestones CREAM-CE capable of large-scale direct job submission Glexec & SCAS capable of large-scale use on WN in logging only.
1 User Analysis Workgroup Discussion  Understand and document analysis models  Best in a way that allows to compare them easily.
Conference name Company name INFSOM-RI Speaker name The ETICS Job management architecture EGEE ‘08 Istanbul, September 25 th 2008 Valerio Venturi.
Julia Andreeva, CERN IT-ES GDB Every experiment does evaluation of the site status and experiment activities at the site As a rule the state.
1 Andrea Sciabà CERN Critical Services and Monitoring - CMS Andrea Sciabà WLCG Service Reliability Workshop 26 – 30 November, 2007.
INFSO-RI Enabling Grids for E-sciencE G-PBox Auth meeting 13/9/2005 Presenter: Vincenzo Ciaschini.
OSG AuthZ components Dane Skow Gabriele Carcassi.
CHEP 2006, February 2006, Mumbai 1 LHCb use of batch systems A.Tsaregorodtsev, CPPM, Marseille HEPiX 2006, 4 April 2006, Rome.
Prototyping production and analysis frameworks for LHC experiments based on LCG, EGEE and INFN-Grid middleware CDF: DAG and Parametric Jobs ALICE: Evolution.
EMI INFSO-RI Argus Policies in Action Valery Tschopp (SWITCH) on behalf of the Argus PT.
INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks New Authorization Service Christoph Witzig,
Workload management, virtualisation, clouds & multicore Andrew Lahiff.
GRID Security & DIRAC A. Casajus R. Graciani A. Tsaregorodtsev.
DIRAC Pilot Jobs A. Casajus, R. Graciani, A. Tsaregorodtsev for the LHCb DIRAC team Pilot Framework and the DIRAC WMS DIRAC Workload Management System.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Update Authorization Service Christoph Witzig,
INFSO-RI Enabling Grids for E-sciencE Policy management and fair share in gLite Andrea Guarise HPDC 2006 Paris June 19th, 2006.
INFSO-RI Enabling Grids for E-sciencE SAML-XACML interoperability Oscar Koeroo.
LHCb Pilot Job Tests. We have not started this test due to several reasons: our analysis job submission (ganga) was not interfaced in time to DIRAC3,
Distributed Physics Analysis Past, Present, and Future Kaushik De University of Texas at Arlington (ATLAS & D0 Collaborations) ICHEP’06, Moscow July 29,
03/09/2007http://pcalimonitor.cern.ch/1 Monitoring in ALICE Costin Grigoras 03/09/2007 WLCG Meeting, CHEP.
Distributed Analysis Tutorial Dietrich Liko. Overview  Three grid flavors in ATLAS EGEE OSG Nordugrid  Distributed Analysis Activities GANGA/LCG PANDA/OSG.
INFN GRID Production Infrastructure Status and operation organization Cristina Vistoli Cnaf GDB Bologna, 11/10/2005.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks OpenSAML extension library and API to support.
WMS baseline issues in Atlas Miguel Branco Alessandro De Salvo Outline  The Atlas Production System  WMS baseline issues in Atlas.
INFSO-RI Enabling Grids for E-sciencE File Transfer Software and Service SC3 Gavin McCance – JRA1 Data Management Cluster Service.
Breaking the frontiers of the Grid R. Graciani EGI TF 2012.
EGEE-III INFSO-RI Enabling Grids for E-sciencE VO Authorization in EGEE Erwin Laure EGEE Technical Director Joint EGEE and OSG Workshop.
SAM architecture EGEE 07 Service Availability Monitor for the LHC experiments Simone Campana, Alessandro Di Girolamo, Nicolò Magini, Patricia Mendez Lorenzo,
DIRAC for Grid and Cloud Dr. Víctor Méndez Muñoz (for DIRAC Project) LHCb Tier 1 Liaison at PIC EGI User Community Board, October 31st, 2013.
LCG Accounting Update John Gordon, CCLRC-RAL 10/1/2007.
1 Grid security Services and Support Vincenzo Ciaschini, INFN CNAF V INFN-GRID workshop 18-20/12/2006.
EGEE is a project funded by the European Union under contract IST Report from the PTF Fabrizio Pacini Datamat S.p.a. Milan, IT-CZ JRA1 meeting,
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.
Architectural Framework Presentation Vincenzo Ciaschini CNAF 15/5/06.
Job Priorities and Resource sharing in CMS A. Sciabà ECGI meeting on job priorities 15 May 2006.
Accounting Review Summary and action list from the (pre)GDB Julia Andreeva CERN-IT WLCG MB 19th April
G-PBox Facts and status JRA1 Authz Coord Meeting January CNAF/INFN Bologna Andrea Ferraro.
DIRAC: Workload Management System Garonne Vincent, Tsaregorodtsev Andrei, Centre de Physique des Particules de Marseille Stockes-rees Ian, University of.
Jean-Philippe Baud, IT-GD, CERN November 2007
Regional Operations Centres Core infrastructure Centres
Summary on PPS-pilot activity on CREAM CE
GDB 8th March 2006 Flavia Donno IT/GD, CERN
CREAM Status and Plans Massimo Sgaravatto – INFN Padova
LHCb experience with GGUS and neighbors
Global Banning List and Authorization Service
Accounting at the T1/T2 Sites of the Italian Grid
LHCb Computing Model and Data Handling Angelo Carbone 5° workshop italiano sulla fisica p-p ad LHC 31st January 2008.
Grid Deployment Board meeting, 8 November 2006, CERN
Short update on the latest gLite status
Glexec/SCAS Pilot: IN2P3-CC status
QoS and SLA in INFN Grid INFN team: Andrea Ceccanti, Vincenzo Ciaschini, Alberto Forti, Andrea Ferraro, Valerio Venturi Location Catania (Italy) Date 4/3/2008.
The LHCb Computing Data Challenge DC06
Presentation transcript:

ECGI meeting on job priorities on May 15th 2006, CNAF Bologna How LHCb thinks to use/integrate g-PBox (or single components) and when Gianluca Castellani INFN/CNAF/LHCb

Outlines What we would like to have from g-PBox What we have done so far How we would like to use g-PBox Which are the advanges of using g-Pbox Conclusions and prospectives

What we would like to have from g-PBox A tool to obtain a priority number once a set of policies (parameters) are feed An user friendly interface to add or change policies on flight The possibility to have a g-Pox decouple from the RB

What we have done so far G.Rubini gave us an introductory presentation on g-PBox We reproduced “Atlas test”  Define two user G. Castellani (production), R.Santinelli (Analisys)  Production can submit its jobs everywhere  Analisys can submit its jobs only to queue at low priority

Some details about the test UI: cert-ui-01.cnaf.infn.it CE Mask:  gri47xl.to.infn.it  pre-ce-01.cnaf.infn.it  cclcgceli01.in2p3.fr  gridit-ce-001.cnaf.infn.it  prod-ce-01.pd.infn.it RB: cert-rb-04.cnaf.infn.it JDL: “sleep 3600” Each user submitted 500 jobs.

Outcomes of the “Atlas test” I 1. g-PBox correctly “enrouted” the jobs according to the owner’s group. 2. At the beginning of the submission (about in the first hour) the ratio between Production and Analisys job was about 3/1 (as expected) 1. Production uses 100% of high-priority queues and 50% of low-priority ones. 2. Analysys sees only 50% of low-priority queue 3. 3/1 = (CPU slots for Production)/(CPU slots for Analisys) = ( ) %/(50)%

Outcomes of the “Atlas test” II 1. g-PBox influences on submission time: about 3% 2. Problems: 1. Some minor configuration issues at CE or g-PBox level 2. A few critical gLite RB bugs were discovered and reported to the developers, at least a Savannah’s bug was opened: 3. It seemed that the CE were not only dedicated to our test (this not really a big issues…) 3. Costumer Support: 1. Mon/Fri 8h/19h 2. Cesini/Selmi were very cooperatives to fix any problem (also those caused by myself)

ECGI meeting on job priorities on May 15th 2006, CNAF Bologna UI VO G-Pbox POLICIES The Grid Work flow VOMS groupsJob lenght Dirac Accounting Site G-Pboxes Job Receiver JobDB PEP optimizer 1 3 Request 4 Decision 5 6 Priority Number + Site “obligations” 7 Matcher 8 Agent Director Pilot agents TASK QUEUEs 2 Pilot Per DN/group Site Obligations: Agent submission will also take into Account site policies (not only data) LFC Priority Number: It summarizes the VO wishes …….used For Task Queues reshuffling

Motivations for G-PBOX Extremely portable and modular system (easy to plug in). Dirac-WMS decoupled by the Policies repository and management system: dynamism. XACML extremely powerful and suitable for defining policies (whatever complexity) Offered facilities for editing/handling policies GSI compatible Strategically worth : VO policies automatically will take into account site policies in a transparent way. Dirac (for free) will be site-decision aware. VO policies are back propagated to the site. If a malicious user (willing to by pass Dirac) goes directly through LCG he will have always to face centrally defined policies

Conclusions and Prospectives Time Line  DC06: There is no way to implement g- PBox in DIRAC by June (2 weeks)  Build the infrastructure in DIRAC to be able to implement a policy enforcement  This will permit to plug in even an external policy decision box  Having and understanding the g-Pbox API will speed up the process of implementation