Mar 27, 20081 gLExec Accounting Solutions in OSG Gabriele Garzoglio gLExec Accounting Solutions in OSG Mar 27, 2008 Middleware Security Group Meeting Igor.

Slides:

Advertisements

Similar presentations

EGEE-II INFSO-RI Enabling Grids for E-sciencE The gLite middleware distribution OSG Consortium Meeting Seattle,

Advertisements

Dec 14, 20061/10 VO Services Project – Status Report Gabriele Garzoglio VO Services Project WBS Dec 14, 2006 OSG Executive Board Meeting Gabriele Garzoglio.

OSG Logging Architecture Update Center for Enabling Distributed Petascale Science Brian L. Tierney: LBNL.

LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.

The SAM-Grid Fabric Services Gabriele Garzoglio (for the SAM-Grid team) Computing Division Fermilab.

Ways to Connect to OSG Tuesday afternoon, 3:00 pm Lauren Michael Research Computing Facilitator University of Wisconsin-Madison.

OSG Site Provide one or more of the following capabilities: – access to local computational resources using a batch queue – interactive access to local.

FESR Consorzio COMETA Grid Introduction and gLite Overview Corso di formazione sul Calcolo Parallelo ad Alte Prestazioni (edizione.

Grid Job and Information Management (JIM) for D0 and CDF Gabriele Garzoglio for the JIM Team.

OSG Middleware Roadmap Rob Gardner University of Chicago OSG / EGEE Operations Workshop CERN June 19-20, 2006.

Publication and Protection of Site Sensitive Information in Grids Shreyas Cholia NERSC Division, Lawrence Berkeley Lab Open Source Grid.

Open Science Grid The OSG Accounting System: GRATIA by Philippe Canal (FNAL) & Matteo Melani (SLAC) Mumbai, India CHEP2006.

1 Evolution of OSG to support virtualization and multi-core applications (Perspective of a Condor Guy) Dan Bradley University of Wisconsin Workshop on.

May 8, 20071/15 VO Services Project – Status Report Gabriele Garzoglio VO Services Project – Status Report Overview and Plans May 8, 2007 Computing Division,

CHEP'07 September D0 data reprocessing on OSG Authors Andrew Baranovski (Fermilab) for B. Abbot, M. Diesburg, G. Garzoglio, T. Kurca, P. Mhashilkar.

G RID M IDDLEWARE AND S ECURITY Suchandra Thapa Computation Institute University of Chicago.

Apr 30, 20081/11 VO Services Project – Stakeholders’ Meeting Gabriele Garzoglio VO Services Project Stakeholders’ Meeting Apr 30, 2008 Gabriele Garzoglio.

PanDA Multi-User Pilot Jobs Maxim Potekhin Brookhaven National Laboratory Open Science Grid WLCG GDB Meeting CERN March 11, 2009.

Sep 21, 20101/14 LSST Simulations on OSG Sep 21, 2010 Gabriele Garzoglio for the OSG Task Force on LSST Computing Division, Fermilab Overview OSG Engagement.

Mine Altunay OSG Security Officer Open Science Grid: Security Gateway Security Summit January 28-30, 2008 San Diego Supercomputer Center.

SAMGrid as a Stakeholder of FermiGrid Valeria Bartsch Computing Division Fermilab.

Use of Condor on the Open Science Grid Chris Green, OSG User Group / FNAL Condor Week, April

Mar 28, 20071/9 VO Services Project Gabriele Garzoglio The VO Services Project Don Petravick for Gabriele Garzoglio Computing Division, Fermilab ISGC 2007.

1 OSG Accounting Service Requirements Matteo Melani SLAC for the OSG Accounting Activity.

May 11, 20091/17 VO Services Project – Stakeholders’ Meeting Gabriele Garzoglio VO Services Project Stakeholders’ Meeting May 11, 2009 Gabriele Garzoglio.

Mar 28, 20071/18 The OSG Resource Selection Service (ReSS) Gabriele Garzoglio OSG Resource Selection Service (ReSS) Don Petravick for Gabriele Garzoglio.

Jan 10, 20091/16 VO Services Project – Stakeholders’ Meeting Gabriele Garzoglio VO Services Project Stakeholders’ Meeting Jan 10, 2009 Gabriele Garzoglio.

Accounting in LCG Dave Kant CCLRC, e-Science Centre.

Mine Altunay July 30, 2007 Security and Privacy in OSG.

1 User Analysis Workgroup Discussion  Understand and document analysis models  Best in a way that allows to compare them easily.

Overview of Privilege Project at Fermilab (compilation of multiple talks and documents written by various authors) Tanya Levshina.

Open Science Grid (OSG) Introduction for the Ohio Supercomputer Center Open Science Grid (OSG) Introduction for the Ohio Supercomputer Center February.

LCG Accounting John Gordon Grid Deployment Board 13 th January 2004.

INFSO-RI Enabling Grids for E-sciencE EGEE is a project funded by the European Union under contract INFSO-RI Grid Accounting.

Oct 19, 20101/16 Adoption of a SAML-XACML Profile for Authorization Interoperability across Grid Middleware in OSG and EGEE CHEP 2010 Oct 19, 2010 Gabriele.

VO Privilege Activity. The VO Privilege Project develops and implements fine-grained authorization to grid- enabled resources and services Started Spring.

Glite. Architecture Applications have access both to Higher-level Grid Services and to Foundation Grid Middleware Higher-Level Grid Services are supposed.

INFSO-RI Enabling Grids for E-sciencE GridICE: Grid and Fabric Monitoring Integrated for gLite-based Sites Sergio Fantinel INFN.

High Energy FermiLab Two physics detectors (5 stories tall each) to understand smallest scale of matter Each experiment has ~500 people doing.

Jun 12, 20071/17 AuthZ Interoperability – Status and Plan Gabriele Garzoglio AuthZ Interoperability Status and Plans June 12, 2007 Middleware Security.

Auditing Project Architecture VERY HIGH LEVEL Tanya Levshina.

OSG Site Admin Workshop - Mar 2008Using gLExec to improve security1 OSG Site Administrators Workshop Using gLExec to improve security of Grid jobs by Alain.

Eileen Berman. Condor in the Fermilab Grid FacilitiesApril 30, 2008  Fermi National Accelerator Laboratory is a high energy physics laboratory outside.

EGEE-II INFSO-RI Enabling Grids for E-sciencE gLite and Condor present and future Claudio Grandi (INFN – Bologna)

DIRAC Pilot Jobs A. Casajus, R. Graciani, A. Tsaregorodtsev for the LHCb DIRAC team Pilot Framework and the DIRAC WMS DIRAC Workload Management System.

Sep 25, 20071/5 Grid Services Activities on Security Gabriele Garzoglio Grid Services Activities on Security Gabriele Garzoglio Computing Division, Fermilab.

OSG Area Coordinator’s Report: Workload Management Maxim Potekhin BNL May 8 th, 2008.

LCG WLCG Accounting: Update, Issues, and Plans John Gordon RAL Management Board, 19 December 2006.

INFSO-RI Enabling Grids for E-sciencE SAML-XACML interoperability Oscar Koeroo.

OSG Storage VDT Support and Troubleshooting Concerns Tanya Levshina.

Sep 17, 20081/16 VO Services Project – Stakeholders’ Meeting Gabriele Garzoglio VO Services Project Stakeholders’ Meeting Sep 17, 2008 Gabriele Garzoglio.

Feb 15, 20071/6 OSG EB Meeting – VO Services Status Gabriele Garzoglio VO Services Status OSG EB Meeting Feb 15, 2007 Gabriele Garzoglio, Fermilab.

Security and VO management enhancements in Panda Workload Management System Jose Caballero Maxim Potekhin Torre Wenaus Presented by Maxim Potekhin at HPDC08.

OSG Status and Rob Gardner University of Chicago US ATLAS Tier2 Meeting Harvard University, August 17-18, 2006.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Job Management Claudio Grandi.

Why you should care about glexec OSG Site Administrator’s Meeting Written by Igor Sfiligoi Presented by Alain Roy Hint: It’s about security.

Enabling Grids for E-sciencE Claudio Cherubino INFN DGAS (Distributed Grid Accounting System)

Panda Monitoring, Job Information, Performance Collection Kaushik De (UT Arlington), Torre Wenaus (BNL) OSG All Hands Consortium Meeting March 3, 2008.

June 5, 2006gLexec1 gLexec (within the OSG and Fermilab) June 5, 2006 Keith Chadwick.

Honolulu - Oct 31st, 2007 Using Glideins to Maximize Scientific Output 1 IEEE NSS 2007 Making Science in the Grid World - Using Glideins to Maximize Scientific.

How to integrate portals with EGI accounting system R.Graciani EGI TF 2012.

HTCondor Accounting Update

Argus EMI Authorization Integration

Job monitoring and accounting data visualization

Operating a glideinWMS frontend by Igor Sfiligoi (UCSD)

FermiGrid - PRIMA, VOMS, GUMS & SAZ

and Alexandre Duarte OurGrid/EELA Interoperability Meeting

f f FermiGrid – Site AuthoriZation (SAZ) Service

Argus Authorization Service Security Training

Wide Area Workload Management Work Package DATAGRID project

Presentation transcript:

Mar 27, gLExec Accounting Solutions in OSG Gabriele Garzoglio gLExec Accounting Solutions in OSG Mar 27, 2008 Middleware Security Group Meeting Igor Sfiligoi and Gabriele Garzoglio Computing Division, Fermilab Overview Pilot Jobs and gLExec in OSG Process Monitor orked by gLExec GUMS PlugIn Integration with the Gratia Accounting System

Mar 27, gLExec Accounting Solutions in OSG 2 Context A pilot job is submitted to a cluster The pilot job lands on a worker node The pilot job calls gLExec to execute user jobs –The pilot job interacts with the site security system via gLExec –Important to account the used resources to the proper user / fqan

Mar 27, gLExec Accounting Solutions in OSG 3 gLExec in OSG NIKHEF provides gLExec, LCAS and LCMAPS frameworks VO Services project provides the PRIMA/GUMS plugin –Contacts GUMS to get the correct mapping –Spawns a Monitor process for accounting

Mar 27, gLExec Accounting Solutions in OSG 4 gLExec GUMS PlugIn Monitor Process started by the GUMS plugin Logs user information to a local file –DN, VO, FQAN, parent processes Tracks the descendants of gLExec –Log info for long lived processes only (> 5min) –Periodically logs accounting info for CPU usage (Usr+Sys)

Mar 27, gLExec Accounting Solutions in OSG 5 Process Tracking Use Condor-provided “procd” –Distributed via VDT Use a secondary GID of the processes for tracking –In this model, the system reserves a range of sGIDs dedicated to process tracking –gLExec will set an sGID before starting the user job –Only superuser can change list of sGIDs Reliably tracks also daemonized processes

Mar 27, gLExec Accounting Solutions in OSG 6 GRATIA Usage of the Records GRATIA probe at the WN parses the gLExec GUMS Monitor log file Collects: –DN, VO, FQAN – wallclock time, Usr and Sys CPU time Information is sent to the Gratia Collector

Mar 27, gLExec Accounting Solutions in OSG 7 Summary OSG uses accounting information from gLExec –Accounting is provided by gLExec GUMS plugin… –…it would be better to move it elsewhere Reliable process tracking based on sGID added recently –Can properly account for daemonized processes