OSG End User Tools Overview OSG Grid school – March 19, 2009 Marco Mambelli - University of Chicago A brief summary about the system.

Slides:

Advertisements

Similar presentations

EGEE-II INFSO-RI Enabling Grids for E-sciencE The gLite middleware distribution OSG Consortium Meeting Seattle,

Advertisements

FP7-INFRA Enabling Grids for E-sciencE EGEE Induction Grid training for users, Institute of Physics Belgrade, Serbia Sep. 19, 2008.

Role Based VO Authorization Services Ian Fisk Gabriele Carcassi July 20, 2005.

JSAGA2 Overview job desc. gLite plug-ins Globus plug-ins JSAGA hidemiddlewareheterogeneity (e.g. gLite, Globus, Unicore) JDLRSL.

 Contributing >30% of throughput to ATLAS and CMS in Worldwide LHC Computing Grid  Reliant on production and advanced networking from ESNET, LHCNET and.

Plateforme de Calcul pour les Sciences du Vivant SRB & gLite V. Breton.

OSG AuthZ Architecture AuthZ Components Legend VO Management Services Grid Site GUMS Site Services SAZ CE Gatekeeper Prima Is Auth? Yes / No SE SRM gPlazma.

Implementing Finer Grained Authorization in the Open Science Grid Gabriele Carcassi, Ian Fisk, Gabriele, Garzoglio, Markus Lorch, Timur Perelmutov, Abhishek.

The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Services Abderrahman El Kharrim

GRID workload management system and CMS fall production Massimo Sgaravatto INFN Padova.

Globus Toolkit 4 hands-on Gergely Sipos, Gábor Kecskeméti MTA SZTAKI

The Globus Toolkit Gary Jackson. Introduction The Globus Toolkit is a product of the Globus Alliance ( It is middleware for developing.

GRID Workload Management System Massimo Sgaravatto INFN Padova.

Grid Services at NERSC Shreyas Cholia Open Software and Programming Group, NERSC NERSC User Group Meeting September 17, 2007.

Makrand Siddhabhatti Tata Institute of Fundamental Research Mumbai 17 Aug

Open Science Grid Software Stack, Virtual Data Toolkit and Interoperability Activities D. Olson, LBNL for the OSG International.

NAREGI WP4 (Data Grid Environment) Hideo Matsuda Osaka University.

OSG Services at Tier2 Centers Rob Gardner University of Chicago WLCG Tier2 Workshop CERN June 12-14, 2006.

OSG Site Provide one or more of the following capabilities: – access to local computational resources using a batch queue – interactive access to local.

OSG Middleware Roadmap Rob Gardner University of Chicago OSG / EGEE Operations Workshop CERN June 19-20, 2006.

INFSO-RI Enabling Grids for E-sciencE The US Federation Miron Livny Computer Sciences Department University of Wisconsin – Madison.

Grids and Portals for VLAB Marlon Pierce Community Grids Lab Indiana University.

Grid Resource Allocation and Management (GRAM) Execution management Execution management –Deployment, scheduling and monitoring Community Scheduler Framework.

Computational grids and grids projects DSS,

How to Install and Use the DQ2 User Tools US ATLAS Tier2 workshop at IU June 20, Bloomington, IN Marco Mambelli University of Chicago.

G RID M IDDLEWARE AND S ECURITY Suchandra Thapa Computation Institute University of Chicago.

L ABORATÓRIO DE INSTRUMENTAÇÃO EM FÍSICA EXPERIMENTAL DE PARTÍCULAS Enabling Grids for E-sciencE Grid Computing: Running your Jobs around the World.

PanDA Multi-User Pilot Jobs Maxim Potekhin Brookhaven National Laboratory Open Science Grid WLCG GDB Meeting CERN March 11, 2009.

Overview of Monitoring and Information Systems in OSG MWGS08 - September 18, Chicago Marco Mambelli - University of Chicago

Grid Compute Resources and Job Management New Mexico Grid School – April 9, 2009 Marco Mambelli – University of Chicago

Open Science Grid OSG CE Quick Install Guide Siddhartha E.S University of Florida.

Architecture and ATLAS Western Tier 2 Wei Yang ATLAS Western Tier 2 User Forum meeting SLAC April

Tools for collaboration How to share your duck tales…

June 24-25, 2008 Regional Grid Training, University of Belgrade, Serbia Introduction to gLite gLite Basic Services Antun Balaž SCL, Institute of Physics.

The Open Science Grid OSG Ruth Pordes Fermilab. 2 What is OSG? A Consortium of people working together to Interface Farms and Storage to a Grid and Researchers.

NW-GRID Campus Grids Workshop Liverpool31 Oct 2007 NW-GRID Campus Grids Workshop Liverpool31 Oct 2007 Moving Beyond Campus Grids Steven Young Oxford NGS.

MTA SZTAKI Hungarian Academy of Sciences Introduction to Grid portals Gergely Sipos

INFSO-RI Enabling Grids for E-sciencE OSG-LCG Interoperability Activity Author: Laurence Field (CERN)

Condor Project Computer Sciences Department University of Wisconsin-Madison Grids and Condor Barcelona,

US LHC OSG Technology Roadmap May 4-5th, 2005 Welcome. Thank you to Deirdre for the arrangements.

The DZero/PPDG Test Bed Test bed composition as of Feb 2002: 3 PC at Fermilab (sammy, samadams, sameggs) Contact: Gabriele Garzoglio 1 PC at Imperial College.

6/23/2005 R. GARDNER OSG Baseline Services 1 OSG Baseline Services In my talk I’d like to discuss two questions:  What capabilities are we aiming for.

VO Privilege Activity. The VO Privilege Project develops and implements fine-grained authorization to grid- enabled resources and services Started Spring.

Glite. Architecture Applications have access both to Higher-level Grid Services and to Foundation Grid Middleware Higher-Level Grid Services are supposed.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Site Architecture Resource Center Deployment Considerations MIMOS EGEE Tutorial.

December 26, 2015 RHIC/USATLAS Grid Computing Facility Overview Dantong Yu Brookhaven National Lab.

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America Alexandre Duarte CERN IT-GD-OPS UFCG LSD 1st EELA Grid School.

Padova, 5 October StoRM Service view Riccardo Zappi INFN-CNAF Bologna.

The CMS Top 5 Issues/Concerns wrt. WLCG services WLCG-MB April 3, 2007 Matthias Kasemann CERN/DESY.

Open Science Grid Build a Grid Session Siddhartha E.S University of Florida.

Tier 3 Support and the OSG US ATLAS Tier2/Tier3 Workshop at UChicago August 20, 2009 Marco Mambelli –

Western Tier 2 Site at SLAC Wei Yang US ATLAS Tier 2 Workshop Harvard University August 17-18, 2006.

Discussion on data transfer options for Tier 3 Tier 3(g,w) meeting at ANL ASC – May 19, 2009 Marco Mambelli – University of Chicago

EGEE-II INFSO-RI Enabling Grids for E-sciencE Overview of gLite, the EGEE middleware Mike Mineter Training Outreach Education National.

OSG Status and Rob Gardner University of Chicago US ATLAS Tier2 Meeting Harvard University, August 17-18, 2006.

SAM architecture EGEE 07 Service Availability Monitor for the LHC experiments Simone Campana, Alessandro Di Girolamo, Nicolò Magini, Patricia Mendez Lorenzo,

OSG STORAGE OVERVIEW Tanya Levshina. Talk Outline  OSG Storage architecture  OSG Storage software  VDT cache  BeStMan  dCache  DFS:  SRM Clients.

Open Science Grid Consortium Storage on Open Science Grid Placing, Using and Retrieving Data on OSG Resources Abhishek Singh Rana OSG Users Meeting July.

Enabling Grids for E-sciencE INFN Workshop – May 7-11 Rimini 1 Grid Accounting Status at INFN Riccardo Brunetti INFN-TORINO.

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Services for Distributed e-Infrastructure Access Tiziana Ferrari on behalf.

Job submission overview Marco Mambelli – August OSG Summer Workshop TTU - Lubbock, TX THE UNIVERSITY OF CHICAGO.

Defining the Technical Roadmap for the NWICG – OSG Ruth Pordes Fermilab.

Grid Colombia Workshop with OSG Week 2 Startup Rob Gardner University of Chicago October 26, 2009.

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI solution for high throughput data analysis Peter Solagna EGI.eu Operations.

Jean-Philippe Baud, IT-GD, CERN November 2007

Grid Computing: Running your Jobs around the World

Extended OSG client for WLCG

Short update on the latest gLite status

gLite The EGEE Middleware Distribution

Installation/Configuration

Presentation transcript:

OSG End User Tools Overview OSG Grid school – March 19, 2009 Marco Mambelli - University of Chicago A brief summary about the system users will interact with when operating on OSG

Grid (CE, SE, …)  Set of services or resources containing data and providing processing power  Computing elements  Storage elements  Information systems  Clients  How to interact?  validation and testing  active use WN Disk CE Door Gatekeeper SE LRM 3/2/09OSG End User Tools - M.Mambelli2

Grid software stack Grid Security Infrastructure Job Management Data Management Information (config & status) Services Core Globus Services Standard Network Protocols Condor Grid Client Workflow system (explicit or ad-hoc) Grid Application (often includes a Portal) 3/2/09OSG End User Tools - M.Mambelli3

Security: Grid Security Infrastructure  Grid certificate: your identity on the Grid  x509 based authentication (DN, CA)  Secure protocols: GRAM, WS-GRAM, GridFTP, GSI-http  Grid proxy: protects your certificate  Virtual Organization Management System (VOMS)  extended (grid) proxy  VO membership from VO servers (scalable)  more granularity: groups and roles  possibility for dynamic control  voms-proxy-init –voms atlas:/atlas/usatlas/Role=software 3/2/09OSG End User Tools - M.Mambelli4

Computing Element  Worker node  real or virtual host  Local Resource Manager (LRM)  PBS, Condor, LSF …  Jobmanager abstraction  GRAM (Globus GT2)  WS-GRAM (Globus GT4)  Other abstractions  Condor-G  Workflow managers (SWIFT, Panda, …) 3/2/09OSG End User Tools - M.Mambelli5

Storage Element  Disk pools  Storage system  DRM, dCache, GPFS, Xrootd  uniform space (file retrieval and naming)  file system abstraction  External access and file transfer  GridFTP (GSI, multiprotocol)  RFT, FTS (reliable)  SRM Storage Resource Management  space management  transfer negotiation and queuing  pinning  File catalog (RLS, LFC, RLI, LFC) 3/2/09OSG End User Tools - M.Mambelli6

Open Science Grid for a user  Set of standard packages (VDT)  Pacman (self contained environment, setup.sh)  Server packages  OSG-Client : for all interactive users  WN (client) : available to all job running on OSG  Standard environment  in the information system and in the job environment  OSG_APP (to install applications), OSG_DATA, OSG_GIRD (WN client)  Information system and monitoring  OSG Information Management (OIM)  VORS 3/2/09OSG End User Tools - M.Mambelli7

Next steps  Hands on session  Check the user tutorial:   References  rNodeClient rNodeClient  nstallationGuide nstallationGuide   Other tutorials:   3/2/09OSG End User Tools - M.Mambelli8