OSG Services at Tier2 Centers Rob Gardner University of Chicago WLCG Tier2 Workshop CERN June 12-14, 2006.

Slides:



Advertisements
Similar presentations
Dec 14, 20061/10 VO Services Project – Status Report Gabriele Garzoglio VO Services Project WBS Dec 14, 2006 OSG Executive Board Meeting Gabriele Garzoglio.
Advertisements

GUMS status Gabriele Carcassi PPDG Common Project 12/9/2004.
 Contributing >30% of throughput to ATLAS and CMS in Worldwide LHC Computing Grid  Reliant on production and advanced networking from ESNET, LHCNET and.
Plateforme de Calcul pour les Sciences du Vivant SRB & gLite V. Breton.
CMS Applications Towards Requirements for Data Processing and Analysis on the Open Science Grid Greg Graham FNAL CD/CMS for OSG Deployment 16-Dec-2004.
Implementing Finer Grained Authorization in the Open Science Grid Gabriele Carcassi, Ian Fisk, Gabriele, Garzoglio, Markus Lorch, Timur Perelmutov, Abhishek.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
Open Science Grid Use of PKI: Wishing it was easy A brief and incomplete introduction. Doug Olson, LBNL PKI Workshop, NIST 5 April 2006.
1 Open Science Grid Middleware at the WLCG LHCC review Ruth Pordes, Fermilab.
Open Science Ruth Pordes Fermilab, July 17th 2006 What is OSG Where Networking fits Middleware Security Networking & OSG Outline.
OSG End User Tools Overview OSG Grid school – March 19, 2009 Marco Mambelli - University of Chicago A brief summary about the system.
Open Science Grid Software Stack, Virtual Data Toolkit and Interoperability Activities D. Olson, LBNL for the OSG International.
OSG Operations and Interoperations Rob Quick Open Science Grid Operations Center - Indiana University EGEE Operations Meeting Stockholm, Sweden - 14 June.
Integration and Sites Rob Gardner Area Coordinators Meeting 12/4/08.
OSG Middleware Roadmap Rob Gardner University of Chicago OSG / EGEE Operations Workshop CERN June 19-20, 2006.
INFSO-RI Enabling Grids for E-sciencE The US Federation Miron Livny Computer Sciences Department University of Wisconsin – Madison.
Mark L. Green, Ph.D. Grid Computational Scientist Monitoring and Information Services Architecture and Deployment University at Buffalo The State University.
1 Open Science Grid.. An introduction Ruth Pordes Fermilab.
VOX Project Status T. Levshina. Talk Overview VOX Status –Registration –Globus callouts/Plug-ins –LRAS –SAZ Collaboration with VOMS EDG team Preparation.
May 8, 20071/15 VO Services Project – Status Report Gabriele Garzoglio VO Services Project – Status Report Overview and Plans May 8, 2007 Computing Division,
G RID M IDDLEWARE AND S ECURITY Suchandra Thapa Computation Institute University of Chicago.
Apr 30, 20081/11 VO Services Project – Stakeholders’ Meeting Gabriele Garzoglio VO Services Project Stakeholders’ Meeting Apr 30, 2008 Gabriele Garzoglio.
Mine Altunay OSG Security Officer Open Science Grid: Security Gateway Security Summit January 28-30, 2008 San Diego Supercomputer Center.
Overview of Monitoring and Information Systems in OSG MWGS08 - September 18, Chicago Marco Mambelli - University of Chicago
DOSAR Workshop, Sao Paulo, Brazil, September 16-17, 2005 LCG Tier 2 and DOSAR Pat Skubic OU.
Mar 28, 20071/9 VO Services Project Gabriele Garzoglio The VO Services Project Don Petravick for Gabriele Garzoglio Computing Division, Fermilab ISGC 2007.
São Paulo Regional Analysis Center SPRACE Status Report 22/Aug/2006 SPRACE Status Report 22/Aug/2006.
Global Grid Forum GridWorld GGF15 Boston USA October Abhishek Singh Rana and Frank Wuerthwein UC San Diegowww.opensciencegrid.org The Open Science.
The Open Science Grid OSG Ruth Pordes Fermilab. 2 What is OSG? A Consortium of people working together to Interface Farms and Storage to a Grid and Researchers.
CERN Using the SAM framework for the CMS specific tests Andrea Sciabà System Analysis WG Meeting 15 November, 2007.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks State of Interoperability Laurence Field.
Mine Altunay July 30, 2007 Security and Privacy in OSG.
INFSO-RI Enabling Grids for E-sciencE OSG-LCG Interoperability Activity Author: Laurence Field (CERN)
Ruth Pordes November 2004TeraGrid GIG Site Review1 TeraGrid and Open Science Grid Ruth Pordes, Fermilab representing the Open Science.
Overview of Privilege Project at Fermilab (compilation of multiple talks and documents written by various authors) Tanya Levshina.
Role Based VO Authorization Services Ian Fisk Gabriele Carcassi July 20, 2005.
US LHC OSG Technology Roadmap May 4-5th, 2005 Welcome. Thank you to Deirdre for the arrangements.
Open Science Grid (OSG) Introduction for the Ohio Supercomputer Center Open Science Grid (OSG) Introduction for the Ohio Supercomputer Center February.
Yet Another Grid Project: The Open Science Grid at SLAC Matteo Melani, Booker Bense and Wei Yang SLAC Hepix Conference 10/13/05, SLAC, Menlo Park, CA,
OSG Integration Activity Report Rob Gardner Leigh Grundhoefer OSG Technical Meeting UCSD Dec 16, 2004.
6/23/2005 R. GARDNER OSG Baseline Services 1 OSG Baseline Services In my talk I’d like to discuss two questions:  What capabilities are we aiming for.
VO Privilege Activity. The VO Privilege Project develops and implements fine-grained authorization to grid- enabled resources and services Started Spring.
Site Validation Session Report Co-Chairs: Piotr Nyczyk, CERN IT/GD Leigh Grundhoefer, IU / OSG Notes from Judy Novak WLCG-OSG-EGEE Workshop CERN, June.
Status Organization Overview of Program of Work Education, Training It’s the People who make it happen & make it Work.
The OSG and Grid Operations Center Rob Quick Open Science Grid Operations Center - Indiana University ATLAS Tier 2-Tier 3 Meeting Bloomington, Indiana.
Testing and integrating the WLCG/EGEE middleware in the LHC computing Simone Campana, Alessandro Di Girolamo, Elisa Lanciotti, Nicolò Magini, Patricia.
Sep 25, 20071/5 Grid Services Activities on Security Gabriele Garzoglio Grid Services Activities on Security Gabriele Garzoglio Computing Division, Fermilab.
WLCG Authentication & Authorisation LHCOPN/LHCONE Rome, 29 April 2014 David Kelsey STFC/RAL.
OSG Deployment Preparations Status Dane Skow OSG Council Meeting May 3, 2005 Madison, WI.
Open Science Grid Build a Grid Session Siddhartha E.S University of Florida.
VOX Project Tanya Levshina. 05/17/2004 VOX Project2 Presentation overview Introduction VOX Project VOMRS Concepts Roles Registration flow EDG VOMS Open.
Открытая решетка науки строя открытое Cyber- инфраструктура для науки GRID’2006 Dubna, Россия Июнь 26, 2006 Robertovich Gardner Университет Chicago.
Sep 17, 20081/16 VO Services Project – Stakeholders’ Meeting Gabriele Garzoglio VO Services Project Stakeholders’ Meeting Sep 17, 2008 Gabriele Garzoglio.
VOX Project Status T. Levshina. 5/7/2003LCG SEC meetings2 Goals, team and collaborators Purpose: To facilitate the remote participation of US based physicists.
1 An update on the Open Science Grid for IHEPCCC Ruth Pordes, Fermilab.
Site Authorization Service Local Resource Authorization Service (VOX Project) Vijay Sekhri Tanya Levshina Fermilab.
Tier 3 Support and the OSG US ATLAS Tier2/Tier3 Workshop at UChicago August 20, 2009 Marco Mambelli –
Placeholder ES 1 CERN IT EGI Technical Forum, Experiment Support group AAI usage, issues and wishes for WLCG Maarten Litmaath CERN.
Integration TestBed (iTB) and Operations Provisioning Leigh Grundhoefer.
SRM-DRM ITB Service Integration Plan Jorge Rodriguez OSG Integration Workshop at UC Feb 15-18, 2005.
OSG Status and Rob Gardner University of Chicago US ATLAS Tier2 Meeting Harvard University, August 17-18, 2006.
SAM architecture EGEE 07 Service Availability Monitor for the LHC experiments Simone Campana, Alessandro Di Girolamo, Nicolò Magini, Patricia Mendez Lorenzo,
Monitoring and Information Services Core Infrastructure (MIS-CI) Service Description Mark L. Green OSG Integration Workshop at UC Feb 15-17, 2005.
OSG Facility Miron Livny OSG Facility Coordinator and PI University of Wisconsin-Madison Open Science Grid Scientific Advisory Group Meeting June 12th.
Defining the Technical Roadmap for the NWICG – OSG Ruth Pordes Fermilab.
Grid Colombia Workshop with OSG Week 2 Startup Rob Gardner University of Chicago October 26, 2009.
1 Open Science Grid Middleware at the WLCG LHCC review Ruth Pordes, Fermilab.
Open Science Grid Progress and Status
Monitoring and Information Services Technical Group Report
Leigh Grundhoefer Indiana University
Presentation transcript:

OSG Services at Tier2 Centers Rob Gardner University of Chicago WLCG Tier2 Workshop CERN June 12-14, 2006

6/14/2006 OSG Services at Tier2 Centers R. Gardner 2 Introduction Tier2 centers in the US are vital components of the OSG and WLCG computing facility  They provide Tier2 resources to US ATLAS and US CMS according to their respective computing models  They additionally provide resources to VO’s outside the LHC community and may “federate” with other infrastructures  They participate in the OSG Integration Testbed (ITB) leading the new releases of the OSG infrastructure  Manpower at the Tier2 centers actively provide feedback and effort in many OSG activities (deployment, documentation, monitoring, information services, interoperability, etc.)  Additionally manpower is used to support data and job management services for ATLAS and CMS  Finally, additional leveraged resources are obtained from non- LHC program funds in many cases (University, other programs)

6/14/2006 OSG Services at Tier2 Centers R. Gardner 3 OSG Service Stack Virtual Data Toolkit (VDT) Common Services NMI + VOMS, CEMon (common EGEE components), MonaLisa, Clarens, AuthZ OSG Release Cache: VDT + Configuration, Validation, VO management ATLAS Services ATLAS Services Infrastructure Applications CMS Services CMS Services Other VO Services Other VO Services NMI releases (Globus + Condor) Fig. from R. Pordes

6/14/2006 OSG Services at Tier2 Centers R. Gardner 4 OSG Service Overview Compute elements  GRAM, GridFTP, information services (GIP), monitoring, worker node client tools (eg. srmcp) Storage elements  SRM-drm, SRM-dCache (provided by VOs), v1.1 Site level services  GUMS - for privilege (authorization) mappings VO level services  VOMS and user role assignments VO edge services  Semi-persistent services & agents as needed by applications Multi-VO, common services  Monitoring repositories, Catalogs, BDII index services, etc

6/14/2006 OSG Services at Tier2 Centers R. Gardner 5 OSG Process Applications  Integration  Provision  Deploy Integration Testbed (15-20) Production (50+) sites Sao Paolo Taiwan, S.Korea ITB OSG

6/14/2006 OSG Services at Tier2 Centers R. Gardner 6 Tier2 Centers in the ITB As reported in GridCat status catalog site ITB releaseservicefacility status Ops map Tier 2 sites

6/14/2006 OSG Services at Tier2 Centers R. Gardner 7 OSG Release Timeline 11/03 2/05 4/05 12/05 9/05 2/06 4/06 7/06 ITB ITB ITB ITB ITB OSG OSG OSG OSG ITB Integration Production

6/14/2006 OSG Services at Tier2 Centers R. Gardner 8 Production Use Cases  Tier2 ATLAS  Panda pilot scheduling system accesses local batch queue via Condor-G and GRAM  OSG priviledge infrastructure for role-based authorization  Panda system requires local DQ2 site level services  ATLAS releases (installed in common application area)  OSG monitors report into Panda monitoring framework CMS  Condor-G interface via GRAM  CMS applications and LCG client tools in common area  GIP+BDII for interoperability with LCG

6/14/2006 OSG Services at Tier2 Centers R. Gardner 9 Analysis Use Cases  Tier2 ATLAS  Panda, DQ2, and OSG infrastructure starting to be used to handle user analysis jobs via ‘analysis pilots’  Development work underway to support a multi-tasking pilot  Priorities can be set within Panda task queue, requiring no changes to the existing site-level authorization CMS  CRAB system requires local PhEDEx service and persistent agents for data management  Existing OSG privilege infrastructure used for authorization  Submission via LCG RB, requires CE information providers Storage  In both cases, site-level storage services are provided by the VO (SRM dCache)

6/14/2006 OSG Services at Tier2 Centers R. Gardner 10 Calibration Use Cases  Tier2 ATLAS  No direct experience yet at Tier2 centers, expect to learn from upcoming calibration service challenges  Calibration datasets will require standard DDM infrastructure  Local access to MySQL databases; Frontier (Squid cache) services may be needed CMS  Also will be utilizing Frontier/Squid caches for calibration and alignment data Future releases of OSG  Will provide Squid by default, available for use for calibration databases as well as experiment software releases

6/14/2006 OSG Services at Tier2 Centers R. Gardner 11 Current OSG Release Description VDT based core infrastructure Privilege infrastructure  VOMS service  PRIMA gatekeeper callout for extended role-based proxy  GUMS site account/DN management GT4 GridFTP GT4 Pre-Web Services and Web Services GRAM Information services: GridCat Catalog, MDS + Generic Information Providers (LCG) MonALISA, Core-MIS and ACDC monitoring tools

6/14/2006 OSG Services at Tier2 Centers R. Gardner 12 VDT Server Content CA Certificates v13 (includes IGTF 1.1 CAs) EDG CRL Update EDG Make Gridmap Fault Tolerant Shell (ftsh) Generic Information Provider (Iowa 15-Feb-2006) Globus Toolkit, pre web-services, client Globus Toolkit, pre web-services, server Globus Toolkit, web-services, client Globus Toolkit, web-services, server GLUE Schema 1.2 draft 7 GPT 3.2 Java SDK 1.4.2_10 KX Logrotate 3.7 MonALISA MyProxy 3.4 MySQL PPDG Cert Scripts 1.7 PRIMA Authorization Module 0.3 RLS, client UberFTP 1.18 Virtual Data System Several in common with WLCG/EGEE

6/14/2006 OSG Services at Tier2 Centers R. Gardner 13 Service Deployment process VDT distributed via Pacman cache as CE server, client, VO management software packages OSG configuration and deployment cache used by Tier2 sites:  OSG CE-Server  OSG WorkerNode-Client GUMS server - and VO accounts, authorizations (file permissions, batch priorities, etc) SRM-dCache deploymed separately -- site, system specific “CE Storage” (transient job directories) and application install areas configured “VO boxes” and services deployed independently

6/14/2006 OSG Services at Tier2 Centers R. Gardner 14 Privilege Authorization Services Site level services to support fine-grained, role-based access to Tier2 resources:  GUMS - Grid User Management System - maps user proxy to local accounts based on role and group  Site admins grant access rights and privileges based on accounts  PRIMA callout from GRAM gatekeeper - assigns account based on GUMS mapping and submits to local scheduler  Roles at Tier2s (eg: usatlas1=production; usatlas2=software; usatlas3=users) Receives updates on mappings from VOMS Reverse map created periodically for accounting purposes (Monalisa presently) More work needed to integrate app framework priorities with site-level infrastructure and accounting services (DN, Group -based fair share mechanisms)

6/14/2006 OSG Services at Tier2 Centers R. Gardner 15 Information Services GIP (Generic Information Provider)  An information service that aggregates static and dynamic resource information  Produces information for use with LDAP-based Grid information systems  Glue 1.2 schema GIP use cases  LCG-OSG interoperability  GridCat cross checks Site level BDII service  Scalability  Query by LCG RB

6/14/2006 OSG Services at Tier2 Centers R. Gardner 16 Monitoring and Accounting Monalisa, site level accounting servcies (native tools), site verify checks & report, GridExerciser OSG 0.4.1

6/14/2006 OSG Services at Tier2 Centers R. Gardner 17 Conclusions Tier2 centers provide vital services and resources for the OSG (and WLCG) computing facilties Tier2 manpower used to support VO specific services -- eg. VO boxes for data management Tier2 direct participation in OSG release testing and validation of services OSG site-level services for security and authorization, site verfication and validation, monitoring & information used heavily by ATLAS and CMS Expect OSG services provided to evolve (next talk) so managing incremental upgrades will be key