Open Science Grid Build a Grid Session Siddhartha E.S University of Florida.

Slides:



Advertisements
Similar presentations
Open Science Grid Discovering and understanding the site environment Or, yet another site test kit.
Advertisements

Chapter 20 Oracle Secure Backup.
Grid Resource Allocation Management (GRAM) GRAM provides the user to access the grid in order to run, terminate and monitor jobs remotely. The job request.
Role Based VO Authorization Services Ian Fisk Gabriele Carcassi July 20, 2005.
GUMS status Gabriele Carcassi PPDG Common Project 12/9/2004.
CERN LCG Overview & Scaling challenges David Smith For LCG Deployment Group CERN HEPiX 2003, Vancouver.
Security Q&A OSG Site Administrators workshop Indianapolis August Doug Olson LBNL.
Academic Technology Services The UCLA Grid Portal - Campus Grids and the UC Grid Joan Slottow and Prakashan Korambath Research Computing Technologies UCLA.
A Computation Management Agent for Multi-Institutional Grids
Security Mechanisms The European DataGrid Project Team
K.Harrison CERN, 23rd October 2002 HOW TO COMMISSION A NEW CENTRE FOR LHCb PRODUCTION - Overview of LHCb distributed production system - Configuration.
Installing and running COMSOL on a Windows HPCS2008(R2) cluster
11 MAINTAINING THE OPERATING SYSTEM Chapter 5. Chapter 5: MAINTAINING THE OPERATING SYSTEM2 CHAPTER OVERVIEW Understand the difference between service.
11 MAINTAINING THE OPERATING SYSTEM Chapter 5. Chapter 5: MAINTAINING THE OPERATING SYSTEM2 CHAPTER OVERVIEW  Understand the difference between service.
OSG End User Tools Overview OSG Grid school – March 19, 2009 Marco Mambelli - University of Chicago A brief summary about the system.
LANDesk Management Gateway
Rsv-control Marco Mambelli – Site Coordination meeting October 1, 2009.
Understanding the CORBA Model. What is CORBA?  The Common Object Request Broker Architecture (CORBA) allows distributed applications to interoperate.
The Glidein Service Gideon Juve What are glideins? A technique for creating temporary, user- controlled Condor pools using resources from.
December 8 & 9, 2005, Austin, TX SURA Cyberinfrastructure Workshop Series: Grid Technology: The Rough Guide Grid Technology: The Rough Guide Grid Building.
OSG Services at Tier2 Centers Rob Gardner University of Chicago WLCG Tier2 Workshop CERN June 12-14, 2006.
OSG Site Provide one or more of the following capabilities: – access to local computational resources using a batch queue – interactive access to local.
03/27/2003CHEP20031 Remote Operation of a Monte Carlo Production Farm Using Globus Dirk Hufnagel, Teela Pulliam, Thomas Allmendinger, Klaus Honscheid (Ohio.
1 BIG FARMS AND THE GRID Job Submission and Monitoring issues ATF Meeting, 20/06/03 Sergio Andreozzi.
1 Overview of the Application Hosting Environment Stefan Zasada University College London.
A. Sim, CRD, L B N L 1 OSG Applications Workshop 6/1/2005 OSG SRM/DRM Readiness and Plan Alex Sim / Jorge Rodriguez Scientific Data Management Group Computational.
G RID M IDDLEWARE AND S ECURITY Suchandra Thapa Computation Institute University of Chicago.
Mine Altunay OSG Security Officer Open Science Grid: Security Gateway Security Summit January 28-30, 2008 San Diego Supercomputer Center.
Module 7: Managing the User Environment by Using Group Policy.
National Computational Science National Center for Supercomputing Applications National Computational Science NCSA-IPG Collaboration Projects Overview.
Evolution of the Open Science Grid Authentication Model Kevin Hill Fermilab OSG Security Team.
Eric Shook, Anand Padmanabhan Grid Research & educatiOn IoWa (GROW) ITS Academic Technologies – Research Services The University of Iowa Iowa City,
CHEP03 Mar 25Mary Thompson Fine-grained Authorization for Job and Resource Management using Akenti and Globus Mary Thompson LBL,Kate Keahey ANL, Sam Lang.
Open Science Grid OSG CE Quick Install Guide Siddhartha E.S University of Florida.
9th EELA TUTORIAL - USERS AND SYSTEM ADMINISTRATORS E-infrastructure shared between Europe and Latin America CE + WN installation and configuration.
Introduction to OSG Security Suchandra Thapa Computation Institute University of Chicago March 19, 20091GSAW 2009 Clemson.
July 11-15, 2005Lecture3: Grid Job Management1 Grid Compute Resources and Job Management.
4th EELA TUTORIAL - USERS AND SYSTEM ADMINISTRATORS E-infrastructure shared between Europe and Latin America CE + WN installation and configuration.
APST Internals Sathish Vadhiyar. apstd daemon should be started on the local resource Opens a port to listen for apst client requests Runs on the host.
Overview of Privilege Project at Fermilab (compilation of multiple talks and documents written by various authors) Tanya Levshina.
Role Based VO Authorization Services Ian Fisk Gabriele Carcassi July 20, 2005.
US LHC OSG Technology Roadmap May 4-5th, 2005 Welcome. Thank you to Deirdre for the arrangements.
Creating and running an application.
Pilot Factory using Schedd Glidein Barnett Chiu BNL
Configuring and Troubleshooting Identity and Access Solutions with Windows Server® 2008 Active Directory®
December 26, 2015 RHIC/USATLAS Grid Computing Facility Overview Dantong Yu Brookhaven National Lab.
VO Box Issues Summary of concerns expressed following publication of Jeff’s slides Ian Bird GDB, Bologna, 12 Oct 2005 (not necessarily the opinion of)
Linux Operations and Administration
Auditing Project Architecture VERY HIGH LEVEL Tanya Levshina.
ALCF Argonne Leadership Computing Facility GridFTP Roadmap Bill Allcock (on behalf of the GridFTP team) Argonne National Laboratory.
OSG Site Admin Workshop - Mar 2008Using gLExec to improve security1 OSG Site Administrators Workshop Using gLExec to improve security of Grid jobs by Alain.
“The Grid” Gatekeeper / Batch Scheduler Workers Worker Node Workers Worker Node Submitter Site Gatekeeper / Batch Scheduler.
Grid Compute Resources and Job Management. 2 Grid middleware - “glues” all pieces together Offers services that couple users with remote resources through.
VOX Project Tanya Levshina. 05/17/2004 VOX Project2 Presentation overview Introduction VOX Project VOMRS Concepts Roles Registration flow EDG VOMS Open.
Site Authorization Service Local Resource Authorization Service (VOX Project) Vijay Sekhri Tanya Levshina Fermilab.
A closer look at the VDT RPMs Alain Roy OSG Software Coordinator.
OSG PKI Transition Impact on CMS. Impact on End User After March , DOEGrids CA will stop issuing or renewing certificates. If a user is entitled.
RI EGI-TF 2010, Tutorial Managing an EGEE/EGI Virtual Organisation (VO) with EDGES bridged Desktop Resources Tutorial Robert Lovas, MTA SZTAKI.
Incident Response Forensics and Review OSG Security Drill OSG Site Administrators workshop Indianapolis August Anand Padmanabhan UIUC.
Open Science Grid Configuring RSV OSG Resource & Service Validation Thomas Wang Grid Operations Center (OSG-GOC) Indiana University.
Why you should care about glexec OSG Site Administrator’s Meeting Written by Igor Sfiligoi Presented by Alain Roy Hint: It’s about security.
3/5/2007Chris Green, FNAL / OSG VO-Level Site Validation.
Enabling Grids for E-sciencE Claudio Cherubino INFN DGAS (Distributed Grid Accounting System)
Training Objectives About D2F Download Installation Configuration
Operating a glideinWMS frontend by Igor Sfiligoi (UCSD)
CRC exercises Not happy with the way the document for testbed architecture is progressing More a collection of contributions from the mware groups rather.
CMS report from FNAL demo week Marco Verlato (INFN-Padova)
Update on EDG Security (VOMS)
Intuit has launched QuickBooks File Doctor tool (QBFD) in QuickBooks File Doctor is a tool that has been designed to recover the damaged company.
From Prototype to Production Grid
Presentation transcript:

Open Science Grid Build a Grid Session Siddhartha E.S University of Florida

Open Science Grid OVERVIEW As OSG-Site Admins we are responsible for providing some services such as –gatekeeper –gridftp –Monitoring and accounting There are some tools/packages which help us provide these services –Pacman –Osg-CE bundle of packages –VDT/Globus –Condor –Gratia(monitoring)

Open Science Grid Diagrammatic representation of an OSG-site, VO and Centralised Services

Open Science Grid PACMAN Pacman is a package management tool that allows you to define, install, and configure software. Different versions of OSG require different versions of Pacman: the newer the OSG software version, the newer the release of Pacman required. For OSG we use Pacman

Open Science Grid SOME NOTES ON PACMAN INSTALLATION A lways remember that we need to have our software installed in a place where we can do something in the network. “usr/local” or “/nfs/xxx” are good candidates. It is also advisable to install Pacman in a directory of its own. Follow the commands in section-1 of the handout to complete installation

Open Science Grid THE CONDOR BATCH SYSTEM Advisable to be created in a separate directory- will allow asynchronous upgrades without loss. Follow the command sequences in section-2 to install CONDOR using Pacman If your operating system is not supported by Pacman we need to use an option called - pretend-platform along with Pacman Give complete paths(No Dots)

Open Science Grid Installing OSG CE Services Services will not necessarily run as root (e.g., by default Globus-ws will run under globus if it exists, or daemon if it doesn't; MySql a will run under daemon) So one needs to make sure Mysql user and group exists by id ' ing it. If not /etc/group or /etc/passwd or /etc/shadow may require a change. Choose an installation directory. This need not be shared by the worker nodes. Any software which needs to be seen from the worker nodes may be provided by the worker node client package. Follow the commands in the handout vdt-install.log--> can be monitored during installation ( good place to look in for errors)

Open Science Grid Obtaining and Configuring PKI certificates Refer to the handout for the certificate request command(Section-4) Fill in the requested information in a proper manner Your host cert request has to be approved and you will receive an notification when you can download the host cert configure the OSG CE software

Open Science Grid Next 3-steps Start Services Start Condor Set up Managed Fork –Follow the commands provided in the handout to complete these steps

Open Science Grid Testing CE node (Grid maps) Grid mapfile is a type of identity mapping service This test is to be done as a normal user and not as root One has to create a proxy for himself Then type grid-proxy-info -identity The as root put an entry in the /etc/grid-security/grid- mapfile The entry would be “Certificate” Save the changes Continue the commands for testing Fork-Queue

Open Science Grid TESTS CONTINUED Test the job manager fork Test the job manager queue Test the GSIFTP services –Commands for these tests are explained in sections 7.2,7.3 and 7.4 of the commands handout.

Open Science Grid Possible Error If you encounter “GRAM Job Submission failed because connection to the server failed”, it could possibly due to mismatch in the hostnames in the certificate and what you are generating in the command. When we are facing this error we need to export GLOBUS_HOSTNAME as the one in the cert. To do this we could go to vdt-local-setup.sh and vdt-local- setup.csh files in nfs/osg/vdt/etc, where nfs/osg is the place where my vdt and osg ce package is installed. restart the gatekeeper using the command /etc/init.d/xinetd restart Use $GLOBUS_HOSTNAME instead of $HOSTNAME

Open Science Grid Site-Verification At this stage it is assumed that you have tested the job-manager queue, gsiftp and are ready to do the site-verify./site_verify.pl –host = osg.hpc.ufl.edu is the command to be executed in the VDT_LOCATION/verify Check whether this command returns a bunch of results as untested. Look up to ensure essential results have not failed You can now give the gatekeeper jobs.

Open Science Grid Diagrammatic Representation of GUMS

Open Science Grid GUMS Grid User Management System (GUMS) The grid credential for each incoming job is associated with an appropriate site credential by GUMS. suited for environment with multiple gatekeepers Individual resource administrators are able to assign different mapping policies to different groups of users and define groups of hosts on which the mappings will be used For installation commands refer to section-8 of the command handout

Open Science Grid Further Info –