Grid NERSC demo Shreyas Cholia Open Software and Programming NERSC User Group Meeting September 19, 2007.

Slides:



Advertisements
Similar presentations
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks MyProxy and EGEE Ludek Matyska and Daniel.
Advertisements

Globus Workshop at CoreGrid Summer School 2006 Dipl.-Inf. Hamza Mehammed Leibniz Computing Centre.
MyProxy Guy Warner NeSC Training.
Cross-site data transfer on TeraGrid using GridFTP TeraGrid06 Institute User Introduction to TeraGrid June 12 th by Krishna Muriki
John Kewley CCLRC Daresbury Laboratory NW-GRID Training Event 26 th January 2007 GROWL Scripts and Web Services John Kewley Grid Technology Group E-Science.
CENG 546 Dr. Esma Yıldırım.  A fundamental enabling technology for the "Grid," letting people share computing power, databases, and other tools securely.
Security Q&A OSG Site Administrators workshop Indianapolis August Doug Olson LBNL.
1 Workshop 20: Teaching a Hands-on Undergraduate Grid Computing Course SIGCSE The 41st ACM Technical Symposium on Computer Science Education Friday.
Job Submission Using PBSPro and Globus Job Commands.
GlueX Computing GlueX Collaboration Meeting – Glasgow Edward Brash – University of Regina August 4 th, 2003.
Slides for Grid Computing: Techniques and Applications by Barry Wilkinson, Chapman & Hall/CRC press, © Chapter 1, pp For educational use only.
GLite authentication and authorization Discipline: Grid Computing, 07/08-2 Practical classes Inês Dutra, DCC/FCUP.
GRID Workload Management System Massimo Sgaravatto INFN Padova.
Grid Services at NERSC Shreyas Cholia Open Software and Programming Group, NERSC NERSC User Group Meeting September 17, 2007.
1 Short Course on Grid Computing Jornadas Chilenas de Computación 2010 INFONOR-CHILE 2010 November 15th - 19th, 2010 Antofagasta, Chile Dr. Barry Wilkinson.
1 Workshop 20: Teaching a Hands-on Undergraduate Grid Computing Course SIGCSE The 41st ACM Technical Symposium on Computer Science Education Friday.
1c.1 Assignment 2 Preliminaries Review (Full details in assignment write-up.)‏ © 2011 B. Wilkinson/Clayton Ferner. Fall 2011 Grid computing course. Modification.
1 Short Course on Grid Computing Jornadas Chilenas de Computación 2010 INFONOR-CHILE 2010 November 15th - 19th, 2010 Antofagasta, Chile Dr. Barry Wilkinson.
ORNL is managed by UT-Battelle for the US Department of Energy Globus: Proxy Lifetime Endpoint Lifetime Oak Ridge Leadership Computing Facility.
Overview of TeraGrid Resources and Usage Selim Kalayci Florida International University 07/14/2009 Note: Slides are compiled from various TeraGrid Documentations.
BaBar WEB job submission with Globus authentication and AFS access T. Adye, R. Barlow, A. Forti, A. McNab, S. Salih, D. H. Smith on behalf of the BaBar.
OSG PKI Grid Admin (GA) Training Mine Altunay, Jim Basney OSG PKI Team October 8, 2012.
High Performance Louisiana State University - LONI HPC Enablement Workshop – LaTech University,
December 8 & 9, 2005, Austin, TX SURA Cyberinfrastructure Workshop Series: Grid Technology: The Rough Guide Grid Technology: The Rough Guide Grid Building.
TRASC Globus Application Launcher VPAC Development Team Sudarshan Ramachandran.
Activating Clarity  Activating Clarity  Activation  Online Activation  Fax Activation  Review and Verify Activation and License Terms  Updating.
Nadia LAJILI User Interface User Interface 4 Février 2002.
LHCb and DataGRID - the workplan for 2001 Eric van Herwijnen Wednesday, 28 march 2001.
LCG Middleware Testing in 2005 and Future Plans E.Slabospitskaya, IHEP, Russia CERN-Russia Joint Working Group on LHC Computing March, 6, 2006.
1 Globus Grid Middleware: Basics, Components, and Services Source: The Globus Project Argonne National Laboratory & University of Southern California
Group 1 : Grid Computing Laboratory of Information Technology Supervisors: Alexander Ujhinsky Nikolay Kutovskiy.
August 13, 2003Eric Hjort Getting Started with Grid Computing in STAR Eric Hjort, LBNL STAR Collaboration Meeting August 13, 2003.
NUG 2004 Grid File Yanker Demo Shreyas Cholia Mass Storage Group, NERSC 06/24/2004.
Andrey Meeting 7 October 2003 General scheme: jobs are planned to go where data are and to less loaded clusters SUNY.
Open Science Grid OSG CE Quick Install Guide Siddhartha E.S University of Florida.
© 2007 UC Regents1 Track 1: Cluster and Grid Computing NBCR Summer Institute Session 1.1: Introduction to Cluster and Grid Computing July 31, 2007 Wilfred.
E-infrastructure shared between Europe and Latin America Security Hands-on Christian Grunfeld, UNLP 8th EELA Tutorial, La Plata, 11/12-12/12,2006.
Association with the Gilda Virtual Organization Certificate,VO membership, and MyProxy Server usage.
E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA Hands-on on security Pedro Rausch IF - UFRJ.
NGS Portal.
EGEE-III INFSO-RI Enabling Grids for E-sciencE Apr. 25, Grid Computing Hands On Training for Users Faculty of Sciences, University.
Condor-G A Quick Introduction Alan De Smet Condor Project University of Wisconsin - Madison.
1 Short Course on Grid Computing Jornadas Chilenas de Computación 2010 INFONOR-CHILE 2010 November 15th - 19th, 2010 Antofagasta, Chile Dr. Barry Wilkinson.
Part 9: MyProxy Pragmatics This presentation and lab ends the GRIDS Center agenda Q: When do we convene again tomorrow?
July 11-15, 2005Lecture3: Grid Job Management1 Grid Compute Resources and Job Management.
Part Five: Globus Job Management A: GRAM B: Globus Job Commands C: Laboratory: globusrun.
12th September 2007UK e-Science All Hands Meeting1 John Kewley Grid Technology Group e-Science Centre STFC Daresbury Laboratory GROWL.
Creating and running an application.
January 26, 2003Eric Hjort HRMs in STAR Eric Hjort, LBNL (STAR/PPDG Collaborations)
UK Grid Meeting Glenn Patrick1 LHCb Grid Activities in UK Grid Prototype and Globus Technical Meeting QMW, 22nd November 2000 Glenn Patrick (RAL)
Portal Update Plan Ashok Adiga (512)
Hands-on security Angelines Alberto Morillas Ciemat.
EGEE is a project funded by the European Union under contract IST Grid proxy and MyProxy Roberto Barbera Univ. of Catania and INFN SEE-GRID.
4th EELA TUTORIAL - USERS AND SYSTEM ADMINISTRATORS E-infrastructure shared between Europe and Latin America Security Hands-on Vanessa.
John Kewley e-Science Centre All Hands Meeting st September, Nottingham GROWL: A Lightweight Grid Services Toolkit and Applications John Kewley.
National Energy Research Scientific Computing Center (NERSC) Visportal : interface to grid enabled NERC resources Cristina Siegerist NERSC Center Division,
Open Science Grid Build a Grid Session Siddhartha E.S University of Florida.
Manchester Computing Supercomputing, Visualization & eScience Seamless Access to Multiple Datasets Mike AS Jones ● Demo Run-through.
LCG2 Tutorial Viet Tran Institute of Informatics Slovakia.
Hands-on security Carlos Fuentes RedIRIS Madrid,26 – 30 de Octubre de 2008.
Hands on Security, Authentication and Authorization Virginia Martín-Rubio Pascual RedIRIS/Red.es Curso Grid y e-Ciencia.
EGI-InSPIRE RI Grid Training for Power Users EGI-InSPIRE N G I A E G I S Grid Training for Power Users Institute of Physics Belgrade.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) 马兰馨 IHEP, CAS Hands on gLite Security.
Enabling Grids for E-sciencE gLite security pratical tutorial Dario Russo INFN Catania Catania,
The NGS Portal Guy Warner NeSC Training.
EGEE is a project funded by the European Union under contract IST Job Submission Giuseppe La Rocca EGEE NA4 Generic Applications INFN Catania.
Contoso Insurance Azure App Services Code Sample
Creating and running applications on the NGS
CRC exercises Not happy with the way the document for testbed architecture is progressing More a collection of contributions from the mware groups rather.
Grid Computing Software Interface
Presentation transcript:

Grid NERSC demo Shreyas Cholia Open Software and Programming NERSC User Group Meeting September 19, 2007

NERSC User Group Meeting, September 17, Overview Getting a certificate Initializing a certificate Registering certificate with NIM Submitting a fork job Submitting a serial job Submitting an MPI job Moving files with GridFTP Third party transfers with GridFTP Using myproxy to store your certificate

NERSC User Group Meeting, September 17, Getting a Certificate Instructions at: – Process: – –Click on Retrieval->New User. –Fill out form. –You will receive an once the certificate is ready. Click on the link. –Click on the import button in your browser. –Export/Backup certificate to local file –Convert file to usercert/key.pem using openssl Renew Annually - repeat above process

NERSC User Group Meeting, September 17, Initializing Your Cert Load the globus module. One of: module load globus module load osg. /usr/common/osg/0.6.0/setup.sh Your certificate is in: $HOME/.globus/usercert.pem $HOME/.globus/userkey.pem Initialize the cert: grid-proxy-init (Creates temporary “proxy” cert valid for 12 hours) View cert: grid-cert-info grid-proxy-info

NERSC User Group Meeting, September 17, Adding Cert Info to NIM Click on Grid certificates tab Click on Add Certificate Enter information from: grid-cert-info -subject grid-cert-info -issuer

NERSC User Group Meeting, September 17, For More Information NERSC Website –

NERSC User Group Meeting, September 17, Fork jobs Run a job for immediate execution: # globus-job-run davinci/jobmanager /bin/hostname davinci Submit a job # globus-job-submit davinci/jobmanager /bin/hostname Query submitted job # globus-job-status DONE Get output # globus-job-get-output davinci

NERSC User Group Meeting, September 17, Serial Jobs # globus-job-submit pdsfgrid/jobmanager-sge /bin/date 2/ # globus-job-get-output 2/ Tue Sep 18 14:47:06 PDT 2007

NERSC User Group Meeting, September 17, Parallel Jobs Create RSL script & (host_xcount=4) (xcount=2) (jobtype=mpi) (directory=/usr/common/homes/s/shreyas/sandbox) (executable=/usr/common/homes/s/shreyas/sandbox/cpi) (stdout=x-gass- cache://$(GLOBUS_GRAM_JOB_CONTACT)stdout anExtraTag) (stderr=x-gass- cache://$(GLOBUS_GRAM_JOB_CONTACT)stderr anExtraTag)

NERSC User Group Meeting, September 17, Parallel Jobs Submit Job # globusrun -r jacquardgrid.nersc.gov/jobmanager-pbs -f cpi.rsl -b Query Q # globus-job-run jacquardgrid/jobmanager /usr/common/nsg/torque/bin/qstat Get results # globus-job-get-output

NERSC User Group Meeting, September 17, Output PBS Leader node is jaccn203 Job setup time: Tue Sep 18 14:51:47 PDT 2007 Setting up security Job startup at Tue Sep 18 14:51:52 PDT pi is approximately , Error is wall clock time = Jobs exit status code is Job STDIN/ jacin03-m.nersc.gov completed Tue Sep 18 14:51:54 PDT 2007 Submitted by shreyas/shreyas using mpccc Job Limits: ncpus=1,nodes=4:ppn=2,walltime=00:30:00 Job Resources used: cput=00:00:00,mem=5340kb,vmem=33288kb,walltime=00:00:02 Nodes used: jaccn181 jaccn183 jaccn194 jaccn203 Killing any leftover processes...

NERSC User Group Meeting, September 17, File Transfer Get File from HPSS # globus-url-copy -nodcau \ gsiftp://garchive.nersc.gov//nersc/ccc/shreyas/Animals/gnu file:///home/shreyas/gnu file:///home/shreyas/gnu Third party copy # globus-url-copy gsiftp://pdsfgrid.nersc.gov//home/shreyas/Animals/gnu gsiftp://davinci.nersc.gov//u/s/shreyas/Animals/gnu Uberftp –Provides interactive access to gridftp server

NERSC User Group Meeting, September 17, MyProxy If you use several client systems Copying your.globus directory everywhere can be tedious Myproxy provides central certificate repository Storing cert # myproxy-init -s myproxy.nersc.gov (Valid for 1 week. Use -c for non-default lifetime) Getting cert # myproxy-logon -s myproxy.nersc.gov (Same effect as grid-proxy-init - you have a temporary proxy valid for 12 hours)