Submit For Job Submission Derrick Kearney HUBzero® Platform for Scientific Collaboration Purdue University This work licensed under Creative Commons See.

Slides:



Advertisements
Similar presentations
Grid Resource Allocation Management (GRAM) GRAM provides the user to access the grid in order to run, terminate and monitor jobs remotely. The job request.
Advertisements

 Statistics package  Graphics package  Programming language  Can be used to share/reproduce analyses  Many new packages being created - can be downloaded.
Riccardo Bruno, INFN.CT Sevilla, 10-14/09/2007 GENIUS Exercises.
Pegasus For Automated Workflows Derrick Kearney HUBzero® Platform for Scientific Collaboration Purdue University This work licensed under Creative Commons.
Oracle Fusion Middleware 11g Abhishek Khanolkar. What is Oracle Fusion? “Oracle Fusion Middleware is a ‘preintegrated’ portfolio of customer- proven software.
Introducing the Command Line CMSC 121 Introduction to UNIX Much of the material in these slides was taken from Dan Hood’s CMSC 121 Lecture Notes.
S/W meeting 18 October 2007RSD 1 Remote Software Deployment Nick West.
CS-502 Fall 2006Project 1, Fork1 Programming Project 1 – Fork.
Project 1CS-3013 A-term Programming Project #1 Forking Processes Due Tuesday, September 8, 11:59 PM.
Operating Systems Review. User Computer, including HW and SW.
Asynchronous Solution Appendix Eleven. Training Manual Asynchronous Solution August 26, 2005 Inventory # A11-2 Chapter Overview In this chapter,
1 Using Workspaces to Develop Simulation Tools Michael McLennan Software Architect HUBzero™ Platform for Scientific Collaboration This work licensed under.
Installing and running COMSOL on a Windows HPCS2008(R2) cluster
Reproducible Environment for Scientific Applications (Lab session) Tak-Lon (Stephen) Wu.
1 Some basic Unix commands u Understand the concept of loggin into and out of a Unix shell u Interact with the system in a basic way through keyboard and.
DIRAC API DIRAC Project. Overview  DIRAC API  Why APIs are important?  Why advanced users prefer APIs?  How it is done?  What is local mode what.
2 $ command Command Line Options ls –a –l hello hi Command Arguments.
Shell Programming, or Scripting Shirley Moore CPS 5401 Fall August 29,
Computer Programming for Biologists Class 2 Oct 31 st, 2014 Karsten Hokamp
1 Uploading and Publishing New Tools Michael McLennan Software Architect HUBzero™ Platform for Scientific Collaboration This work licensed under Creative.
V Avon High School Tech Crew Agenda Old Business –Delete Files New Business –Week 10 Topics: Coming up: –Yearbook Picture: Feb 7 12:20PM.
Help session: Unix basics Keith 9/9/2011. Login in Unix lab  User name: ug0xx Password: ece321 (initial)  The password will not be displayed on the.
Lesson 7-Creating and Changing Directories. Overview Using directories to create order. Managing files in directories. Using pathnames to manage files.
LATTICE TECHNOLOGY, INC. For Version 4.0 and later XVL Processor Tutorial For Version 4.0 and later.
Client Installation StratusLab Tutorial (Orsay, France) 28 November 2012.
MaterialsHub - A hub for computational materials science and tools.  MaterialsHub aims to provide an online platform for computational materials science.
Regression Testing Michael McLennan HUBzero® Platform for Scientific Collaboration Purdue University This work licensed under Creative Commons See license.
The gLite API – PART I Giuseppe LA ROCCA INFN Catania ACGRID-II School 2-14 November 2009 Kuala Lumpur - Malaysia.
Project 1, Command Shell CS-502 (EMC) Fall Programming Project #1 Command Shell CS-502, Operating Systems EMC, Fall 2009 (Slides include materials.
Suite zTPFGI Facilities. Suite Focus Three of zTPFGI’s facilities:  zAutomation  zTREX  Logger.
SAS lecture - week 31 Some Basics of CQUEST The operating system in the ESC lab (1046) is Linux. The operating system in RW labs (107/109 and 211) is Windows.
Nadia LAJILI User Interface User Interface 4 Février 2002.
Javascript Cog Kit By Zhenhua Guo. Grid Applications Currently, most grid related applications are written as separate software. –server side: Globus,
UNIX Commands. Why UNIX Commands Are Noninteractive Command may take input from the output of another command (filters). May be scheduled to run at specific.
LINUX programming 1. INDEX UNIT-III PPT SLIDES Srl. No. Module as per Session planner Lecture No. PPT Slide No. 1.Problem solving approaches in Unix,Using.
Group 1 : Grid Computing Laboratory of Information Technology Supervisors: Alexander Ujhinsky Nikolay Kutovskiy.
Task Space Searcher (TSS) By Tyler Bletsch For CSC714, Prof. Mueller 28 November 2005 Background image adapted from Blatte’s fractal wallpapers (
ENEE150 – 0202 ANDREW GOFFIN Introduction to ENEE150.
 Name: Santiago Bock   Telephone:  Office Number: 5106 Sennott Square  Office Hours: Tuesdays.
Introducing the Rappture Toolkit Michael McLennan Software Architect HUBzero™ Platform for Scientific Collaboration.
Introduction to Scripting Workshop October
Enabling Grids for E-sciencE Workload Management System on gLite middleware - commands Matthieu Reichstadt CNRS/IN2P3 ACGRID School, Hanoi.
CS 245 – Part 1 Using Operating Systems and Networks for Programmers Jiang Guo Dept. of Computer Science California State University Los Angeles.
Using Simulation Workspaces Michael McLennan HUBzero® Platform for Scientific Collaboration Purdue University 1 This work licensed under Creative Commons.
Korea Workshop May GAE CMS Analysis (Example) Michael Thomas (on behalf of the GAE group)
1 Introducing the Rappture Toolkit Michael McLennan Software Architect HUBzero™ Platform for Scientific Collaboration This work licensed under Creative.
Open project in Microsoft Visual Studio → build program in “Release” mode.
HUBbub 2013: Developing hub tools that submit HPC jobs Rob Campbell Purdue University Thursday, September 5, 2013.
1. Starting 1 Let’s Learn Saenthong School, January – February 2016 Teacher: Aj. Andrew Davison, CoE, PSU Hat Yai Campus
Millions of Jobs or a few good solutions …. David Abramson Monash University MeSsAGE Lab X.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Practical using WMProxy advanced job submission.
Advanced Visualization Michael McLennan HUBzero® Platform for Scientific Collaboration Purdue University This work licensed under Creative Commons See.
 Prepared by: Eng. Maryam Adel Abdel-Hady
RI EGI-TF 2010, Tutorial Managing an EGEE/EGI Virtual Organisation (VO) with EDGES bridged Desktop Resources Tutorial Robert Lovas, MTA SZTAKI.
Geant4 GRID production Sangwan Kim, Vu Trong Hieu, AD At KISTI.
SEE-GRID-SCI WRF-ARW model: Grid usage The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures.
Bash Scripting CIRC Summer School 2016 Baowei Liu CIRC Summer School 2016 Baowei Liu.
Advanced Computing Facility Introduction
Practical using C++ WMProxy API advanced job submission
GRID COMPUTING.
Unix Scripts and PBS on BioU
Integrating Scientific Tools and Web Portals
CIRC Winter Boot Camp 2017 Baowei Liu
Using simulation workspaces to “submit” jobs and workflows
Andy Wang Object Oriented Programming in C++ COP 3330
MaterialsHub - A hub for computational materials science and tools.
INTRODUCTION Spice Excel.
CSCI The UNIX System Shell Startup and Variables
Andy Wang Object Oriented Programming in C++ COP 3330
Working in The IITJ HPC System
Presentation transcript:

Submit For Job Submission Derrick Kearney HUBzero® Platform for Scientific Collaboration Purdue University This work licensed under Creative Commons See license online: by-nc-sa/3.0

User's Workspace Terminal Grid HUBzero Infrastructure Running locally in a Workspace Grid $ echo hi Submit Proxy Grid Tool Session Containers

Big Work, Big Computers my_super_awesome_program_to_cure_cancer.exe Grid 100,000 Idle CPUs

Big Work, Big Computers my_super_awesome_program_to_cure_cancer.exe $ ssh password: password rejected Error #9: Your not cool enough $ ssh password: password rejected Error #9: Your not cool enough Grid 100,000 Idle CPUs

Big Work, Big Computers my_super_awesome_program_to_cure_cancer.exe $ ssh password: $ make all install Library Version Errors... $ ssh password: $ make all install Library Version Errors... Grid 100,000 Idle CPUs

Big Work, Big Computers my_super_awesome_program_to_cure_cancer.exe Grid 100,000 Idle CPUs $ ssh password: $./my_super_awesome_program_to_cure_cancer.exe \ datafile1 datafile2 $ ssh password: $./my_super_awesome_program_to_cure_cancer.exe \ datafile1 datafile2

Big Work, Big Computers my_super_awesome_program_to_cure_cancer.exe Grid 100,000 Idle CPUs $ ssh password: password rejected Error #9: Your friend isn't cool enough either! $ ssh password: password rejected Error #9: Your friend isn't cool enough either!

User's Workspace Terminal Grid HUBzero Infrastructure Running locally in a Workspace Grid $ echo hi hi $ Submit Proxy Grid Tool Session Containers

User's Workspace Terminal Grid HUBzero Infrastructure Submitting from a Workspace Tool Session Containers $ submit echo hi Grid Submit Proxy

User's Workspace Terminal Grid HUBzero Infrastructure Submitting from a Workspace Tool Session Containers $ submit echo hi Grid Submit Proxy

User's Workspace Terminal Grid HUBzero Infrastructure Submitting from a Workspace Tool Session Containers $ submit echo hi Grid Submit Proxy

User's Workspace Terminal Grid HUBzero Infrastructure Submitting from a Workspace Tool Session Containers $ submit echo hi hi $ Grid Submit Proxy

User's Workspace Terminal Grid HUBzero Infrastructure The Submit Command Tool Session Containers $ submit --help -l, --local Execute command locally -v, --venue Remote job destination -i, --inputfile Input file -p, --parameters Parameter sweep variables. -d, --data Parametric variable data - csv format... Grid Submit Proxy

User's Workspace Terminal Grid HUBzero Infrastructure Submitting from a Workspace Tool Session Containers Grid Submit Proxy $ submit --local echo hi

User's Workspace Terminal Grid HUBzero Infrastructure Submitting from a Workspace Tool Session Containers Grid Submit Proxy $ submit --local echo hi

User's Workspace Terminal Grid HUBzero Infrastructure Submitting from a Workspace Tool Session Containers Grid Submit Proxy $ submit --local echo hi hi $

Grid HUBzero Infrastructure What commands can I submit? Tool Session Containers Grid Submit Proxy Submitting locally with --local Any executable can be used when submitting locally Submitting to the grid Only executables staged in /apps can be submitted to the grid

Grid HUBzero Infrastructure Try submitting a command locally Tool Session Containers Grid Submit Proxy $ submit --local \ /apps/pegtut/current/bin/sayhi f.a Hello pete!

Grid HUBzero Infrastructure Try submitting a command to the grid Tool Session Containers Grid Submit Proxy $ submit /apps/pegtut/current/bin/sayhi f.a Hello pete!

Sweeping Variables def Vc(Vin,R,C,lowerbound,upperbound): # store all solutions s = {'t':[],'v(t)':[]} for t in frange(lowerbound,upperbound,(upperbound-lowerbound)/100.0): v = Vin*(1-exp(-t/(R*C))) s['t'].append(t) s['v(t)'].append(v) return s Vc(t)= Vin*(1-exp(-t/(R*C)))

Sweeping Variables $ ls /apps/pegtut/current/examples/sim1.py --help Usage: sim1.py [options] Options: -h, --help show this help message and exit --Vin=VIN Input voltage to the system with units of volts (V) --R=R Impedance of the resistor with units of ohms --C=C Capacitance of the capacitor with units of farads (F) --lowerbound=LB Lower bound of the time frame to examine in seconds (s) --upperbound=UB Upper bound of the time frame to examine in seconds (s) --log=LOG Name of the file used to store results Vc(t)= Vin*(1-exp(-t/(R*C)))

Sweeping Variables $ /apps/pegtut/current/examples/sim1.py $ /apps/pegtut/current/examples/sim1.py --Vin 10 $ /apps/pegtut/current/examples/sim1.py --R 100e3 $ /apps/pegtut/current/examples/sim1.py –C 100e-6 Vc(t)= Vin*(1-exp(-t/(R*C)))

Sweeping Variables $ /apps/pegtut/current/examples/sim1.py $ /apps/pegtut/current/examples/sim1.py --Vin 1 $ /apps/pegtut/current/examples/sim1.py --Vin 2 $ /apps/pegtut/current/examples/sim1.py --Vin 3 $ /apps/pegtut/current/examples/sim1.py --Vin 4 $ /apps/pegtut/current/examples/sim1.py --Vin 5 Vc(t)= Vin*(1-exp(-t/(R*C)))

Sweeping Variables Vc(t)= Vin*(1-exp(-t/(R*C))) $ submit -p --Vin

Sweeping Variables Vc(t)= Vin*(1-exp(-t/(R*C))) $ submit -p --Vin

Sweeping Variables Vc(t)= Vin*(1-exp(-t/(R*C))) $ submit -p --Vin

Sweeping Variables Vc(t)= Vin*(1-exp(-t/(R*C))) $ submit -d input.csv./sim1.py --Vin input.csv

Sweeping Variables Vc(t)= Vin*(1-exp(-t/(R*C))) $ submit -d input.csv./sim1.py --Vin –C input.csv 1, 100e-6 2, 100e-6 3, 100e-6 4, 100e-6 5, 100e-6

Sweeping Variables Vc(t)= Vin*(1-exp(-t/(R*C))) $ submit -p params./sim1.py --Vin –C params parameter parameter

Sweeping Variables Vc(t)= Vin*(1-exp(-t/(R*C))) $ cp -r /apps/pegtut/current/examples/capacitor_voltage. $ cd capacitor_voltage $ submit --local -p --Vin $ submit --local -p --Vin $ submit --local -p --Vin $ submit --local -d input.csv./sim1.py --Vin $ submit --local -p params./sim1.py --Vin $ submit --local -p -p \./sim1.py --Vin --C # Check your plots quickly with “./plotsweep.py ” : $./plotsweep.py