Manchester Computing Supercomputing, Visualization & eScience Seamless Access to Multiple Datasets Mike AS Jones ● Demo Run-through

Manchester Computing Supercomputing, Visualization & eScience The SAMD Graphical User Interface

Manchester Computing Supercomputing, Visualization & eScience First obtain Globus Single Sign-on Credentials Create a GSI Proxy Impersonation Certificate

Manchester Computing Supercomputing, Visualization & eScience All communications are securely controlled with the use of the GSI proxy.

Manchester Computing Supercomputing, Visualization & eScience Your Proxy is a public key pair cryptographically signed using your UK eScience Certificate. For SAMD to do this you must unlock your UK eScience Certificate.

Manchester Computing Supercomputing, Visualization & eScience Your UK eScience Certificate is valuable; when stored it should be locked with a long passphrase. For further information see the UK eScience Certificate Authority:

Manchester Computing Supercomputing, Visualization & eScience Proxy initialisation will take a few seconds. “Proxy Initialised” will appear in the Console Output window if the operation was successful.

Manchester Computing Supercomputing, Visualization & eScience The next section: Querying the Datastore. SAMD will probe the MIMAS NS Time-series Dataset. Steps 1, (with steps 2,3,4 & 6 behind the scenes)

Manchester Computing Supercomputing, Visualization & eScience Search for interest rate data: Type “interest rate” in the Search Term Box

Manchester Computing Supercomputing, Visualization & eScience Click Search to Seamlessly search the datastore

Manchester Computing Supercomputing, Visualization & eScience Console Output window shows “Searching ONS databank...”

Manchester Computing Supercomputing, Visualization & eScience A short time later, matching datasets appear in the “Search Results” window. The Console Window shows “Search complete”.

Manchester Computing Supercomputing, Visualization & eScience Scroll left-right-up-down to see all the options. Select the dataset of interest (no pun intended). We select 'seasonally unadjusted, quarterly, inter- bank interest rate data.'

Manchester Computing Supercomputing, Visualization & eScience Save the reference to this data in the saved results window by clicking the arrow.

Manchester Computing Supercomputing, Visualization & eScience A reference to this data is now stored in the SAMD interface. We are now free to search for other data (or select some more from the current search).

Manchester Computing Supercomputing, Visualization & eScience Search for gross domestic product data: Type “gross domestic product” in the Search Term Box

Manchester Computing Supercomputing, Visualization & eScience Click Search to Seamlessly search the datastore

Manchester Computing Supercomputing, Visualization & eScience Again “Searching the ONS database...” appears in the Console Window.

Manchester Computing Supercomputing, Visualization & eScience...and again, a short time later, matching datasets appear in the “Search Results” window. The Console Window shows “Search complete”.

Manchester Computing Supercomputing, Visualization & eScience Scroll to the relevant results.

Manchester Computing Supercomputing, Visualization & eScience Select “quarterly, seasonally adjusted Gross Domestic Product”

Manchester Computing Supercomputing, Visualization & eScience Transfer the reference to the saved results.

Manchester Computing Supercomputing, Visualization & eScience For the purposes of this Demo we now have references to all the data that is necessary.

Manchester Computing Supercomputing, Visualization & eScience However we need this data in a Comma Separated Table. (the default is Space Separated)

Manchester Computing Supercomputing, Visualization & eScience We can choose from a list of available outputs as shown.

Manchester Computing Supercomputing, Visualization & eScience Further configuration of the output format can be set by rearranging the Saved results and selecting the start and end year.

Manchester Computing Supercomputing, Visualization & eScience The next section: Locating a Compute Engine. The HPC engines in a grid tell a Grid Information Index Server about themselves, and we want to find all compute nodes able to run our analysis.

Manchester Computing Supercomputing, Visualization & eScience Select a Jobmanager Type. This is not usually interesting but can be used to select between fork (immediate) and batch (queued) job submission paradigms.

Manchester Computing Supercomputing, Visualization & eScience default: will search for nodes that specify a default. fork: will search for nodes with immediate access. lsf, pbs,... are specific types of queuing systems.

Manchester Computing Supercomputing, Visualization & eScience As this is a demo, we select fork.

Manchester Computing Supercomputing, Visualization & eScience Part of the SAMD project was to take an analysis and write in a high performance computing language. This was compiled for a specific type of compute engine. We specify this here too.

Manchester Computing Supercomputing, Visualization & eScience The analysis for SAMD was compiled for the 64 bit IRIX architecture.

Manchester Computing Supercomputing, Visualization & eScience We select IRIX64

Manchester Computing Supercomputing, Visualization & eScience IRIX64 is selected!

Manchester Computing Supercomputing, Visualization & eScience The SAMD Code was also written to take advantage of parallel computers. We ask for an HPC engine with at least 4 Processors.

Manchester Computing Supercomputing, Visualization & eScience Then we search

Manchester Computing Supercomputing, Visualization & eScience Then we wait (up to a minute).

Manchester Computing Supercomputing, Visualization & eScience The UK level 2 testbed grid has two machines that fit the bill at this time. We choose one.

Manchester Computing Supercomputing, Visualization & eScience We choose bezier, a 32 node infinate reality equipt (we didn't know this), IRIX based Origin 300 at Manchester.

Manchester Computing Supercomputing, Visualization & eScience double clicking this selects it.

Manchester Computing Supercomputing, Visualization & eScience The next section: Getting and moving data. SAMD will orchestrate the movement of data from MIMAS to the HPC engine we just chose. Steps 1 through 8.

Manchester Computing Supercomputing, Visualization & eScience Select references to data from the “Saved Data” Window.

Manchester Computing Supercomputing, Visualization & eScience Select multiple bits as required.

Manchester Computing Supercomputing, Visualization & eScience Click the arrow to seamlessly transfer the data from MIMAS to bezier directly. Formatting data in the transferred results indicates that work is in progress.

Manchester Computing Supercomputing, Visualization & eScience Gobbledegook in the “Transferred Results” window means that the file transferred and is called that on the remote machine.

Manchester Computing Supercomputing, Visualization & eScience The next section: Stage and Submit the Job SAMD will stage (upload in this case the executable) and submit the job on the HPC engine: Step 9.

Manchester Computing Supercomputing, Visualization & eScience Switch to the “Submit Job” Tab.

Manchester Computing Supercomputing, Visualization & eScience Browse for the executable on the local machine.

Manchester Computing Supercomputing, Visualization & eScience A file dialogue box appears.

Manchester Computing Supercomputing, Visualization & eScience Browse to the location of the executable.

Manchester Computing Supercomputing, Visualization & eScience Click open.

Manchester Computing Supercomputing, Visualization & eScience Submit Job Tab updates to show the location of the local copy of the executable.

Manchester Computing Supercomputing, Visualization & eScience Highlight the transferred result (to tell the job where it's data will be on the HPC engine).

Manchester Computing Supercomputing, Visualization & eScience Stage the job.

Manchester Computing Supercomputing, Visualization & eScience The next section: Querying the Datastore. SAMD will probe the MIMAS NS Time-series Dataset. Steps 1, (with steps 2,3,4 & 6 behind the scenes)

Manchester Computing Supercomputing, Visualization & eScience Executable is now being transferred from the local machine to the HPC engine.

Manchester Computing Supercomputing, Visualization & eScience Job status is now Active (ie running) Job status could be PENDING, ERROR, ACTIVE, DONE,...

Manchester Computing Supercomputing, Visualization & eScience The next section: Get the Output SAMD will stage get the output from the HPC engine: Step 10.

Manchester Computing Supercomputing, Visualization & eScience When Job Status is DONE click the Get Output button.

Manchester Computing Supercomputing, Visualization & eScience Console reports that Since it knows the Job is done, it will retrieve the output. (Otherwise it will retrieve the status).

Manchester Computing Supercomputing, Visualization & eScience The output text is displayed and can be saved locally.

Manchester Computing Supercomputing, Visualization & eScience Click save to save file.

Manchester Computing Supercomputing, Visualization & eScience Choose where to save file.

Manchester Computing Supercomputing, Visualization & eScience Choose what to call saved file.

Manchester Computing Supercomputing, Visualization & eScience Click Save button.

Manchester Computing Supercomputing, Visualization & eScience SAMD automatically cleans up files on the HPC engine.