Community Grids Lab1 CICC Project Meeting VOTable Developed VotableToSpreadsheet Service which accepts VOTable file location as an input, converts to Excel.

Slides:



Advertisements
Similar presentations
/ 1 N. Williams Grid Middleware Experiences Nadya Williams OCI Grid Computing, University of Zurich
Advertisements

Jaime Frey, Todd Tannenbaum Computer Sciences Department University of Wisconsin-Madison OGF.
Jaime Frey Computer Sciences Department University of Wisconsin-Madison OGF 19 Condor Software Forum Routing.
Community Grids Lab1 CICC Project Meeting Developed two web services (VotableToTabbedFileService – convert votable to tabbed file) and (ExtractionFromSingleVotableService.
Generic MPI Job Submission by the P-GRADE Grid Portal Zoltán Farkas MTA SZTAKI.
Test harness and reporting framework Shava Smallen San Diego Supercomputer Center Grid Performance Workshop 6/22/05.
WS-JDML: A Web Service Interface for Job Submission and Monitoring Stephen M C Gough William Lee London e-Science Centre Department of Computing, Imperial.
MyProxy Guy Warner NeSC Training.
J. Hanton - P. Herquet - F. Lequeux - A. Romeyer1 CONDOR-G Installation July 2004 : one independent PC for Grid FTP as a client to UCL August 2004 : complete.
Grid Resource Allocation Management (GRAM) GRAM provides the user to access the grid in order to run, terminate and monitor jobs remotely. The job request.
Dan Bradley Computer Sciences Department University of Wisconsin-Madison Schedd On The Side.
CERN LCG Overview & Scaling challenges David Smith For LCG Deployment Group CERN HEPiX 2003, Vancouver.
CSF4, SGE and Gfarm Integration Zhaohui Ding Jilin University.
Building and Testing OGCE Software on the NMI Build and Test Facility Marlon Pierce Indiana University.
1 Concepts of Condor and Condor-G Guy Warner. 2 Harvesting CPU time Teaching labs. + Researchers Often-idle processors!! Analyses constrained by CPU time!
Setting up of condor scheduler on computing cluster Raman Sehgal NPD-BARC.
A Computation Management Agent for Multi-Institutional Grids
WP 1 Grid Workload Management Massimo Sgaravatto INFN Padova.
Dr. David Wallom Use of Condor in our Campus Grid and the University September 2004.
CMS HLT production using Grid tools Flavia Donno (INFN Pisa) Claudio Grandi (INFN Bologna) Ivano Lippi (INFN Padova) Francesco Prelz (INFN Milano) Andrea.
GRID workload management system and CMS fall production Massimo Sgaravatto INFN Padova.
Workload Management Workpackage Massimo Sgaravatto INFN Padova.
GRID Workload Management System Massimo Sgaravatto INFN Padova.
Workload Management Massimo Sgaravatto INFN Padova.
First steps implementing a High Throughput workload management system Massimo Sgaravatto INFN Padova
Evaluation of the Globus GRAM Service Massimo Sgaravatto INFN Padova.
Minerva Infrastructure Meeting – October 04, 2011.
Introduction to Condor DMD/DFS J.Knudstrup December 2005.
Miron Livny Computer Sciences Department University of Wisconsin-Madison Harnessing the Capacity of Computational.
Alain Roy Computer Sciences Department University of Wisconsin-Madison An Introduction To Condor International.
Vladimir Litvin, Harvey Newman Caltech CMS Scott Koranda, Bruce Loftis, John Towns NCSA Miron Livny, Peter Couvares, Todd Tannenbaum, Jamie Frey Wisconsin.
National Alliance for Medical Image Computing Grid Computing with BatchMake Julien Jomier Kitware Inc.
Track 1: Cluster and Grid Computing NBCR Summer Institute Session 2.2: Cluster and Grid Computing: Case studies Condor introduction August 9, 2006 Nadya.
The Glidein Service Gideon Juve What are glideins? A technique for creating temporary, user- controlled Condor pools using resources from.
Workload Management WP Status and next steps Massimo Sgaravatto INFN Padova.
Condor Tugba Taskaya-Temizel 6 March What is Condor Technology? Condor is a high-throughput distributed batch computing system that provides facilities.
03/27/2003CHEP20031 Remote Operation of a Monte Carlo Production Farm Using Globus Dirk Hufnagel, Teela Pulliam, Thomas Allmendinger, Klaus Honscheid (Ohio.
Grids and Portals for VLAB Marlon Pierce Community Grids Lab Indiana University.
Grid Computing I CONDOR.
COMP3019 Coursework: Introduction to GridSAM Steve Crouch School of Electronics and Computer Science.
Condor Birdbath Web Service interface to Condor
1 Overview of the Application Hosting Environment Stefan Zasada University College London.
3-2.1 Topics Grid Computing Meta-schedulers –Condor-G –Gridway Distributed Resource Management Application (DRMAA) © 2010 B. Wilkinson/Clayton Ferner.
1 st December 2003 JIM for CDF 1 JIM and SAMGrid for CDF Mòrag Burgon-Lyon University of Glasgow.
Part 6: (Local) Condor A: What is Condor? B: Using (Local) Condor C: Laboratory: Condor.
DataGrid WP1 Massimo Sgaravatto INFN Padova. WP1 (Grid Workload Management) Objective of the first DataGrid workpackage is (according to the project "Technical.
Resource Brokering in the PROGRESS Project Juliusz Pukacki Grid Resource Management Workshop, October 2003.
Condor: High-throughput Computing From Clusters to Grid Computing P. Kacsuk – M. Livny MTA SYTAKI – Univ. of Wisconsin-Madison
Grid Compute Resources and Job Management. 2 Local Resource Managers (LRM)‏ Compute resources have a local resource manager (LRM) that controls:  Who.
July 11-15, 2005Lecture3: Grid Job Management1 Grid Compute Resources and Job Management.
Review of Condor,SGE,LSF,PBS
Condor Project Computer Sciences Department University of Wisconsin-Madison Grids and Condor Barcelona,
Pilot Factory using Schedd Glidein Barnett Chiu BNL
Job Submission with Globus, Condor, and Condor-G Selim Kalayci Florida International University 07/21/2009 Note: Slides are compiled from various TeraGrid.
Grid Compute Resources and Job Management. 2 Grid middleware - “glues” all pieces together Offers services that couple users with remote resources through.
Jaime Frey Computer Sciences Department University of Wisconsin-Madison What’s New in Condor-G.
The Gateway Computational Web Portal Marlon Pierce Indiana University March 15, 2002.
Miron Livny Computer Sciences Department University of Wisconsin-Madison Condor and (the) Grid (one of.
Matthew Farrellee Computer Sciences Department University of Wisconsin-Madison Condor and Web Services.
JSS Job Submission Service Massimo Sgaravatto INFN Padova.
Open Science Grid Build a Grid Session Siddhartha E.S University of Florida.
Status of Globus activities Massimo Sgaravatto INFN Padova for the INFN Globus group
Grid Workload Management (WP 1) Massimo Sgaravatto INFN Padova.
The OxGrid Resource Broker David Wallom. Overview OxGrid Resource Broking Why build our own Job Submission and other tools Future developments.
First evaluation of the Globus GRAM service Massimo Sgaravatto INFN Padova.
CICC Project Meeting Developed two web services (VotableToTabbedFileService – convert votable to tabbed file) and (ExtractionFromSingleVotableService –
Genre1: Condor Grid: CSECCR
The Condor JobRouter.
GRID Workload Management System for CMS fall production
Condor-G: An Update.
Presentation transcript:

Community Grids Lab1 CICC Project Meeting VOTable Developed VotableToSpreadsheet Service which accepts VOTable file location as an input, converts to Excel spreadsheet using jakartas HSSF(POI Project's pure Java implementation of the Excel '97(-2002 ) ml and returns excel file URL as an output. ml

Community Grids Lab2 condor Condor is a workload management system Users submit jobs to Condor, Condor places them into a queue, chooses when and where to run the jobs based upon a policy, carefully monitors their progress, and ultimately informs the user upon completion

Community Grids Lab3 We have installed and configured Condor 6.8 on Linux machine. We are able to submit and execute simple jobs using vanilla universe like, universe = vanilla executable = /home/condor/test.sh output = test.out error = test.error log = test.log Where test.sh can have simple Linux command #!/bin/sh echo $HOSTNAME

Community Grids Lab4 We have also configured condor-g (window to the Grid) Condor-g is nothing but a condor uses Globus to provide underlying software needed to utilize grid resources such as authentication, remote program execution and data transfer Condor-G is a program that manages both a queue of jobs and the resources from one or more sites where those jobs can execute. It communicates with these resources and transfers files to and from these resources using Globus mechanisms. (In particular, Condor-G uses the GRAM protocol for job submission, and it runs a local GASS server for file transfers).

Community Grids Lab5 We are now able to submit globus jobs to gf1, NCSA and SDSC teragrid machines Example executable = test.sh output = test.out error = test.error log = test.log universe = grid grid_type = gt2 globusscheduler = gf1.ucs.indiana.edu/job- manager leave_in_queue = jobstatus == 4 queue

Community Grids Lab6 We are also able to submit globus jobs to gf1, NCSA and SDSC teragrid by advertising classad of machines. Example of classad for gf1.ucs.indiana.edu MyType = "Machine" TargetType = "Job" Name = "condorTest02" Machine = "condorTest02" gatekeeper_url = "gf1.ucs.indiana.edu/jobmanager" UpdatesSequenced = 9 CurMatches = 0 Requirements = TARGET.JobUniverse == 9 Rank = CurrentRank = OpSys = "LINUX" Arch = "INTEL" State = "Unclaimed" Activity = "Idle" LoadAvg = Memory = 2048 WantAdRevaluate = True StartdIpAddr = " "

Community Grids Lab7 And submit description file for classad mechanism universe = grid grid_type = gt2 notification = never globusscheduler = $$(gatekeeper_url) executable = test.sh transfer_executable = true output = hg-host.$(Cluster).out error = hg-host.$(Cluster).error log = hg-host.$(Cluster).log requirements = TARGET.gatekeeper_url =!= UNDEFINED queue

Community Grids Lab8 We are also able to configured condor birdbath to submit jobs through condor SOAP API Right now we are working on modifying OGCE condor job submission portal code to work with condor 6.8 and globus jobs. All information about condor installation, setup, job submission and SOAP API is available on