Using the EMI testbed ARC middleware Marek Kočan University of P. J. Šafárik, Košice.

Slides:



Advertisements
Similar presentations
Globus Workshop at CoreGrid Summer School 2006 Dipl.-Inf. Hamza Mehammed Leibniz Computing Centre.
Advertisements

FP7-INFRA Enabling Grids for E-sciencE EGEE Induction Grid training for users, Institute of Physics Belgrade, Serbia Sep. 19, 2008.
Development of test suites for the certification of EGEE-II Grid middleware Task 2: The development of testing procedures focused on special details of.
Grid and CDB Janusz Martyniak, Imperial College London MICE CM37 Analysis, Software and Reconstruction.
EMI INFSO-RI The ARC data client in EMI-1 Jon Kerr Nilsen Dept. of Physics, Univ. of Oslo On behalf of the ARC Data PT.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Services Abderrahman El Kharrim
USING THE GLOBUS TOOLKIT This summary by: Asad Samar / CALTECH/CMS Ben Segal / CERN-IT FULL INFO AT:
1 Bridging Clouds with CernVM: ATLAS/PanDA example Wenjing Wu
Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks GINGIN Grid Interoperation on Data Movement.
Building service testbeds on FIRE D5.2.5 Virtual Cluster on Federated Cloud Demonstration Kit August 2012 Version 1.0 Copyright © 2012 CESGA. All rights.
EMI INFSO-RI EMI roadmap (development plan) Balázs Kónya (Lund University, Sweden) EMI Technical Director 25 th October 2010, Brussels, EGI-TCB.
Overview of the NorduGrid Information System Balázs Kónya 3 rd NorduGrid Workshop 23 May, 2002, Helsinki.
Grid Computing - AAU 14/ Grid Computing Josva Kleist Danish Center for Grid Computing
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Middleware: gLite Information Systems (IS) EGEE Tutorial 23 rd APAN Meeting,
GRID Computing: Ifrastructure, Development and Usage in Bulgaria M. Dechev, G. Petrov, E. Atanassov.
INFSO-RI Enabling Grids for E-sciencE Logging and Bookkeeping and Job Provenance Services Ludek Matyska (CESNET) on behalf of the.
Computational grids and grids projects DSS,
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks gLite IPv6 compliance project tests Further.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Information System on gLite middleware Vincent.
Overview of user client usage: ARC Iván Márton Zsombor Nagy.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE middleware: gLite Data Management EGEE Tutorial 23rd APAN Meeting, Manila Jan.
June 24-25, 2008 Regional Grid Training, University of Belgrade, Serbia Introduction to gLite gLite Basic Services Antun Balaž SCL, Institute of Physics.
INFSO-RI Enabling Grids for E-sciencE OSG-LCG Interoperability Activity Author: Laurence Field (CERN)
EMI is partially funded by the European Commission under Grant Agreement RI Discovering Infrastructures with EMI Registry (EMIR) Emidio Giorgio.
1 Grid2Win: porting of gLite middleware to Windows Dario Russo INFN Catania
INFSO-RI Enabling Grids for E-sciencE Αthanasia Asiki Computing Systems Laboratory, National Technical.
EMI INFSO-RI Overview of the EMI development objectives Balázs Kónya (Lund University) EMI Technical Director EMI All Hands Prague, 23rd November.
Installation and Configuration of A-REX Iván Márton (NIIFI) Zsombor Nagy (NIIFI)
High-Performance Computing Lab Overview: Job Submission in EDG & Globus November 2002 Wei Xing.
FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America Grid2Win: Porting of gLite middleware to.
© Geodise Project, University of Southampton, Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003.
FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America gLite Information System Claudio Cherubino.
EMI INFSO-RI ARC tools for revision and nightly functional tests Jozef Cernak, Marek Kocan, Eva Cernakova (P. J. Safarik University in Kosice, Kosice,
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Grid2Win : gLite for Microsoft Windows Roberto.
The NorduGrid toolkit user interface Mattias Ellert Presented at the 3 rd NorduGrid workshop, Helsinki,
EGEE-II INFSO-RI Enabling Grids for E-sciencE Command Line Grid Programming Spiros Spirou Greek Application Support Team NCSR “Demokritos”
GIIS Implementation and Requirements F. Semeria INFN European Datagrid Conference Amsterdam, 7 March 2001.
1 Egrid portal Stefano Cozzini and Angelo Leto. 2 Egrid portal Based on P-GRADE Portal 2.3 –LCG-2 middleware support: broker, CEs, SEs, BDII –MyProxy.
European Middleware Initiative (EMI) – Training Kathryn Cassidy, TCD EMI NA2.
Gennaro Tortone, Sergio Fantinel – Bologna, LCG-EDT Monitoring Service DataTAG WP4 Monitoring Group DataTAG WP4 meeting Bologna –
EMI Inter-component and Large Scale Testing Infrastructure Danilo Dongiovanni INFN-CNAF.
Co-ordination & Harmonisation of Advanced e-Infrastructures for Research and Education Data Sharing Research Infrastructures Grant Agreement n
EMI is partially funded by the European Commission under Grant Agreement RI Common Authentication Library Daniel Kouril, for the CaNL PT EGI CF.
EMI INFSO-RI Testbed for project continuous Integration Danilo Dongiovanni (INFN-CNAF) -SA2.6 Task Leader Jozef Cernak(UPJŠ, Kosice, Slovakia)
Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA Grid2Win : gLite for Microsoft Windows Elisa Ingrà - INFN.
A System for Monitoring and Management of Computational Grids Warren Smith Computer Sciences Corporation NASA Ames Research Center.
E-science grid facility for Europe and Latin America Updates on Information System Annamaria Muoio - INFN Tutorials for trainers 01/07/2008.
John Gordon EMI TF and EGI CF March 2012 Accounting Workshop.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Services for Distributed e-Infrastructure Access Tiziana Ferrari on behalf.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.
Implementation of GLUE 2.0 support in the EMI Data Area Elisabetta Ronchieri on behalf of JRA1’s GLUE 2.0 Working Group INFN-CNAF 13 April 2011, EGI User.
SEE-GRID-SCI WRF-ARW model: Grid usage The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures.
EMI INFSO-RI EMI 1 (Kebnekaise) Updates C. Aiftimiei (INFN) EMI Release Manager.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI solution for high throughput data analysis Peter Solagna EGI.eu Operations.
EMI is partially funded by the European Commission under Grant Agreement RI EMI Status And Plans Laurence Field, CERN Towards an Integrated Information.
Argus EMI Authorization Integration
Grid2Win Porting of gLite middleware to Windows XP platform
Delegation methods in EMI
Practical: The Information Systems
gLite Grid Services Salma Saber
EMI Interoperability Activities
Grid2Win: Porting of gLite middleware to Windows XP platform
Towards an Integrated Information system: the EMI view
Grid2Win: Porting of gLite middleware to Windows XP platform
Information System Virginia Martín-Rubio Pascual
EMI 1 (Kebnekaise) Updates
Grid2Win: Porting of gLite middleware to Windows XP platform
Data services in gLite “s” gLite and LCG.
EGEE Middleware: gLite Information Systems (IS)
Presentation transcript:

Using the EMI testbed ARC middleware Marek Kočan University of P. J. Šafárik, Košice

EMI INFSO-RI /05/2010 EMI Hot Topic, JSC, FZJ 2 Outline ARC services available ARC Information system (how to query/register) ARC infrastructure usage

EMI INFSO-RI ARC services available Instant CA – demo user/host certificates generation ( Classic ARC and WS-ARC Grid-Monitors ( emi.grid.upjs.sk/gridmonitor.html) emi.grid.upjs.sk/gridmonitor.html Classic ARC GIIS and WS-ARC ISIS Cloud ( ) Classic ARC CE WS-ARC CE Unixacl Storage Element ARC echo service

EMI INFSO-RI ARC Information system ARC GIIS – Scalable, production quality dynamic distributed information system allowing registration of Classic ARC Services – OpenLDAP-based system which is derived from the Globus Monitoring and Discovery Services framework – It uses BDII together with a updated MDS LDAP Schema – More info at WS-ARC Information System Indexing Services (ISIS) Cloud – Forms a set of information containers – Every generic ARC Service pushes information about itself into a nearby ISIS container – The information stored in every ISIS is propagated among all ISIS services. – More info at

EMI INFSO-RI ARC Information system (cont.) Querying information from Classic ARC GIIS – ldapsearch -h arc-emi.grid.upjs.sk -p x –b 'mds-vo-name=ARC- EMI,o=grid' -s base giisregistrationstatus Querying information from ISIS Cloud – isistest -i -m Query "/RegEntry"

EMI INFSO-RI ARC Information system (cont.) Registering resource to Classic ARC GIIS – needs to add following block to arc.conf #cluster registration block [infosys/cluster/registration/clustertoARC-EMI] targethostname="arc-emi.grid.upjs.sk" targetport="2135" targetsuffix="mds-vo-name=ARC-EMI,o=grid" regperiod="120"

EMI INFSO-RI ARC Information system (cont.) Registering WS-ARC service to WS-ARC ISIS Cloud PT75S/infosys:Period> "your service endpoint" PT1M/infosys:Expiration> 4 or [register] index_server1= ["service type"] endpoint="your service endpoint"

EMI INFSO-RI ARC infrastructure usage Generating proxy credentials arcproxy -S testers.eu-emi.eu man arcproxy for more info

EMI INFSO-RI ARC infrastructure usage arcsub – Submits jobs to grid enabled computing resources – Supports submission to Classic ARC CE, WS-ARC CE, gLite Cream, Unicore (in experimental version currently) – Submits jobs described in XRSL, JSDL and JDL formats – man arcsub for more info arcsub jsdl_hostname.xml – Submits job to randomly chosen cluster (from set of registered clusters) arcsub -c ARC0:ldap://rc0.grid.upjs.sk:2135/nordugrid-cluster- name=rc0.grid.upjs.sk,Mds-Vo-name=local,o=grid jsdl_hostname.xml – Submits job to concrete Classic ARC cluster arcsub -c ARC1: jdl_hostname.jdl – Submits job to concrete WS-ARC cluster

EMI INFSO-RI ARC infrastructure usage (cont.) arcstat – obtains the status of jobs that have been submitted to grid enabled resources – man arcstat for more info arcstat – Retrieves status of concrete job arcstat -a – Retrieves status of all jobs submitted (and not already cleaned)

EMI INFSO-RI ARC infrastructure usage (cont.) arccat – Can be used to view the stdout or stderr of a running job(s) – man arccat for more info arccat – Prints stdout of concrete running job arccat -a – Prints stdout of all running jobs

EMI INFSO-RI ARC infrastructure usage (cont.) arcget – Retrieves the results from a job – man arcget for more info arcget –k – Retrieves the results of finished job and keeps files on cluster (-k option) arcget –a – Retrieves results of all finished jobs and cleans all files on cluster

EMI INFSO-RI ARC infrastructure usage (cont.) arckill – Kills running job(s) – man arckill for more info arckill – Kills concrete running job arckill –a – Kills all running jobs

EMI INFSO-RI ARC infrastructure usage (cont.) arcclean – Removes a job(s) from the remote cluster – man arcclean for more info arcclean – Removes job from the remote cluster arcclean –a – Removes all jobs from the remote cluster

EMI INFSO-RI ARC infrastructure usage (cont.) arcls – Lists files and directories in grid storage elements and file index catalogues – Following protocols may be used file (file:// prefix may be omitted), http, https, httpg, ftp, gsiftp, lfc, rls, srm – man arcls for more info arcls –l – Lists files/directories generated by running job (-l for long format) arcls gsiftp://rc0.grid.upjs.sk:2811/unixacl/ – Lists content of Unixacl SE

EMI INFSO-RI ARC infrastructure usage (cont.) arccp – copies files to, from and between grid storage elements – Following protocols may be used file (file:// prefix may be omitted), http, https, httpg, ftp, gsiftp, lfc, rls, srm – man arccp for more info arccp fileToGSI gsiftp://rc0.grid.upjs.sk:2811/unixacl/uploadedFile – Uploads file to an Unixacl SE arccp gsiftp://rc0.grid.upjs.sk:2811/unixacl/10mb-file downloadedFile – Downloads file from Unixacl SE

EMI INFSO-RI ARC infrastructure usage (cont.) arcrm – deletes files on grid storage elements – Following protocols may be used file (file:// prefix may be omitted), http, https, httpg, ftp, gsiftp, lfc, rls, srm – man arrm for more info arcrm gsiftp://rc0.grid.upjs.sk:2811/unixacl/uploadedFile – removes file from Unixacl SE

EMI INFSO-RI ARC infrastructure usage (cont.) Additional commands – arcinfo - obtains status and information of clusters on the grid – arcecho - client for the ARC echo service – arcmigrate - migrates queued jobs from one cluster to another cluster. Currently migration only works between clusters running A-REX (WS-ARC CE) – arcresume – resumes a job that has been submitted to grid enabled resources. – arcsync - synchronizes local jobs list with the information at a given cluster or index server – arcresub - resubmits jobs to grid enabled computing resources

EMI is partially funded by the European Commission under Grant Agreement RI Thank you!