Liudmila Stepanova SINP-MSU team Testing gLite Worker Nodes LCG Dubna, Jul 26, 2007.

Slides:



Advertisements
Similar presentations
EGEE is a project funded by the European Union under contract IST EGEE Tutorial Turin, January Hands on Job Services.
Advertisements

EU 2nd Year Review – Jan – Title – n° 1 WP1 Speaker name (Speaker function and WP ) Presentation address e.g.
Workload management Owen Maroney, Imperial College London (with a little help from David Colling)
INFSO-RI Enabling Grids for E-sciencE Workload Management System and Job Description Language.
Workload Management meeting 07/10/2004 Federica Fanzago INFN Padova Grape for analysis M.Corvo, F.Fanzago, N.Smirnov INFN Padova.
Development of test suites for the certification of EGEE-II Grid middleware Task 2: The development of testing procedures focused on special details of.
INFSO-RI Enabling Grids for E-sciencE EGEE Middleware The Resource Broker EGEE project members.
Special Jobs Claudio Cherubino INFN - Catania. 2 MPI jobs on gLite DAG Job Collection Parametric jobs Outline.
Basic Grid Job Submission Alessandra Forti 28 March 2006.
DIRAC API DIRAC Project. Overview  DIRAC API  Why APIs are important?  Why advanced users prefer APIs?  How it is done?  What is local mode what.
CN1260 Client Operating System Kemtis Kunanuraksapong MSIS with Distinction MCT, MCITP, MCTS, MCDST, MCP, A+
1 Kolkata, Asia Joint CHAIN/EU-IndiaGrid2/EPIKH School for Grid Site Administrators, The EPIKH Project (Exchange Programme.
Chapter Seven Advanced Shell Programming. 2 Lesson A Developing a Fully Featured Program.
FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America Special Jobs Matias Zabaljauregui UNLP.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) WMPROXY API Python & C++ Diego Scardaci
GLite D. Zaborov, SA1-SA3 meeting, Dubna1 Dmitry Zaborov (SINP-MSU-Moscow) Dubna, Jul 26, 2007 Testing 3.0 under SLC 4.
The gLite API – PART I Giuseppe LA ROCCA INFN Catania ACGRID-II School 2-14 November 2009 Kuala Lumpur - Malaysia.
Computational grids and grids projects DSS,
:: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: GridKA School 2009 MPI on Grids 1 MPI On Grids September 3 rd, GridKA School 2009.
Enabling Grids for E-sciencE Workload Management System on gLite middleware Matthieu Reichstadt CNRS/IN2P3 ACGRID School, Hanoi (Vietnam)
Interactive Job Monitor: CafMon kill CafMon tail CafMon dir CafMon log CafMon top CafMon ps LcgCAF: CDF submission portal to LCG resources Francesco Delli.
DataGrid WP1 Massimo Sgaravatto INFN Padova. WP1 (Grid Workload Management) Objective of the first DataGrid workpackage is (according to the project "Technical.
Nadia LAJILI User Interface User Interface 4 Février 2002.
INFSO-RI Enabling Grids for E-sciencE Workload Management System Mike Mineter
LCG Middleware Testing in 2005 and Future Plans E.Slabospitskaya, IHEP, Russia CERN-Russia Joint Working Group on LHC Computing March, 6, 2006.
Group 1 : Grid Computing Laboratory of Information Technology Supervisors: Alexander Ujhinsky Nikolay Kutovskiy.
EGEE-III INFSO-RI Enabling Grids for E-sciencE Nov. 18, EGEE and gLite are registered trademarks gLite Middleware Usage Dusan.
EGEE-III INFSO-RI Enabling Grids for E-sciencE Feb. 06, Introduction to High Performance and Grid Computing Faculty of Sciences,
Job Management DIRAC Project. Overview  DIRAC JDL  DIRAC Commands  Tutorial Exercises  What do you have learned? KEK 10/2012DIRAC Tutorial.
Unified scripts ● Currently they are composed of a main shell script and a few auxiliary ones that handle mostly the local differences. ● Local scripts.
E-infrastructure shared between Europe and Latin America GENIUS PORTAL Valeria Ardizzone INFN-Catania 1° EELA Grid School Itacuruçà Island, State of Rio.
1 Andrea Sciabà CERN Critical Services and Monitoring - CMS Andrea Sciabà WLCG Service Reliability Workshop 26 – 30 November, 2007.
1 Grid2Win: porting of gLite middleware to Windows Dario Russo INFN Catania
FP7-INFRA Enabling Grids for E-sciencE EGEE Induction Grid training for users, Institute of Physics Belgrade, Serbia Sep. 19, 2008.
INFSO-RI Enabling Grids for E-sciencE Αthanasia Asiki Computing Systems Laboratory, National Technical.
CE: compute element TP: CE & WN Compute Element Worker Node Installation configuration.
INFSO-RI Enabling Grids for E-sciencE Job Submission Tutorial (material from INFN Catania)
Workload Management System Jason Shih WLCG T2 Asia Workshop Dec 2, 2006: TIFR.
1 DIRAC Job submission A.Tsaregorodtsev, CPPM, Marseille LHCb-ATLAS GANGA Workshop, 21 April 2004.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Command Line Grid Programming Spiros Spirou Greek Application Support Team NCSR “Demokritos”
INFSO-RI Enabling Grids for E-sciencE EGEE is a project funded by the European Union under contract IST Job sandboxes.
EGEE-II INFSO-RI Enabling Grids for E-sciencE YAIM Overview MiMOS Grid tutorial HungChe, ASGC OPS Team.
Data Management The European DataGrid Project Team
Development of test suites for the certification of EGEE-II Grid middleware Task 2: The development of testing procedures focused on special details of.
E-infrastructure shared between Europe and Latin America FP6−2004−Infrastructures−6-SSA Special Jobs Valeria Ardizzone INFN - Catania.
Liudmila Stepanova WN Testing. submit_test.sh Script provides an interface to the submission of a job via job management system gLite WMS via NS. Script.
IP Address Checker SNMP Checker Duplication Checker IP address range Available IP addresses SNMP Agents and Routers Network ICMP Agent MIB Managed Device.
EGEE is a project funded by the European Union under contract IST Experiment Software Installation toolkit on LCG-2
User Interface UI TP: UI User Interface installation & configuration.
Presentation of the results khiat abdelhamid
Job Management Beijing, 13-15/11/2013. Overview Beijing, /11/2013 DIRAC Tutorial2  DIRAC JDL  DIRAC Commands  Tutorial Exercises  What do you.
Enabling Grids for E-sciencE Work Load Management & Simple Job Submission Practical Shu-Ting Liao APROC, ASGC EGEE Tutorial.
Grid2Win Porting of gLite middleware to Windows XP platform
Information System testing for LCG-1
Real Time Fake Analysis at PIC
Advanced Topics: MPI jobs
gLite MPI Job Amina KHEDIMI CERIST
Workload Management System on gLite middleware
Special jobs with the gLite WMS
Grid2Win: Porting of gLite middleware to Windows XP platform
Grid2Win: Porting of gLite middleware to Windows XP platform
Workload Management System
ETL Job Scheduler Job Database Server User Interface Scheduler
Job workflow Pre production operations:
5. Job Submission Grid Computing.
gLite Job Management Christos Theodosiou
The EU DataGrid Fabric Management Services
GENIUS Grid portal Hands on
Hands on Session: DAG Job Submission
Job Submission M. Jouvin (LAL-Orsay)
Presentation transcript:

Liudmila Stepanova SINP-MSU team Testing gLite Worker Nodes LCG Dubna, Jul 26, 2007

Goal of work The main goal of work is to create a set of tests for testing Worker Nodes (WN).The set of tests create by Dr. Zaborov (UI-test set) was used as basis.

The decision of a problem To achieve the goal the following steps were performed: 1. Choose the test set for Worker Node (WN) from Dr. Zaborov’s UI-test set ; 2. Create shell scripts for test set running (submit_test.sh, test.sh) ; 3. Create additional test (WN-inf-edg-wl-scratch.sh)‏ 4. Run shell scripts, convert test results to human- readable format, analyze test results. 5.The tests have been submitted on all clusters of Tier2.

submit_test.sh The script provides an interface to the submission of a job via job management system gLite WMS. Script also creates a JDL file with next attributes: Executable=“test.sh”; InputSandbox={ “test.sh”, “zaborov_tests.tgz”, “tests.list” } Arguments=“LFC_HOST”; StdOutput="std.html"; StdError="std.err"; OutputSandbox = { "std.html", "std.err", "log.tgz" }; Where: - test.sh is an executable script - zaborov_tests.tgz is the tarball of all tests. - tests.list is the list of tests to perform. - LFC_HOST=prod-lfc-shared-central.cern.ch - std.html result table in the HTML format - log.tgz is the tarball of test log files

Two kinds of a choice of CE are incorporated in the script 1.All the jobs are to be run on any CE in the domain cern.ch.This can be achieved using in the JDL file the following expression: Requirements=RegExp(".*cern.*",other.GlueCEUniq ueID)‏ 2.User chooses all CE get for everyone names of sites using ldapsearch: ldapsearch -x -LLL -h lcgbdii02.gridpp.rl.ac.uk -p b "mds-vo-name=$SITENAME, \ mds-vo-name=local,o=grid" \ '(&(GlueCEUniqueID=*)(GlueCEAccessControlBaseRu le=vo\:dteam))'|grep ^GlueCEUniqueID |cut -d' ' -f2 |tr \ "\n" " "`)

test.sh The script reads names of tests from the file tests.list and starts them. The output in HTML format is formed HTML table: Test Result INFN-TORINO

Made modifications WN tests are based on UI tests of Zaborov. UI-inf-ldapsearch.sh is modified for WN. The name of modified script is WN -inf-ldapsearch.sh This script for testing WN does not use -H giishost:port command line option. Environment variable $SITE_GIIS_URL is used for definition GIIS: GIIS_HOST=`echo $SITE_GIIS_URL` SITE_GIIS_URL=`host $GIIS_HOST | sed -e 's/^.*address//'` GIIS=ldap://$SITE_GIIS_URL:2135 IP address of GIIS_HOST is used as there are sites( INFN- TORINO,Ru-Troitsk-INR-LCG2) where WN are located in a local network and PBS_HOST=$CE_HOST. CE_HOST in /etc/hosts has local ip address.

New test The new test script WN-inf-edg-wl-scratch.sh checks the presence of EDG_WL_SCRATCH directory. If the directory exists script shows total disk space, used space and available space on physical device which EDG_WL_SWCRATCH directory is located.

Example of output: CE=t2-ce-01.to.infn.it WN=egee-wn156.torinoegee VO=dteam SE_HOST=grid009.to.infn.it LFC_HOME=/grid/dteam Test Result WN-commands-exist-all.sh INFO WN-data-lcg-alias.sh OK Clicking on the result button gives the logs of test: WN-data-lcg-cp.sh OK === info EDG_WL_SCRATCH test === WN-data-lcg-cr.sh OK Variable EDG_WL_SCRATCH not assign WN-data-lcg-list.sh OK WN-data-lfc-acl.sh OK WN-data-lfc-comment.sh OK WN-data-lfc-ln.sh OK WN-inf-edg-wl-scratch.sh NOTICE