4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Wide Area Workload Management Work Package DATAGRID project Parallel session report Cristina Vistoli.

Slides:



Advertisements
Similar presentations
WP1 Grid Workload Management Massimo Sgaravatto INFN Padova
Advertisements

Installation and evaluation of the Globus toolkit WP 1 INFN-GRID Workload management WP 1 DATAGRID WP 2.1 INFN-GRID Massimo Sgaravatto INFN Padova.
INFN & Globus activities Massimo Sgaravatto INFN Padova.
Grid Workload Management (WP 1) Report to INFN-GRID TB Massimo Sgaravatto INFN Padova.
WP 1 Members of Wp1: INFN Cesnet DATAMAT PPARC. WP 1 What does WP1 do? Broker Submission mechanism JDL/JCL and other UIs Logging computational economics.
Work Package 1 Installation and Evaluation of the Globus Toolkit Massimo Sgaravatto INFN Padova.
EU 2nd Year Review – Jan – Title – n° 1 WP1 Speaker name (Speaker function and WP ) Presentation address e.g.
NorduGrid Grid Manager developed at NorduGrid project.
Evaluation of the Globus Toolkit: Status Roberto Cucchi – INFN Cnaf Antonia Ghiselli – INFN Cnaf Giuseppe Lo Biondo – INFN Milano Francesco Prelz – INFN.
Condor-G: A Computation Management Agent for Multi-Institutional Grids James Frey, Todd Tannenbaum, Miron Livny, Ian Foster, Steven Tuecke Reporter: Fu-Jiun.
Condor and GridShell How to Execute 1 Million Jobs on the Teragrid Jeffrey P. Gardner - PSC Edward Walker - TACC Miron Livney - U. Wisconsin Todd Tannenbaum.
WP 1 Grid Workload Management Massimo Sgaravatto INFN Padova.
CMS HLT production using Grid tools Flavia Donno (INFN Pisa) Claudio Grandi (INFN Bologna) Ivano Lippi (INFN Padova) Francesco Prelz (INFN Milano) Andrea.
DataGrid is a project funded by the European Union 22 September 2003 – n° 1 EDG WP4 Fabric Management: Fabric Monitoring and Fault Tolerance
EU-GRID Work Program Massimo Sgaravatto – INFN Padova Cristina Vistoli – INFN Cnaf as INFN members of the EU-GRID technical team.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
GRID workload management system and CMS fall production Massimo Sgaravatto INFN Padova.
Status of Globus activities within INFN Massimo Sgaravatto INFN Padova for the INFN Globus group
Workload Management Workpackage Massimo Sgaravatto INFN Padova.
INFN-GRID Globus evaluation Massimo Sgaravatto INFN Padova for the INFN Globus group
Report on the INFN-GRID Globus evaluation Massimo Sgaravatto INFN Padova for the INFN Globus group
GRID Workload Management System Massimo Sgaravatto INFN Padova.
Globus activities within INFN Massimo Sgaravatto INFN Padova for the INFN Globus group
Workload Management Massimo Sgaravatto INFN Padova.
First steps implementing a High Throughput workload management system Massimo Sgaravatto INFN Padova
Status of Globus activities within INFN (update) Massimo Sgaravatto INFN Padova for the INFN Globus group
First ideas for a Resource Management Architecture for Productions Massimo Sgaravatto INFN Padova.
Evaluation of the Globus GRAM Service Massimo Sgaravatto INFN Padova.
Resource Management Reading: “A Resource Management Architecture for Metacomputing Systems”
Miron Livny Computer Sciences Department University of Wisconsin-Madison Harnessing the Capacity of Computational.
Grid Computing 7700 Fall 2005 Lecture 17: Resource Management Gabrielle Allen
Track 1: Cluster and Grid Computing NBCR Summer Institute Session 2.2: Cluster and Grid Computing: Case studies Condor introduction August 9, 2006 Nadya.
EDG - WP1 (Grid Work Scheduling) Status and plans Massimo Sgaravatto - INFN Padova Francesco Prelz – INFN Milano.
“Grey areas” of the new architecture Massimo Sgaravatto INFN Padova.
INFN-GRID Globus evaluation (WP 1) Massimo Sgaravatto INFN Padova for the INFN Globus group
5 November 2001F Harris GridPP Edinburgh 1 WP8 status for validating Testbed1 and middleware F Harris(LHCb/Oxford)
Workload Management WP Status and next steps Massimo Sgaravatto INFN Padova.
Job Submission Condor, Globus, Java CoG Kit Young Suk Moon.
Grid Computing I CONDOR.
Grid Workload Management & Condor Massimo Sgaravatto INFN Padova.
DataGrid Applications Federico Carminati WP6 WorkShop December 11, 2000.
DataGrid is a project funded by the European Union CHEP 2003 – March 2003 – M. Sgaravatto – n° 1 The EU DataGrid Workload Management System: towards.
DataGrid WP1 Massimo Sgaravatto INFN Padova. WP1 (Grid Workload Management) Objective of the first DataGrid workpackage is (according to the project "Technical.
Grid Workload Management Massimo Sgaravatto INFN Padova.
Condor: High-throughput Computing From Clusters to Grid Computing P. Kacsuk – M. Livny MTA SYTAKI – Univ. of Wisconsin-Madison
Grid checkpointing in the European DataGrid Project Alessio Gianelle – INFN Padova Rosario Peluso – INFN Padova Francesco Prelz – INFN Milano Massimo Sgaravatto.
Report from USA Massimo Sgaravatto INFN Padova. Introduction Workload management system for productions Monte Carlo productions, data reconstructions.
July 11-15, 2005Lecture3: Grid Job Management1 Grid Compute Resources and Job Management.
Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze.
Review of Condor,SGE,LSF,PBS
Proposal for a IS schema Massimo Sgaravatto INFN Padova.
EGEE is a project funded by the European Union under contract INFSO-RI Practical approaches to Grid workload management in the EGEE project Massimo.
6 march Building the INFN Grid Proposal outline a.ghiselli,l.luminari,m.sgaravatto,c.vistoli INFN Grid meeting, milano.
Summary from WP 1 Parallel Section Massimo Sgaravatto INFN Padova.
Report from DataGrid Technical Board Massimo Sgaravatto INFN Padova.
Miron Livny Computer Sciences Department University of Wisconsin-Madison Condor and (the) Grid (one of.
EDG - WP1 (Grid Work Scheduling) Status and plans Massimo Sgaravatto INFN Padova.
JSS Job Submission Service Massimo Sgaravatto INFN Padova.
Status of Globus activities Massimo Sgaravatto INFN Padova for the INFN Globus group
Grid Workload Management (WP 1) Massimo Sgaravatto INFN Padova.
WP1 Status and plans Francesco Prelz, Massimo Sgaravatto 4 th EDG Project Conference Paris, March 6 th, 2002.
Grid Activities in CMS Asad Samar (Caltech) PPDG meeting, Argonne July 13-14, 2000.
First evaluation of the Globus GRAM service Massimo Sgaravatto INFN Padova.
Workload Management Workpackage
First proposal for a modification of the GIS schema
Peter Kacsuk – Sipos Gergely MTA SZTAKI
Wide Area Workload Management Work Package DATAGRID project
Resource and Service Management on the Grid
I Datagrid Workshop- Marseille C.Vistoli
GRID Workload Management System for CMS fall production
Presentation transcript:

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Wide Area Workload Management Work Package DATAGRID project Parallel session report Cristina Vistoli

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Tasks list: Task 1. 2 Job description and resource spec : Responsible DATAmat : Task 1.3 Job decomposition partitioning: Responsible INFN Task 1.4 Scheduling Responsible INFN Task 1.5 Services and access control Responsible Cesnet

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Datagrid WP1 Estimated Resources: 670 mm in 3 years Funded/Unfunded resources INFN 108 Datamat Czech Rep. 18 PPARC

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Datagrid WP1- Effort/Task Breakdown Task INFN 9 PPARC 9 DATAMAT 3 CESnet 3 Task DATAMAT 80 INFN 10 Task INFN 108 CESnet 24 Task INFN 152 Datamat 13 CESnet 18 Task CESnet 75 INFN 57 Task INFN 40 Task INFN 24 PPARC 9 CESnet 24 DATAMAT 12

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Datagrid WP1- Effort/Deliverable Breakdown D1.1 effort 44: task 1.1 (24) + task 1.4 (20) D1.2 effort 15: task 1.4 (15) D1.3 effort 85: task 1.4 (85) D1.4 effort 141: task 1.3 +task 1,7 (9) D1.5 effort 100 : task1.2 +task 1.7 (10) DATAMat D1.6 effort 144: task task 1.7 (12)CESnet D1.7 effort 101 : task 1.4 (63) + task 1.7 (38)

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Contact Points INFN: C.Vistoli, M.Sgaravatto PPARC: A.Martin Datamat: Ugo Grandinetti, S.Beco Cesnet: L.Matiska. M.Ruda WP mailing list:

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Progress Report 2 INFN weeks with Globus Team members Steve Tuecke, Lee Liming + Miron Livny (Condor Team) Ann Cheverniak, Steve Fitzgerald Workload management applied to real HEP application: Event production CMS-HLT production analyzed in terms of GRID requirements and Globus tools availability Simplified description :

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli PDBOBA…CERN IOOOO HLT production simplified description

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Application Grid Requirements The GRID requirements are: Resource Broker to select, find and allocate the computing resources where the jobs have to be submitted. move the input files in the selected farm and submit the jobs collect the output files produced in each farm in a central repository

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Resource Broker Architecture condor_submit Globus submit Master Condor schedd Grid Information Server (LDAP Server) Job code (class-ads) condor-rm log LDAP to Class-ads converter GRAM Job Manager LSF PC Cluster PBS PC Cluster Condor PC Cluster

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Work needed GRAM: to provide all the necessary info to the Grid Information System Collaboration with Globus team on-going: Massimo Sgaravatto presented Need to define “what” is the necessary info in PC cluster resource environment Collaboration with WP£ GRAM/job manager persistent and fault tolerant Massimo Sgaravatto presentation Collaboration with WP3/WP4 Export the information about computing systems availability, disk and network from the LDAP server to the Class-ad collection

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Work needed - cont Master implements job policy and monitor the job execution. A real possibility to verify if match-making algorithm satisfy application requirements. Master access Class-ads collection to select the GRAM resource Input files are moved to the selected resource using ‘gsiftp’ using third part transfer. Input files are for this use-case small, no need for advance network reservation

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Work needed - cont To complete the application execution: Output files have to be collected in the central repository Data Mover and Replica Manager are useful tools. Strict collaboration with WP2 Investigate the use of advance network reservation for these files transfer: GARA

4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli WP interactions Application requirements from WP8.. And WP3 Job Management, Accounting, Bookeeping WP4 Data Access, Replica Management API, access format WP2 Network Reservation : WP7 Security and Access Control WP?