6 march 20001 Building the INFN Grid Proposal outline a.ghiselli,l.luminari,m.sgaravatto,c.vistoli INFN Grid meeting, milano.

Slides:



Advertisements
Similar presentations
WP1 Grid Workload Management Massimo Sgaravatto INFN Padova
Advertisements

Installation and evaluation of the Globus toolkit WP 1 INFN-GRID Workload management WP 1 DATAGRID WP 2.1 INFN-GRID Massimo Sgaravatto INFN Padova.
INFN & Globus activities Massimo Sgaravatto INFN Padova.
Work Package 1 Installation and Evaluation of the Globus Toolkit Massimo Sgaravatto INFN Padova.
Database Architectures and the Web
Grid Resource Allocation Management (GRAM) GRAM provides the user to access the grid in order to run, terminate and monitor jobs remotely. The job request.
Evaluation of the Globus Toolkit: Status Roberto Cucchi – INFN Cnaf Antonia Ghiselli – INFN Cnaf Giuseppe Lo Biondo – INFN Milano Francesco Prelz – INFN.
4/2/2002HEP Globus Testing Request - Jae Yu x Participating in Globus Test-bed Activity for DØGrid UTA HEP group is playing a leading role in establishing.
CMS HLT production using Grid tools Flavia Donno (INFN Pisa) Claudio Grandi (INFN Bologna) Ivano Lippi (INFN Padova) Francesco Prelz (INFN Milano) Andrea.
DataGrid is a project funded by the European Union 22 September 2003 – n° 1 EDG WP4 Fabric Management: Fabric Monitoring and Fault Tolerance
EU-GRID Work Program Massimo Sgaravatto – INFN Padova Cristina Vistoli – INFN Cnaf as INFN members of the EU-GRID technical team.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
USING THE GLOBUS TOOLKIT This summary by: Asad Samar / CALTECH/CMS Ben Segal / CERN-IT FULL INFO AT:
GRID workload management system and CMS fall production Massimo Sgaravatto INFN Padova.
Chapter 1: Introduction
Workload Management Workpackage Massimo Sgaravatto INFN Padova.
GRID Workload Management System Massimo Sgaravatto INFN Padova.
Globus activities within INFN Massimo Sgaravatto INFN Padova for the INFN Globus group
Workload Management Massimo Sgaravatto INFN Padova.
Status of Globus activities within INFN (update) Massimo Sgaravatto INFN Padova for the INFN Globus group
1/16/2008CSCI 315 Operating Systems Design1 Introduction Notice: The slides for this lecture have been largely based on those accompanying the textbook.
Ian Fisk and Maria Girone Improvements in the CMS Computing System from Run2 CHEP 2015 Ian Fisk and Maria Girone For CMS Collaboration.
Database Architectures and the Web Session 5
INFN-GRID Globus evaluation (WP 1) Massimo Sgaravatto INFN Padova for the INFN Globus group
Workload Management WP Status and next steps Massimo Sgaravatto INFN Padova.
DISTRIBUTED COMPUTING
ARGONNE  CHICAGO Ian Foster Discussion Points l Maintaining the right balance between research and development l Maintaining focus vs. accepting broader.
CoG Kit Overview Gregor von Laszewski Keith Jackson.
DATAGRID ConferenceTestbed0 - resources in Italy Luciano Gaido 1 DATAGRID WP6 Testbed0 resources in Italy Amsterdam March,
◦ What is an Operating System? What is an Operating System? ◦ Operating System Objectives Operating System Objectives ◦ Services Provided by the Operating.
1 School of Computer, National University of Defense Technology A Profile on the Grid Data Engine (GridDaEn) Xiao Nong
Grid Workload Management & Condor Massimo Sgaravatto INFN Padova.
DataGrid Applications Federico Carminati WP6 WorkShop December 11, 2000.
11 December 2000 Paolo Capiluppi - DataGrid Testbed Workshop CMS Applications Requirements DataGrid Testbed Workshop Milano, 11 December 2000 Paolo Capiluppi,
DOSAR Workshop, Sao Paulo, Brazil, September 16-17, 2005 LCG Tier 2 and DOSAR Pat Skubic OU.
Grid Workload Management Massimo Sgaravatto INFN Padova.
ATLAS and GridPP GridPP Collaboration Meeting, Edinburgh, 5 th November 2001 RWL Jones, Lancaster University.
Instrumentation of the SAM-Grid Gabriele Garzoglio CSC 426 Research Proposal.
December 10,1999: MONARC Plenary Meeting Harvey Newman (CIT) Phase 3 Letter of Intent (1/2)  Short: N Pages è May Refer to MONARC Internal Notes to Document.
NOVA Networked Object-based EnVironment for Analysis P. Nevski, A. Vaniachine, T. Wenaus NOVA is a project to develop distributed object oriented physics.
Virtual Batch Queues A Service Oriented View of “The Fabric” Rich Baker Brookhaven National Laboratory April 4, 2002.
GRID ARCHITECTURE Chintan O.Patel. CS 551 Fall 2002 Workshop 1 Software Architectures 2 What is Grid ? "...a flexible, secure, coordinated resource- sharing.
CLRC and the European DataGrid Middleware Information and Monitoring Services The current information service is built on the hierarchical database OpenLDAP.
What is SAM-Grid? Job Handling Data Handling Monitoring and Information.
Globus Toolkit Massimo Sgaravatto INFN Padova. Massimo Sgaravatto Introduction Grid Services: LHC regional centres need distributed computing Analyze.
THE INFN GRID PROJECT zScope: Study and develop a general INFN computing infrastructure, based on GRID technologies, to be validated (as first use case)
Introduction to Grids By: Fetahi Z. Wuhib [CSD2004-Team19]
6/23/2005 R. GARDNER OSG Baseline Services 1 OSG Baseline Services In my talk I’d like to discuss two questions:  What capabilities are we aiming for.
29/1/2002A.Ghiselli, INFN-CNAF1 DataTAG / WP4 meeting Cern, 29 January 2002 Agenda  start at  Project introduction, Olivier Martin  WP4 introduction,
Globus and PlanetLab Resource Management Solutions Compared M. Ripeanu, M. Bowman, J. Chase, I. Foster, M. Milenkovic Presented by Dionysis Logothetis.
Networking: Applications and Services Antonia Ghiselli, INFN Stu Loken, LBNL Chairs.
Condor on WAN D. Bortolotti - INFN Bologna T. Ferrari - INFN Cnaf A.Ghiselli - INFN Cnaf P.Mazzanti - INFN Bologna F. Prelz - INFN Milano F.Semeria - INFN.
Tier3 monitoring. Initial issues. Danila Oleynik. Artem Petrosyan. JINR.
GRID ANATOMY Advanced Computing Concepts – Dr. Emmanuel Pilli.
Summary from WP 1 Parallel Section Massimo Sgaravatto INFN Padova.
Status of Globus activities Massimo Sgaravatto INFN Padova for the INFN Globus group
Grid Workload Management (WP 1) Massimo Sgaravatto INFN Padova.
1 A Scalable Distributed Data Management System for ATLAS David Cameron CERN CHEP 2006 Mumbai, India.
January 20, 2000K. Sliwa/ Tufts University DOE/NSF ATLAS Review 1 SIMULATION OF DAILY ACTIVITITIES AT REGIONAL CENTERS MONARC Collaboration Alexander Nazarenko.
Grid Activities in CMS Asad Samar (Caltech) PPDG meeting, Argonne July 13-14, 2000.
Architecture for Resource Allocation Services Supporting Interactive Remote Desktop Sessions in Utility Grids Vanish Talwar, HP Labs Bikash Agarwalla,
G. Russo, D. Del Prete, S. Pardi Kick Off Meeting - Isola d'Elba, 2011 May 29th–June 01th A proposal for distributed computing monitoring for SuperB G.
10-Feb-00 CERN HepCCC Grid Initiative ATLAS meeting – 16 February 2000 Les Robertson CERN/IT.
Chapter 1: Introduction What is an Operating System? Mainframe Systems Desktop Systems Multiprocessor Systems Distributed Systems Clustered System Real.
Towards a High Performance Extensible Grid Architecture Klaus Krauter Muthucumaru Maheswaran {krauter,
Workload Management Workpackage
Report from WLCG Workshop 2017: WLCG Network Requirements GDB - CERN 12th of July 2017
Database Architectures and the Web
Wide Area Workload Management Work Package DATAGRID project
I Datagrid Workshop- Marseille C.Vistoli
Presentation transcript:

6 march Building the INFN Grid Proposal outline a.ghiselli,l.luminari,m.sgaravatto,c.vistoli INFN Grid meeting, milano

6 march Contributions from: -Monarc test-bed -Globus tests -Grid workshop and tutorials (Padova CHEP-2000)

6 march What’s a Grid? GRID is a set of services for -Obtaining information about grid components -Locating and scheduling resources -Communicating -Accessing code and data -Measuring performance -Authenticating users and resources -Ensuring the privacy of communication …. -From “The GRID” edited by I.Foster and C.Kesselman

6 march HEP computing Key elements: -Data type and size per LHC exp.: RAW(1MB per event, B/year/exp.), ESD(100KB per event), AOD(10KB per event) -Data processing: event reconstruction, calibration, data reduction, event selection,analysis, event simulation -Distributed, hierarchical regional centers architecture: Tier 0 (CERN), Tier 1, …desktop

6 march Data model: data organized in objects stored in distributed object data base -Desirable ODBMS features: -Object stored and managed with C++ -Client/Server architecture -Data distributed in several data servers -Fault tolerance and data replication and transparent access to the nearest copy…..

6 march Approaching computing and data Grid Three main working-areas -Identify GRID requirements -Application programming technologies -Grid layout model

6 march Identify GRID requirements infn-grid WG identified 4 main topics: 1)Wide area Workload management 2)Wide area data management 3)Wide area application monitoring 4)Computing fabric and general utilities for a global managed grid They are included in the present EU-GRID WorkPackages

6 march HEP Application programming model Commercial technologies:Java, CORBA… -Need interfaces to GRID services Grid oriented programming: can take full advantages of the grid services User oriented technologies …… Application programming technologies

6 march GRID Layout Model Data Server CPU Client Layout evolution: Multi-level server hierarchy desktop CERN WAN Condor pool CPU Client

6 march Modeling the grid Starting from the applications: 1)Traditional appl. running on single computer and with local I/O 2)Client/Server appl. running on multiple computers and with local or remote I/O 3)Grid oriented appl. integrated with grid services

6 march Grid for traditional applications -Use case: Event simulation -Condor based grid: possible I/O bottleneck, no CPU load balance -Globus based grid: Design and implement High Throughput Scheduler How to handle Appl. I/O: -Modifying the appl., or with DATA mover -Globus/Condor based grid: job submission from Globus client to Condor pool and vice versa. Personal Condor configuration could be an interesting HTB for Globus.

6 march workplan -Use case : CMS HLT ph.1, ATLAS trigger studies -Deliverable: HTB, DATA Mover -Test bed: Globus pool, Condor pool, PC machines -Schedule: 6-8 months (rough estimate) -FTE: 5 (rough estimate)

6 march Grid for client/server applications -use case: analysis application with Objectivity 5.2 in multiserver configuration (later on Objectivity can be substituted by Espresso or other…) -Issues to investigate: -How clients and servers can interoperate with the grid -Advance co-reservation and co-scheduling -Monitoring and real-time workload management

6 march Cnaf PD Mi server Clients 10Mbps Wan link Pisa Genova Clients 10Mbps Wan link Clients 10Mbps Wan link Testbed Layout Garr-b TEN-155 Bologna RM server Other servers or clients 10Mbps Wan link CERN Bari

6 march Workplan -Use case : analysis applications -Deliverable: -Appl. Programming interface to grid services -Monitoring and real-time workload management -Advance co-reservation and co-scheduling -Test bed: dedicated grid layout with 10Mb as minimum bandwidth -Schedule: 2 years -FTE: 9

6 march Interactive networked applications -Interactive phisics analysis -Remote monitoring and control -Requirements: High performance rather than high throughput -Workplan: tbd

6 march Grid oriented applications They are implemented in terms of various application toolkits component, grid services and grid fabric mechanisms -tbd

6 march Summary schema applicationsBatch traditional appl. Client/Server batch applications Client/Server Interactive applications Grid enabled applications Application oriented Grid Services/ Grid application toolkits HTB (High Th. Broker) Data mover Monitoring tools Real-time workload management HPerformance Scheduler HPerformance data access Remote data access, remote computing, remote instrumentation, Application monitoring Grid Common Services (Resource and appl. independent services) Common Informatio n Base (MDS), Authentication, authorization, policy (GSI) Global Resource Monitoring GRAM (Globus Resource Allocation Manager) Grid Fabric/local Resources Computing resources local schedulers, site accounting ….. Network services Data storageInstrumentation

6 march Conclusions 1.Grid Fabric: Definition of CPU resources, Network services, Network layout 2.Grid services: Decide the Grid Common Services to adopt. MDS, GSI, GRAM…… 3.Application oriented Grid Services: Identification of the most important Services satisfying the application use cases. 4.Application development using the GRID. 5.Project management organization: Project organization: top – down approach based on application use cases. Coordination in order to identify common services to be shared between the applications. GRID Architecture needs GRID Organization Coordination/integration with the European Grid Project