Www.gisela-grid.eu Grid Initiatives for e-Science virtual communities in Europe and Latin America DIRAC TEAM CPPM – CNRS DIRAC Grid Middleware.

Slides:

Advertisements

Similar presentations

Grid Initiatives for e-Science virtual communities in Europe and Latin America WP6: Infrastructure & Application-oriented Services for.

Advertisements

Computing Lectures Introduction to Ganga 1 Ganga: Introduction Object Orientated Interactive Job Submission System –Written in python –Based on the concept.

1 Grid services based architectures Growing consensus that Grid services is the right concept for building the computing grids; Recent ARDA work has provoked.

6/4/20151 Introduction LHCb experiment. LHCb experiment. Common schema of the LHCb computing organisation. Common schema of the LHCb computing organisation.

Workload Management Massimo Sgaravatto INFN Padova.

1 Bridging Clouds with CernVM: ATLAS/PanDA example Wenjing Wu

Stuart K. PatersonCHEP 2006 (13 th –17 th February 2006) Mumbai, India 1 from DIRAC.Client.Dirac import * dirac = Dirac() job = Job() job.setApplication('DaVinci',

IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.

DIRAC API DIRAC Project. Overview  DIRAC API  Why APIs are important?  Why advanced users prefer APIs?  How it is done?  What is local mode what.

Analysis demos from the experiments. Analysis demo session Introduction –General information and overview CMS demo (CRAB) –Georgia Karapostoli (Athens.

Pilots 2.0: DIRAC pilots for all the skies Federico Stagni, A.McNab, C.Luzzi, A.Tsaregorodtsev On behalf of the DIRAC consortium and the LHCb collaboration.

QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.

1 1 Hybrid Cloud Solutions (Private with Public Burst) Accelerate and Orchestrate Enterprise Applications.

Customized cloud platform for computing on your terms !

CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.

KARMA with ProActive Parallel Suite 12/01/2009 Air France, Sophia Antipolis Solutions and Services for Accelerating your Applications.

DIRAC Web User Interface A.Casajus (Universitat de Barcelona) M.Sapunov (CPPM Marseille) On behalf of the LHCb DIRAC Team.

SICSA student induction day, 2009Slide 1 Social Simulation Tutorial Session 6: Introduction to grids and cloud computing International Symposium on Grid.

BESIII distributed computing and VMDIRAC

IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.

Nicholas LoulloudesMarch 3 rd, 2009 g-Eclipse Testing and Benchmarking Grid Infrastructures using the g-Eclipse Framework Nicholas Loulloudes On behalf.

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America GENIUS server installation and configuration.

INFSO-RI Enabling Grids for E-sciencE The GENIUS Grid portal Tony Calanducci INFN Catania - Italy First Latin American Workshop.

YAN, Tian On behalf of distributed computing group Institute of High Energy Physics (IHEP), CAS, China CHEP-2015, Apr th, OIST, Okinawa.

INFSO-RI Module 01 ETICS Overview Alberto Di Meglio.

Grid Basics Adarsh Patil

1 DIRAC – LHCb MC production system A.Tsaregorodtsev, CPPM, Marseille For the LHCb Data Management team CHEP, La Jolla 25 March 2003.

INFSO-RI Module 01 ETICS Overview Etics Online Tutorial Marian ŻUREK Baltic Grid II Summer School Vilnius, 2-3 July 2009.

BESIII Production with Distributed Computing Xiaomei Zhang, Tian Yan, Xianghu Zhao Institute of High Energy Physics, Chinese Academy of Sciences, Beijing.

EGEE-III INFSO-RI Enabling Grids for E-sciencE Application Porting Support in EGEE Gergely Sipos MTA SZTAKI EGEE’08.

Getting started DIRAC Project. Outline  DIRAC information system  Documentation sources  DIRAC users and groups  Registration with DIRAC  Getting.

LHCb-ATLAS GANGA Workshop, 21 April 2004, CERN 1 DIRAC Software distribution A.Tsaregorodtsev, CPPM, Marseille LHCb-ATLAS GANGA Workshop, 21 April 2004.

INFSO-RI Enabling Grids for E-sciencE Experience of using gLite for analysis of ATLAS combined test beam data A. Zalite / PNPI.

CHEP 2006, February 2006, Mumbai 1 LHCb use of batch systems A.Tsaregorodtsev, CPPM, Marseille HEPiX 2006, 4 April 2006, Rome.

EGEE-III INFSO-RI Enabling Grids for E-sciencE Ricardo Rocha CERN (IT/GS) EGEE’08, September 2008, Istanbul, TURKEY Experiment.

INFSO-RI Enabling Grids for E-sciencE The gLite File Transfer Service: Middleware Lessons Learned form Service Challenges Paolo.

INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.

Development of e-Science Application Portal on GAP WeiLong Ueng Academia Sinica Grid Computing

WebFlow High-Level Programming Environment and Visual Authoring Toolkit for HPDC (desktop access to remote resources) Tomasz Haupt Northeast Parallel Architectures.

The GridPP DIRAC project DIRAC for non-LHC communities.

GRID Security & DIRAC A. Casajus R. Graciani A. Tsaregorodtsev.

ANALYSIS TOOLS FOR THE LHC EXPERIMENTS Dietrich Liko / CERN IT.

DIRAC Pilot Jobs A. Casajus, R. Graciani, A. Tsaregorodtsev for the LHCb DIRAC team Pilot Framework and the DIRAC WMS DIRAC Workload Management System.

OSG Area Coordinator’s Report: Workload Management Maxim Potekhin BNL May 8 th, 2008.

VLDATA Common solution for the (very-)large data challenge EINFRA-1, focus on topics (4) & (5)

DIRAC Project A.Tsaregorodtsev (CPPM) on behalf of the LHCb DIRAC team A Community Grid Solution The DIRAC (Distributed Infrastructure with Remote Agent.

EGI Technical Forum Amsterdam, 16 September 2010 Sylvain Reynaud.

The ATLAS Strategy for Distributed Analysis on several Grid Infrastructures D. Liko, IT/PSS for the ATLAS Distributed Analysis Community.

Automatic testing and certification procedure for IGI products in the EMI era and beyond Sara Bertocco INFN Padova on behalf of IGI Release Team EGI Community.

1 DIRAC agents A.Tsaregorodtsev, CPPM, Marseille ARDA Workshop, 7 March 2005, CERN.

+ Support multiple virtual environment for Grid computing Dr. Lizhe Wang.

CHEP 2006, February 2006, Mumbai 1 DIRAC, the LHCb Data Production and Distributed Analysis system A.Tsaregorodtsev, CPPM, Marseille CHEP 2006,

The GridPP DIRAC project DIRAC for non-LHC communities.

Breaking the frontiers of the Grid R. Graciani EGI TF 2012.

DIRAC for Grid and Cloud Dr. Víctor Méndez Muñoz (for DIRAC Project) LHCb Tier 1 Liaison at PIC EGI User Community Board, October 31st, 2013.

DIRAC Distributed Computing Services A. Tsaregorodtsev, CPPM-IN2P3-CNRS FCPPL Meeting, 29 March 2013, Nanjing.

1 DIRAC Project Status A.Tsaregorodtsev, CPPM-IN2P3-CNRS, Marseille 10 March, DIRAC Developer meeting.

IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.

The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.

1 Building application portals with DIRAC A.Tsaregorodtsev, CPPM-IN2P3-CNRS, Marseille 27 April 2010, Journée LuminyGrid, Marseille.

Multi-community e-Science service connecting grids & clouds R. Graciani 1, V. Méndez 2, T. Fifield 3, A. Tsaregordtsev 4 1 University of Barcelona 2 University.

Workload Management Workpackage

Volunteer Computing for Science Gateways

AWS Integration in Distributed Computing

Blueprint of Persistent Infrastructure as a Service

DIRAC services.

Tools and Services Workshop Overview of Atmosphere

WLCG Collaboration Workshop;

VMDIRAC status Vanessa HAMAR CC-IN2P3.

Status of Grids for HEP and HENP

Presentation transcript:

Grid Initiatives for e-Science virtual communities in Europe and Latin America DIRAC TEAM CPPM – CNRS DIRAC Grid Middleware

DIRAC DIRAC project was initially developed for the LHCb Collaboration as a system for massive production of the simulation data in order to design and optimize the LHCb experimental setup –DIRAC integrates computing resources available at LHCb production sites as well as on the LCG grid –The principal idea is to group heterogeneous computing resources for various activities with minimal human intervention 2

DIRAC Overview Now, DIRAC is a general-purpose Grid Middleware providing access to different kinds of distributed computing resources. It includes: –General Framework for building secure distributed computing systems –Powerful Workload Management System based on Pilot Jobs –Data Management System with native File Catalog providing access to standard Grid storage resources –Higher level services to automate massive payload creation and execution –Friendly Web and command line user interfaces 3

DIRAC overview: key features Fault tolerance –Multiple levels of redundancy built in Modular extendable structure –Allow to easily build custom service necessary for specific applications Example extensions: –MPI for EELA –Amazon EC2 Cloud VM Manager for Belle Experiment 4

DIRAC overview: basics WMS is based on the “Pull” paradigm coupled with the idea of the Pilot Jobs –Execution environment is checked and guaranteed –Easiness of incorporated various computing resources Central TaskQueue for users and production jobs –Apply VO policies in a single place –Different kind of schedulers can be used 5

DIRAC overview: interfaces DIRAC GUI is web based interface build in JS Multiplatform and secured Mimicking the desktop application Short learning curve for users 6 DIRAC CLI is based on DIRAC API written in Python DIRAC API is used in external projects like Ganga

DIRAC overview: performance DIRAC3 performance measured in the recent production and FEST’09 runs –Up to 25K concurrent jobs in 150 distinct sites 7

Contact us Andrei Tsaregorodtsev Vanessa Hamar 8