Stephen Burke - WP8 Status - 9/5/2002 Partner Logo WP8 Status Stephen Burke, PPARC/RAL.

Slides:



Advertisements
Similar presentations
DataTAG WP4 Meeting CNAF Jan 14, 2003 Interfacing AliEn and EDG 1/13 Stefano Bagnasco, INFN Torino Interfacing AliEn to EDG Stefano Bagnasco, INFN Torino.
Advertisements

EU DataGrid progress Fabrizio Gagliardi EDG Project Leader
Andrew McNab - Manchester HEP - 15 February 2002 Testbed Release in the UK EDG Testbed 1 GridPP sources of information GridPP VO GIIS and Resource Broker.
30-31 Jan 2003J G Jensen, RAL/WP5 Storage Elephant Grid Access to Mass Storage.
DataGrid is a project funded by the European Union CHEP 2003 – March 2003 – HEP Assessment of EDG – n° 1 HEP Applications Evaluation of the EDG Testbed.
Stephen Burke - WP8 Status - 14/2/2002 Partner Logo WP8 Status Stephen Burke, PPARC/RAL.
ATLAS/LHCb GANGA DEVELOPMENT Introduction Requirements Architecture and design Interfacing to the Grid Ganga prototyping A. Soroko (Oxford), K. Harrison.
Partner Logo UK GridPP Testbed Rollout John Gordon GridPP 3rd Collaboration Meeting Cambridge 15th February 2002.
Andrew McNab - Manchester HEP - 22 April 2002 EU DataGrid Testbed EU DataGrid Software releases Testbed 1 Job Lifecycle Authorisation at your site More.
Andrew McNab - Manchester HEP - 2 May 2002 Testbed and Authorisation EU DataGrid Testbed 1 Job Lifecycle Software releases Authorisation at your site Grid/Web.
Andrew McNab - Manchester HEP - 22 April 2002 EU DataGrid Testbed EU DataGrid Software releases Testbed 1 Job Lifecycle Authorisation at your site More.
1 CHEP 2000, Roberto Barbera Recent grid activities at INFN Catania (*) HEPiX/HEPNT 2002, Catania, (*) work in collaboration with.
EU 2nd Year Review – Jan – Title – n° 1 WP1 Speaker name (Speaker function and WP ) Presentation address e.g.
The DataGrid Project NIKHEF, Wetenschappelijke Jaarvergadering, 19 December 2002
DataGrid Kimmo Soikkeli Ilkka Sormunen. What is DataGrid? DataGrid is a project that aims to enable access to geographically distributed computing power.
Magda – Manager for grid-based data Wensheng Deng Physics Applications Software group Brookhaven National Laboratory.
Bookkeeping data Monitoring info Get jobs Site A Site B Site C Site D Agent Production service Monitoring service Bookkeeping service Agent © Andrei Tsaregorodtsev.
GridPP9 – 5 February 2004 – Data Management DataGrid is a project funded by the European Union GridPP is funded by PPARC WP2+5: Data and Storage Management.
EGEE is a project funded by the European Union under contract IST JRA1 Testing Activity: Status and Plans Leanne Guy EGEE Middleware Testing.
1 DataGRID Application Status and plans
5 November 2001F Harris GridPP Edinburgh 1 WP8 status for validating Testbed1 and middleware F Harris(LHCb/Oxford)
Don Quijote Data Management for the ATLAS Automatic Production System Miguel Branco – CERN ATC
WP9 – Earth Observation Applications – n° 1 Experiences with Testbed1, plans and objectives for Testbed 2 Testbed retreat th August 2002
3 Sept 2001F HARRIS CHEP, Beijing 1 Moving the LHCb Monte Carlo production system to the GRID D.Galli,U.Marconi,V.Vagnoni INFN Bologna N Brook Bristol.
EGEE is a project funded by the European Union under contract IST Testing processes Leanne Guy Testing activity manager JRA1 All hands meeting,
SLICE Simulation for LHCb and Integrated Control Environment Gennady Kuznetsov & Glenn Patrick (RAL) Cosener’s House Workshop 23 rd May 2002.
QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.
Grid Workload Management & Condor Massimo Sgaravatto INFN Padova.
Cosener’s House – 30 th Jan’031 LHCb Progress & Plans Nick Brook University of Bristol News & User Plans Technical Progress Review of deliverables.
- Iain Bertram R-GMA and DØ Iain Bertram RAL 13 May 2004 Thanks to Jeff Templon at Nikhef.
Grid Status - PPDG / Magda / pacman Torre Wenaus BNL U.S. ATLAS Physics and Computing Advisory Panel Review Argonne National Laboratory Oct 30, 2001.
DataGrid is a project funded by the European Union VisualJob Demonstation EDG 1.4.x 2003 The EU DataGrid How the use of distributed resources can help.
LHCb and DataGRID - the workplan for 2001 Eric van Herwijnen Wednesday, 28 march 2001.
LCG Middleware Testing in 2005 and Future Plans E.Slabospitskaya, IHEP, Russia CERN-Russia Joint Working Group on LHC Computing March, 6, 2006.
WP8 Status – Stephen Burke – 30th January 2003 WP8 Status Stephen Burke (RAL) (with thanks to Frank Harris)
Grid Workload Management Massimo Sgaravatto INFN Padova.
5 Sep 2002F Harris Plenary Budapest1 WP8 Report F Harris (Oxford/CERN)
CMS Stress Test Report Marco Verlato (INFN-Padova) INFN-GRID Testbed Meeting 17 Gennaio 2003.
7April 2000F Harris LHCb Software Workshop 1 LHCb planning on EU GRID activities (for discussion) F Harris.
Δ Storage Middleware GridPP10 What’s new since GridPP9? CERN, June 2004.
Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 The Storage Element.
Common Use Cases for a HEP Common Architecture Layer J. Templon, NIKHEF/WP8.
CLRC and the European DataGrid Middleware Information and Monitoring Services The current information service is built on the hierarchical database OpenLDAP.
LCG EGEE is a project funded by the European Union under contract IST LCG PEB, 7 th June 2004 Prototype Middleware Status Update Frédéric Hemmer.
Owen SyngeTitle of TalkSlide 1 Storage Management Owen Synge – Developer, Packager, and first line support to System Administrators. Talks Scope –GridPP.
Stephen Burke – Data Management - 3/9/02 Partner Logo Data Management Stephen Burke, PPARC/RAL Jeff Templon, NIKHEF.
WP3 Information and Monitoring Rob Byrom / WP3
K. Harrison CERN, 3rd March 2004 GANGA CONTRIBUTIONS TO ADA RELEASE IN MAY - Outline of Ganga project - Python support for AJDL - LCG analysis service.
Data Transfer Service Challenge Infrastructure Ian Bird GDB 12 th January 2005.
David Foster LCG Project 12-March-02 Fabric Automation The Challenge of LHC Scale Fabrics LHC Computing Grid Workshop David Foster 12 th March 2002.
ESA DataGrid Review – 10 June 2002 – n° 1 Summary item 3  ESA and WP9 part 1 (45m)  DataGrid Frascati infrastructure (AT, 7.5m)  ESA GOME application.
Author - Title- Date - n° 1 Partner Logo WP5 Status John Gordon Budapest September 2002.
Grid Workload Management (WP 1) Massimo Sgaravatto INFN Padova.
Release 1.2 Status Ingo Augustin CERN. July 3, 2002I. Augustin, WP8, CERN2 Recent main activities Intensive debugging effort of WP1 and Loose Cannons.
The DataGrid Project NIKHEF, Wetenschappelijke Jaarvergadering, 19 December 2002
Grid Status - PPDG / Magda / pacman Torre Wenaus BNL DOE/NSF Review of US LHC Software and Computing Fermilab Nov 29, 2001.
Enabling Grids for E-sciencE CMS/ARDA activity within the CMS distributed system Julia Andreeva, CERN On behalf of ARDA group CHEP06.
Distributed Analysis Tutorial Dietrich Liko. Overview  Three grid flavors in ATLAS EGEE OSG Nordugrid  Distributed Analysis Activities GANGA/LCG PANDA/OSG.
Stephen Burke – Sysman meeting - 22/4/2002 Partner Logo The Testbed – A User View Stephen Burke, PPARC/RAL.
Testbed Release Criteria and Plans WP8 Loose Cannons & App Representatives.
CERN Certification & Testing LCG Certification & Testing Team (C&T Team) Marco Serra - CERN / INFN Zdenek Sekera - CERN.
EGEE is a project funded by the European Union under contract IST Issues from current Experience SA1 Feedback to JRA1 A. Pacheco PIC Barcelona.
Acronyms GAS - Grid Acronym Soup, LCG - LHC Computing Project EGEE - Enabling Grids for E-sciencE.
BaBar-Grid Status and Prospects
The EDG Testbed Deployment Details
Work Package 9 – EO Applications
Moving the LHCb Monte Carlo production system to the GRID
Leanne Guy EGEE JRA1 Test Team Manager
Stephen Burke, PPARC/RAL Jeff Templon, NIKHEF
Gridifying the LHCb Monte Carlo production system
Presentation transcript:

Stephen Burke - WP8 Status - 9/5/2002 Partner Logo WP8 Status Stephen Burke, PPARC/RAL

Stephen Burke - WP8 Status - 09/05/ /11 What is WP8? u HEP applications workpackage u Link between experiment software and EDG middleware u Works with WP9 (Earth Observation) and WP10 (Biomedical) u Applications Technical Working Group (ATWG) is a forum to define application requirements for the EDG middleware: Loose Cannons + experiment reps + WP9 + WP10

Stephen Burke - WP8 Status - 09/05/ /11 Testing the Testbed u LCs have continued to be active in testing the functionality of the testbed u Automated test software under development, with publication of the results on the web u Ongoing feedback to middleware developers u Some application tests, but still limited because the job broker at CERN is unstable u Stability is our highest priority!

Stephen Burke - WP8 Status - 09/05/ /11 Current Problems u Job submission works, but has been unstable, with jobs failing for various reasons much of the time u It isnt yet fully defined how to set up an application environment (where to find compilers, experiment software etc) at each site, but this is improving u There are work-arounds for the initial problems with data storage, but the solutions are awkward and insecure u Documentation is improving but still not perfect!

Stephen Burke - WP8 Status - 09/05/ /11 Near Future u Release is still not stable, development has now stopped and applications will wait for 1.2 u Move to 1.2 in ~ 2 weeks subject to testing u Should have new functionality, particularly from WP2 (Replica Manager, new GDMP) which should make data access easier u Should also resolve problems with access to Castor u Hope to have a stable production testbed soon u Hope to see a rollout to more sites soon

Stephen Burke - WP8 Status - 09/05/ /11 Common Use Cases u ATWG is trying to define common needs of all applications, with the long-term aim of having a common software layer between the experiment software and the Grid middleware u Discussions held with the four LHC experiments to clarify their approaches and extract common themes u Various kinds of information: use cases, implementation/architecture ideas, requirements, scenarios, …

Stephen Burke - WP8 Status - 09/05/ /11 Common Use Cases (2) u Preliminary Common Use Case document produced u Some overlap with LCG, RTAG will produce an updated use case document, work will continue in WP8 u Working with computer scientists at Annecy to produce UML diagrams

Stephen Burke - WP8 Status - 09/05/ /11 UML Diagram

Stephen Burke - WP8 Status - 09/05/ /11 Administrivia u WP8 plenary on 14 th June at CERN (3 per year) u Application Technical Working Group (ATWG) has been meeting once a month, in future will alternate WP8-only and WP meetings to separate HEP- specific issues from general ones u Architecture Task Force has been re-activated, Jeff Templon (NIKHEF) is the WP8 representative

Stephen Burke - WP8 Status - 09/05/ /11 WP8 Technical Requirements u 1. Realistic Large-Scale Tests n Reliability! Need reliable dg- job-* command suite u 2. Data management n Reliability! Need reliable gdmp-* command suite, file- transfer commands u 3. Mass Storage Support n Working access to MSS (CASTOR and HPSS at CERN, Lyon) u 4. Lightweight User Interface n Put on a laptop or std. Desktop machine u 5. Portability n Demonstrable portability of middleware: a) use other resources, b) debugging u 6. Scratch Space n Job requests X amount of scratch space to be available during execution, system tells job where it is u 7. Output File Support n JDL support for output files: specify where output should go in JDL, not in job script