Storage / Data TEG Introduction

Slides:



Advertisements
Similar presentations
Data Management TEG Status Dirk Duellmann & Brian Bockelman WLCG GDB, 9. Nov 2011.
Advertisements

Data & Storage Management TEGs Summary of recommendations Wahid Bhimji, Brian Bockelman, Daniele Bonacorsi, Dirk Duellmann GDB, CERN 18 th April 2012.
Discussion – Development of 2012 NGAC Guidance Ivan DeLoatch NGAC Meeting January 12, 2012.
Detecting Network Attachment IETF61 Chairs: Pekka Nikander, Greg Daley.
INFO 355Week #61 Systems Analysis II Essentials of design INFO 355 Glenn Booker.
Filesytems and file access Wahid Bhimji University of Edinburgh, Sam Skipsey, Chris Walker …. Apr-101Wahid Bhimji – Files access.
CERN IT Department CH-1211 Genève 23 Switzerland t Plans and Architectural Options for Physics Data Analysis at CERN D. Duellmann, A. Pace.
Storage TEG “emerging” observations and recommendations Wahid Bhimji With contributions from the SM editors (listed in intro)
LHCb input to DM and SM TEGs. Remarks to DM and SM TEGS Introduction m We have already provided some input during our dedicated session of the TEG m Here.
PhysX CoE: LHC Data-intensive workflows and data- management Wahid Bhimji, Pete Clarke, Andrew Washbrook – Edinburgh And other CoE WP4 people…
Workshop summary Ian Bird, CERN WLCG Workshop; DESY, 13 th July 2011 Accelerating Science and Innovation Accelerating Science and Innovation.
Your university or experiment logo here Storage and Data Management - Background Jens Jensen, STFC.
CERN IT Department CH-1211 Geneva 23 Switzerland GT WG on Storage Federations First introduction Fabrizio Furano
Software Engineering COSC 4460 Class 4 Cherry Owen.
OSG Technology Area Brian Bockelman Area Coordinator’s Meeting February 15, 2012.
Evolution of storage and data management Ian Bird GDB: 12 th May 2010.
Storage Interfaces Introduction Wahid Bhimji University of Edinburgh Based on previous discussions with Working Group: (Brian Bockelman, Simone Campana,
WLCG Technical Evolution Group: Operations and Tools Maria Girone & Jeff Templon Kick-off meeting, 24 th October 2011.
Data Placement Intro Dirk Duellmann WLCG TEG Workshop Amsterdam 24. Jan 2012.
Data management demonstrators Ian Bird; WLCG MB 18 th January 2011.
Storage Interfaces and Access pre-GDB Wahid Bhimji University of Edinburgh On behalf of all those who participated.
Storage Management Technical Evolution Group Wahid Bhimji Daniele Bonacorsi.
February 2014 Andrew Tyson Sue Bottomley Personal budgets and the SEND Reforms.
Handling of T1D0 in CCRC’08 Tier-0 data handling Tier-1 data handling Experiment data handling Reprocessing Recalling files from tape Tier-0 data handling,
SRM v2.2 Production Deployment SRM v2.2 production deployment at CERN now underway. – One ‘endpoint’ per LHC experiment, plus a public one (as for CASTOR2).
Testing Infrastructure Wahid Bhimji Sam Skipsey Intro: what to test Existing testing frameworks A proposal.
CMS: T1 Disk/Tape separation Nicolò Magini, CERN IT/SDC Oliver Gutsche, FNAL November 11 th 2013.
Storage Interfaces and Access: Interim report Wahid Bhimji University of Edinburgh On behalf of WG: Brian Bockelman, Philippe Charpentier, Simone Campana,
Budget Information Session January 28, 2009 Sizing Up the Challenge Our Approach Questions and Answers Timeline and Next Steps.
Wahid Bhimji (Some slides are stolen from Markus Schulz’s presentation to WLCG MB on 19 June Apologies to those who have seen some of this before)
WLCG/DPHEP workshops in Lisbon Andrea Sciabà On behalf of the workshop organisers WLCG GDB December 9, 2015.
Evolution of WLCG infrastructure Ian Bird, CERN Overview Board CERN, 30 th September 2011 Accelerating Science and Innovation Accelerating Science and.
Outcome should be a documented strategy Not everything needs to go back to square one! – Some things work! – Some work has already been (is being) done.
Report TEG WLCG Data and Storage Management Giacinto DONVITO INFN-IGI 14/05/12Workshop INFN CCR - GARR
Technology Research Project Chun-Yi Lin Jan 14, 2008 Chun-Yi Lin Jan 14, 2008.
Riccardo Zappi INFN-CNAF SRM Breakout session. February 28, 2012 Ingredients 1. Basic ingredients (Fabric & Conn. level) 2. (Grid) Middleware ingredients.
Operations Coordination Team Maria Girone, CERN IT-ES GDB, 11 July 2012.
Baseline Services Group Status of File Transfer Service discussions Storage Management Workshop 6 th April 2005 Ian Bird IT/GD.
WLCG Jamboree – Summary of Day 1 Simone Campana, Maria Girone, Maarten Litmaath, Gavin McCance, Andrea Sciaba, Dan van der Ster.
Ian Bird, CERN WLCG Project Leader Amsterdam, 24 th January 2012.
Traceability & Isolation WG Vincent BRILLAULT, CERN/EGI-CSIRT GDB June 2016, CERN.
Evolution of storage and data management
Investigating System Requirements
CRM Project: Requirements Analysis and Definition
WLCG Network Discussion
Sviluppi in ambito WLCG Highlights
Report from WLCG Workshop 2017: WLCG Network Requirements GDB - CERN 12th of July 2017
Global Data Access – View from the Tier 2
Vincenzo Spinoso EGI.eu/INFN
WLCG Data Steering Group
Methodology: Aspects: cost models, modelling of system, understanding of behaviour & performance, technology evolution, prototyping  Develop prototypes.
Joint JRA1/JRA3/NA4 session
Storage Interfaces and Access: Introduction
How to enable computing
Taming the protocol zoo
Final summary Ian Bird Amsterdam, DAaM 18th June 2010.
Proposal for obtaining installed capacity
Presents RAP Week 6 MARCH 11TH 2013.
EGI UMD Storage Software Repository (Mostly former EMI Software)
Agenda Where we are (Amsterdam Architecture)
Status report on the activities of TF-CS/OTA
Status and Future Steps
IWW12 recommendations What are AMVs?
Semantic Web for HealthCare and Life Sciences Interest Group F2F Jan 25-26, 2006, Cambridge Tonya Hongsermeier, Eric Neumann, co-chairs Brian Gilman,
Roger Marks (Huawei) capable 16 January 2019
The ATLAS Computing Model
Roger Marks (Huawei) capable 9 January 2019
Roger Marks (Huawei) capable 18 July 2019
Task Force Peer reviews and quality Eurostat
Roger Marks (Huawei) capable 16 September 2019
Presentation transcript:

Storage / Data TEG Introduction Wahid Bhimji For the storage and data TEGs 7th February 2012 Process so far and to come What will be discussed today For more info see the Twikis: WLCGTEGDataManagement WLCGTEGStorageManagement

Exploration/ Orientation Process Initial questionnaire Defined topics [TopicsDataStorageTEG] Questions to experiments: Experiment Presentations and Twikis [ALICE; ATLAS;CMS;LHCb] Storage Middleware presentations Face-to-face session for each topic plus broader discussions. Now close to agreed set of observations During Feb and March: Develop an overarching picture AND Iteration under each topic (EVO meeting for each one) Information Gathering 24/25 Jan: Face-to-Face Meeting Indico : 165687 Synthesis / Exploration/ Orientation 7th Feb GDB meeting “Emerging” recommendations Refinement “Draft” Recommendations

Process for “Refinement” Overarching view : aka “The diagram” Layers Data Management / Storage Split; Experiment/ Site responsibilities. A subgroup will develop: building on straw man (see Dirks presentation) Topics Editors and writing groups to compress ideas Everybody (in TEG) feeding in - Direct emails to editors; Discussion on lists. EVO meeting on each topic Principles Focus on requirements / principles As expressed by experiments, recognizing concerns of sites, and developers input. Match existing technologies to principles While using solutions open to changing technology

Responsible TEG (for ease of management) TOPICS: As grouped at F2F See Twiki for full details including on overlaps Editor. Experiment co-editor and Writing group for each TBC Data Management Data placement (DM2) and Federation (DM3) Andrew Hanushevsky + Dirk WAN Protocols (DM4) and “FTS” (DM5) Markus Schulz Catalogues (DM9) and Namespaces (DM10) [Brian Bockelman] Security Security and Access Control (DM6/SM6) Maarten Litmaath Storage Management Separation of Disk and Tape (SM3) Andrew Lahiff Storage Interfaces (SM4): SRM and Clouds Paul Millar Management and operation of storage at sites (SM7) Andreases Heiss and Petzold Storage I/O (SM1) , LAN Protocols (SM5) and Evolution of Storage (SM2) Giacinto Donvito Also DM1: Review of Demonstrators (will be in report); DM7 POOL (see Dirk today); DM8 ROOT

This meeting “Emerging” direction / recommendations: Capturing what was discussed at Amsterdam Compiled by “editors” and merged by chairs to single talk per TEG. Some contributions post-Amsterdam from others in TEG but not yet circulated to TEG. Very much still to be discussed within TEG: NOT approved recommendations Feedback / Input very welcome.