Summary of the dCache workshop

Slides:



Advertisements
Similar presentations
Distributed Xrootd Derek Weitzel & Brian Bockelman.
Advertisements

Storage: Futures Flavia Donno CERN/IT WLCG Grid Deployment Board, CERN 8 October 2008.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT dpm-xrootd v3 Creating Federated Data Stores for the LHC David.
Filesytems and file access Wahid Bhimji University of Edinburgh, Sam Skipsey, Chris Walker …. Apr-101Wahid Bhimji – Files access.
LHCC Comprehensive Review – September WLCG Commissioning Schedule Still an ambitious programme ahead Still an ambitious programme ahead Timely testing.
16 th May 2006Alessandra Forti Storage Alessandra Forti Group seminar 16th May 2006.
RAL PPD Site Update and other odds and ends Chris Brew.
July-2008Fabrizio Furano - The Scalla suite and the Xrootd1 cmsd xrootd cmsd xrootd cmsd xrootd cmsd xrootd Client Client A small 2-level cluster. Can.
CERN IT Department CH-1211 Geneva 23 Switzerland t Storageware Flavia Donno CERN WLCG Collaboration Workshop CERN, November 2008.
D C a c h e Michael Ernst Patrick Fuhrmann Tigran Mkrtchyan d C a c h e M. Ernst, P. Fuhrmann, T. Mkrtchyan Chep 2003 Chep2003 UCSD, California.
Maarten Litmaath (CERN), GDB meeting, CERN, 2006/02/08 VOMS deployment Extent of VOMS usage in LCG-2 –Node types gLite 3.0 Issues Conclusions.
Redirector xrootd proxy mgr Redirector xrootd proxy mgr Xrd proxy data server N2N Xrd proxy data server N2N Global Redirector Client Backend Xrootd storage.
GLite – An Outsider’s View Stephen Burke RAL. January 31 st 2005gLite overview Introduction A personal view of the current situation –Asked to be provocative!
Light weight Disk Pool Manager experience and future plans Jean-Philippe Baud, IT-GD, CERN September 2005.
Owen SyngeTitle of TalkSlide 1 Storage Management Owen Synge – Developer, Packager, and first line support to System Administrators. Talks Scope –GridPP.
Report on Installed Resource Capacity Flavia Donno CERN/IT-GS WLCG GDB, CERN 10 December 2008.
1 User Analysis Workgroup Discussion  Understand and document analysis models  Best in a way that allows to compare them easily.
Overview of Privilege Project at Fermilab (compilation of multiple talks and documents written by various authors) Tanya Levshina.
INFSO-RI Enabling Grids for E-sciencE Enabling Grids for E-sciencE Pre-GDB Storage Classes summary of discussions Flavia Donno Pre-GDB.
WLCG Grid Deployment Board, CERN 11 June 2008 Storage Update Flavia Donno CERN/IT.
OSG AuthZ components Dane Skow Gabriele Carcassi.
XRootD & ROOT Considered Root Workshop Saas-Fee September 15-18, 2015 Andrew Hanushevsky, SLAC
XROOTD AND FEDERATED STORAGE MONITORING CURRENT STATUS AND ISSUES A.Petrosyan, D.Oleynik, J.Andreeva Creating federated data stores for the LHC CC-IN2P3,
Oracle for Physics Services and Support Levels Maria Girone, IT-ADC 24 January 2005.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Upcoming Features and Roadmap Ricardo Rocha ( on behalf of the.
Andrea Manzi CERN On behalf of the DPM team HEPiX Fall 2014 Workshop DPM performance tuning hints for HTTP/WebDAV and Xrootd 1 16/10/2014.
INFSO-RI Enabling Grids for E-sciencE SRMv2.2 in DPM Sophie Lemaitre Jean-Philippe.
LCG Accounting Update John Gordon, CCLRC-RAL WLCG Workshop, CERN 24/1/2007 LCG.
Data management demonstrators Ian Bird; WLCG MB 18 th January 2011.
Federated Data Stores Volume, Velocity & Variety Future of Big Data Management Workshop Imperial College London June 27-28, 2013 Andrew Hanushevsky, SLAC.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Overview of DMLite Ricardo Rocha ( on behalf of the LCGDM team.
WLCG Grid Deployment Board CERN, 14 May 2008 Storage Update Flavia Donno CERN/IT.
Scientific Computing in PPD and other odds and ends Chris Brew.
SESEC Storage Element (In)Security hepsysman, RAL 0-1 July 2009 Jens Jensen.
CMS: T1 Disk/Tape separation Nicolò Magini, CERN IT/SDC Oliver Gutsche, FNAL November 11 th 2013.
DCache/XRootD Dmitry Litvintsev (DMS/DMD) FIFE workshop1Dmitry Litvintsev.
Andrea Manzi CERN EGI Conference on Challenges and Solutions for Big Data Processing on cloud 24/09/2014 Storage Management Overview 1 24/09/2014.
EGEE is a project funded by the European Union under contract IST Issues from current Experience SA1 Feedback to JRA1 A. Pacheco PIC Barcelona.
New Features of Xrootd SE Wei Yang US ATLAS Tier 2/Tier 3 meeting, University of Texas, Arlington,
OSG STORAGE OVERVIEW Tanya Levshina. Talk Outline  OSG Storage architecture  OSG Storage software  VDT cache  BeStMan  dCache  DFS:  SRM Clients.
Security recommendations DPM Jean-Philippe Baud CERN/IT.
DPM: Future Proof Storage Ricardo Rocha ( on behalf of the DPM team ) EMI INFSO-RI
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES Solutions for WAN data access: xrootd and NFSv4.1 Andrea Sciabà.
EMI is partially funded by the European Commission under Grant Agreement RI Future Proof Storage with DPM Oliver Keeble (on behalf of the CERN IT-GT-DMS.
Implementation of GLUE 2.0 support in the EMI Data Area Elisabetta Ronchieri on behalf of JRA1’s GLUE 2.0 Working Group INFN-CNAF 13 April 2011, EGI User.
WLCG Operations Coordination Andrea Sciabà IT/SDC GDB 11 th September 2013.
SRM v2.2: service availability testing and monitoring SRM v2.2 deployment Workshop - Edinburgh, UK November 2007 Flavia Donno IT/GD, CERN.
Nov 05, 2008, PragueSA3 Workshop1 A short presentation from Owen Synge SA3 and dCache.
Security recommendations for dCache
dCache Paul Millar, on behalf of the dCache Team
a brief summary for users
Daniele Bonacorsi Andrea Sciabà
Jean-Philippe Baud, IT-GD, CERN November 2007
WLCG IPv6 deployment strategy
DPM at ATLAS sites and testbeds in Italy
Ricardo Rocha ( on behalf of the DPM team )
AII v2 Ronald Starink Luis Fernando Muñoz Mejías
Global Data Access – View from the Tier 2
Status of the SRM 2.2 MoU extension
Database Services at CERN Status Update
Experiences with http/WebDAV protocols for data access in high throughput computing
dCache – protocol developments and plans
SRM2 Migration Strategy
Data Federation with Xrootd Wei Yang US ATLAS Computing Facility meeting Southern Methodist University, Oct 11-12, 2011.
Quality Control in the dCache team.
Ákos Frohner EGEE'08 September 2008
DCache things Paul Millar … on behalf of the dCache team.
WLCG Storage planning Andrea Sciabà
INFNGRID Workshop – Bari, Italy, October 2004
Mr. M. D. Jamadar Assistant Professor
Presentation transcript:

Summary of the dCache workshop Andrea Sciabà WLCG Tier-1 Service Coordination Meeting 31/03/2011

Outline New configuration 1.9.5  1.9.12 migration DESY Grid Lab Protocols SRM Other news Golden releases Presentation Title - 2

New configuration Much more flexible Much easier to understand Default values defined in /opt/d-cache/share/defaults and well documented An empty dcache.conf is valid Script to migrate from old to new configuration Script to check the configuration for errors without having to start the service Presentation Title - 3

Migration to new release Tutorial for the migration to 1.9.12 Procedure was well documented and easy, worked flawlessly Large sites can update first head nodes and then each pool, as a rolling upgrade Presentation Title - 4

DESY Grid Lab Used to compare different protocols and different clients ROOT, Hammercloud, CMS analysis Presentation Title - 5

NFS 4.1 pNFS not in SL6, but will be in SL6.2 (backported from kernel 2.6.38) DESY Linux pNFS kernel for SL5 already available Performance tests show it is (almost) never worse than xroot, often much better Presentation Title - 6

Xroot, Global namespace Several performance improvements Java native IO for disk operations, block read-ahead, vector reads for ROOT files, …) Tested using script from R. Brun on Non-optimized ATLAS files (no basket ordering) Optimized ATLAS files Different TTreeCache sizes Different # of branches Results show only slightly slower than SLAC xrootd Global namespace dCache integration in B. Bockelman’s Global Redirector schema via CMSD protocol Presentation Title - 7

SRM Smaller memory footprint Supports SLL instead of GSI (prototype) Synchronous getTURL Falls back to asynchronous if it takes too much Presentation Title - 8

Other news Migration module dCache “Book” being completely updated Very powerful, can be used to balance space usage, supports cksum validation, preserves pins… dCache “Book” being completely updated gPLAZMA II Plugins, flexible configuration Compatible with gPLAZMA1 configuration Will map DN  UID and FQAN  GID, GID, … Ready for ARGUS integration (e.g. for DN blackisting) New logging system GLUE 2.0 supported Improved Pin Manager Presentation title - 9

Golden releases 1.9.5: current golden release, supported until the end of the next LHC winter technical break 1.9.12: new golden release, from April to at least 2012 Proposal: sites upgrade during next technical break Presentation title - 10

Conclusions An overview of what comes with 1.9.12 All European Tier-1’s present Several improvements from the administrator’s point of view Very interesting performance tests Choice of protocol is not really performance-driven The proposed goal is to migrate to it before the 2012 data taking Presentation title - 11