Security recommendations DPM Jean-Philippe Baud CERN/IT.

Slides:



Advertisements
Similar presentations
30-31 Jan 2003J G Jensen, RAL/WP5 Storage Elephant Grid Access to Mass Storage.
Advertisements

DPM Basics and its status and plans Wahid Bhimji University of Edinburgh GridPP Storage Workshop – Apr 2010 Apr-101Wahid Bhimji – DPM.
DPM Name Server (DPNS) Namespace Authorization Location of physical files DPM Server Requests queuing and processing Space Management SRM Servers v1.1,
Storage: Futures Flavia Donno CERN/IT WLCG Grid Deployment Board, CERN 8 October 2008.
Andrew McNab - EDG Access Control - 14 Jan 2003 EU DataGrid security with GSI and Globus Andrew McNab University of Manchester
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Services Abderrahman El Kharrim
Makrand Siddhabhatti Tata Institute of Fundamental Research Mumbai 17 Aug
OSG End User Tools Overview OSG Grid school – March 19, 2009 Marco Mambelli - University of Chicago A brief summary about the system.
DPM CCRC - 1 Research and developments DPM status and plans Jean-Philippe Baud.
Andrew McNab - SlashGrid, HTTPS, fileGridSite SlashGrid, HTTPS and fileGridSite 30 October 2002 Andrew McNab, University of Manchester
INFSO-RI Enabling Grids for E-sciencE gLite Data Management Services - Overview Mike Mineter National e-Science Centre, Edinburgh.
LFC tutorial Jean-Philippe Baud, IT-GT, CERN July 2010.
Data management in grid. Comparative analysis of storage systems in WLCG.
StoRM Some basics and a comparison with DPM Wahid Bhimji University of Edinburgh GridPP Storage Workshop 31-Mar-101Wahid Bhimji – StoRM.
EGEE-III INFSO-RI Enabling Grids for E-sciencE The Medical Data Manager : the components Johan Montagnat, Romain Texier, Tristan.
The LCG File Catalog (LFC) Jean-Philippe Baud – Sophie Lemaitre IT-GD, CERN May 2005.
CERN IT Department CH-1211 Geneva 23 Switzerland t Storageware Flavia Donno CERN WLCG Collaboration Workshop CERN, November 2008.
Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 The Storage Element.
INFSO-RI Enabling Grids for E-sciencE DPM Administration Jean-Philippe Baud (Sophie Lemaitre)
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE middleware: gLite Data Management EGEE Tutorial 23rd APAN Meeting, Manila Jan.
EMI is partially funded by the European Commission under Grant Agreement RI Argus Policies Tutorial Valery Tschopp - SWITCH EGI TF Prague.
Enabling Grids for E-sciencE Introduction Data Management Jan Just Keijser Nikhef Grid Tutorial, November 2008.
Maarten Litmaath (CERN), GDB meeting, CERN, 2006/02/08 VOMS deployment Extent of VOMS usage in LCG-2 –Node types gLite 3.0 Issues Conclusions.
Light weight Disk Pool Manager experience and future plans Jean-Philippe Baud, IT-GD, CERN September 2005.
SRM Monitoring 12 th April 2007 Mirco Ciriello INFN-Pisa.
INFSO-RI Enabling Grids for E-sciencE Experiences with LFC and comparison with RNS Erwin Laure Jean-Philippe.
WebFTS File Transfer Web Interface for FTS3 Andrea Manzi On behalf of the FTS team Workshop on Cloud Services for File Synchronisation and Sharing.
OSG AuthZ components Dane Skow Gabriele Carcassi.
Jens G Jensen RAL, EDG WP5 Storage Element Overview DataGrid Project Conference Heidelberg, 26 Sep-01 Oct 2003.
Glite. Architecture Applications have access both to Higher-level Grid Services and to Foundation Grid Middleware Higher-Level Grid Services are supposed.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Site Architecture Resource Center Deployment Considerations MIMOS EGEE Tutorial.
INFSO-RI Enabling Grids for E-sciencE Introduction Data Management Ron Trompert SARA Grid Tutorial, September 2007.
EMI INFSO-RI Argus Policies in Action Valery Tschopp (SWITCH) on behalf of the Argus PT.
Linux Operations and Administration
EGEE is a project funded by the European Union under contract IST VO box: Experiment requirements and LCG prototype Operations.
Puppet management for DPM Martin Hellmich Andrea Manzi IT/SDC
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Upcoming Features and Roadmap Ricardo Rocha ( on behalf of the.
1 AHM, 2–4 Sept 2003 e-Science Centre GRID Authorization Framework for CCLRC Data Portal Ananta Manandhar.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Data management in LCG and EGEE David Smith.
Andrea Manzi CERN On behalf of the DPM team HEPiX Fall 2014 Workshop DPM performance tuning hints for HTTP/WebDAV and Xrootd 1 16/10/2014.
INFSO-RI Enabling Grids for E-sciencE /10/20054th EGEE Conference - Pisa1 gLite Configuration and Deployment Models JRA1 Integration.
EMI INFSO-RI Argus The EMI Authorization Service Valery Tschopp (SWITCH) Argus Product Team.
INFSO-RI Enabling Grids for E-sciencE SRMv2.2 in DPM Sophie Lemaitre Jean-Philippe.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Overview of DMLite Ricardo Rocha ( on behalf of the LCGDM team.
SESEC Storage Element (In)Security hepsysman, RAL 0-1 July 2009 Jens Jensen.
Placeholder ES 1 CERN IT EGI Technical Forum, Experiment Support group AAI usage, issues and wishes for WLCG Maarten Litmaath CERN.
Andrea Manzi CERN EGI Conference on Challenges and Solutions for Big Data Processing on cloud 24/09/2014 Storage Management Overview 1 24/09/2014.
EMI is partially funded by the European Commission under Grant Agreement RI Argus Policies Tutorial Valery Tschopp (SWITCH) – Argus Product Team.
Storage Element Security Jens G Jensen, WP5 Barcelona, May 2003.
Bologna, March 30, 2006 Riccardo Zappi / Luca Magnoni INFN-CNAF, Bologna.
Introduction to Storage Element Hsin-Wei Wu Academia Sinica Grid Computing Center, Taiwan.
DPM: Future Proof Storage Ricardo Rocha ( on behalf of the DPM team ) EMI INFSO-RI
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Standard Protocols in DPM Ricardo Rocha.
EMI is partially funded by the European Commission under Grant Agreement RI DPM in EMI-II HTTP and NFS interfaces Oliver Keeble On behalf of DPM.
INFSO-RI Enabling Grids for E-sciencE GUMS vs. LCMAPS Oscar Koeroo.
EMI is partially funded by the European Commission under Grant Agreement RI Future Proof Storage with DPM Oliver Keeble (on behalf of the CERN IT-GT-DMS.
CERN IT Department CH-1211 Genève 23 Switzerland t DPM status and plans David Smith CERN, IT-DM-SGT Pre-GDB, Grid Storage Services 11 November.
Security recommendations for dCache
EGEE Data Management Services
Jean-Philippe Baud, IT-GD, CERN November 2007
AuthN and AuthZ in StoRM A short guide
DPM Installation Configuration
Classic Storage Element
StoRM: a SRM solution for disk based storage systems
Status of the SRM 2.2 MoU extension
The lightweight Grid-enabled Disk Pool Manager (DPM)
Troubleshooting su Installazione SE [DPM]
The INFN Tier-1 Storage Implementation
Data Management cluster summary
INFNGRID Workshop – Bari, Italy, October 2004
Presentation transcript:

Security recommendations DPM Jean-Philippe Baud CERN/IT

EMI INFSO-RI /09/2010 Security recommendations DPM, EGI TF, Amsterdam 2 Introduction Disk Pool Manager (DPM) – Manages storage on disk servers – SRM support (1.1, 2.1 and 2.2) – rfio, gridftp, http(s), xroot – NFS 4.1 interface under development Deployment status – ~200 DPMs in production – 70 VOs supported

EMI INFSO-RI Architecture Very important to backup ! Store physical files -- Namespace -- Authorization -- Replicas -- DPM config -- All requests (SRM, transfers…) Standard Storage Interface Can all be installed on a single machine Data Control

EMI INFSO-RI /09/2010 Security recommendations DPM, EGI TF, Amsterdam 4 Starting/Stopping services General pattern: – service start|stop|restart|status Head node: – dpm – dpnsdaemon – srmv1, srmv2, srmv2.2 – dpm-manager-xrd, dpm-manager-cms (optional xrootd) Disk nodes: – globus-gridftp-server – rfiod – dpm-xrd, dpm-cms (optional xrootd) – dpm-httpd (optional http(s))

EMI INFSO-RI /09/2010 Security recommendations DPM, EGI TF, Amsterdam 5 Log files (1) The services are logging to local log files – DPM server: /var/log/dpm/log – DPM Name Server: /var/log/dpns/log – SRM servers: /var/log/srmv1/log, /var/log/srmv2/log, /var/log/srmv2.2/log – RFIO server: /var/log/rfiod/log – DPM-enabled GridFTP: /var/log/dpm- gsiftp/gridftp.log, /var/log/dpm-gsiftp/dpm- gsiftp.log – Optional web server (Apache); errors also in syslog: /var/log/dpm-httpd/access, /var/log/dpm- httpd/errors – Optional xrootd: /var/log/xrootd/log, /var/log/olbd/log

EMI INFSO-RI /09/2010 Security recommendations DPM, EGI TF, Amsterdam 6 Log files (2) Log format: – 04/13 13:24: ,0 Cns_srv_lstat: NS092 - lstat request by /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=baud/CN=373165/CN=Jean-Philippe Baud (101,101) from lxbra2301.cern.ch – 04/13 13:24: ,0 Cns_srv_lstat: NS098 - lstat 0 /dpm/cern.ch/home/dteam/baud – 04/13 13:24: ,0 Cns_srv_lstat: returns 0 Important messages to look for: – “timeout” – “Csec”, “is banned” – “error:” and “error :” – Number of threads in use Log files are rotated daily, keeping the last 90 days.

EMI INFSO-RI /09/2010 Security recommendations DPM, EGI TF, Amsterdam 7 Service ports dpnsdaemon (5010) - DPM name service for the hierarchical namespace and metadata dpm(5015) - storage management, proprietary protocol srmv1(8443), srmv2(8444), srmv2.2(8446) – storage management, web service protocols over httpg secure rfio(5001, ) - file access protocol gridftp(2811, ) - grid file transfer protocol http(s)(80,443) - HTTP(S) file access protocol (optional) xroot(1094,1095)- xroot file access protocol (optional) ldap (2170) - standard BDII GIP IPv6 support

EMI INFSO-RI /09/2010 Security recommendations DPM, EGI TF, Amsterdam 8 Authentication/Authorization Authentication – X509 proxies with or without VOMS extension – Handled by 2 plugins: Csec and cgsi Authorization – Virtual ids: DNs are mapped to virtual uids when first seen FQANs are mapped to virtual gids when first seen – ACLs on: Name space entries (Posix) Disk pools Dedicated spaces – Privileged operations(pool creation, filesystem drain, …) can only be triggered by superuser on trusted hosts – Physical files are owned by ‘dpmmgr’ Files could be on centrally managed Worker Nodes

EMI INFSO-RI /09/2010 Security recommendations DPM, EGI TF, Amsterdam 9 Configuration files Most of the configuration parameters are kept in the DB: – Disk pool attributes, filesystem statuses … sysconfig files – DPNS_HOST and DPM_HOST – ALLOW_COREDUMP – Log files location DB connect strings – /opt/lcg/etc/NSCONFIG, /opt/lcg/etc/DPMCONFIG /etc/shift.conf – Trusted hosts DPNS TRUST … DPM TRUST … RFIO TRUST – RFIO options (buffer sizes)

EMI INFSO-RI /09/2010 Security recommendations DPM, EGI TF, Amsterdam 10 Banning Requests are rejected if any of DN, CA, VO or primary FQAN attribute is banned Requests having a proxy including a banned secondary FQAN are allowed to proceed as if that FQAN had not been present in the proxy The banning information is cached in the DPNS DB for fast access The banning can be done in 2 ways: – Sysadmin can use dpns-modifyusrmap and dpns- modifygrpmap – A cron job can query the Argus service and automatically update the DPNS DB Banning is part of DPM 1.8.0

EMI INFSO-RI /09/2010 Security recommendations DPM, EGI TF, Amsterdam 11 Documentation – User documentation – Admin documentation – Roadmap – Source code – Current version number tions tions – Tutorials