DPM Python tools Ivan Calvet IT/SDC-ID DPM Workshop 10 th October 2014.

Slides:



Advertisements
Similar presentations
Prime’ Senior Project. Presentation Outline What is Our Project? Problem Definition What does our system do? How does the system work? Implementation.
Advertisements

Grid and CDB Janusz Martyniak, Imperial College London MICE CM37 Analysis, Software and Reconstruction.
Red Hat Linux Network. Red Hat Network Red Hat Network is the environment for system- level support and management of Red Hat Linux networks. Red Hat.
Basic Grid Job Submission Alessandra Forti 28 March 2006.
Report Distribution Report Distribution in PeopleTools 8.4 Doug Ostler & Eric Knapp 7264.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment, Enhanced Chapter 8: Implementing and Managing Printers.
MCTS Guide to Microsoft Windows Server 2008 Network Infrastructure Configuration Chapter 8 Introduction to Printers in a Windows Server 2008 Network.
Member of the ExperTeam Group Ralf Ratering Pallas GmbH Hermülheimer Straße Brühl, Germany
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 8: Implementing and Managing Printers.
Makrand Siddhabhatti Tata Institute of Fundamental Research Mumbai 17 Aug
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES News on monitoring for CMS distributed computing operations Andrea.
Module 1: Installing Internet Information Services 5.0.
Verify Hardware Requirements Install Windows Server 2008 R2 Configure Active Directory Install SQL Server 2008 Install SharePoint Server 2010 Configure.
SRM at Clemson Michael Fenn. What is a Storage Element? Provides grid-accessible storage space. Is accessible to applications running on OSG through either.
GLite D. Zaborov, SA1-SA3 meeting, Dubna1 Dmitry Zaborov (SINP-MSU-Moscow) Dubna, Jul 26, 2007 Testing 3.0 under SLC 4.
Introduction to CVMFS A way to distribute HEP software on cloud Tian Yan (IHEP Computing Center, BESIIICGEM Cloud Computing Summer School.
Project Server 2003: DC340: Security (Part 1 of 2): How to securely deploy Project Server in an enterprise environment Pradeep GanapathyRaj (PM), Karthik.
Monitoring in EGEE EGEE/SEEGRID Summer School 2006, Budapest Judit Novak, CERN Piotr Nyczyk, CERN Valentin Vidic, CERN/RBI.
CERN IT Department CH-1211 Geneva 23 Switzerland t Storageware Flavia Donno CERN WLCG Collaboration Workshop CERN, November 2008.
02/07/09 1 WLCG NAGIOS Kashif Mohammad Deputy Technical Co-ordinator (South Grid) University of Oxford.
Write-through Cache System Policies discussion and A introduction to the system.
DIRAC Review (13 th December 2005)Stuart K. Paterson1 DIRAC Review Exposing DIRAC Functionality.
Author - Title- Date - n° 1 Partner Logo WP5 Summary Paris John Gordon WP5 6th March 2002.
First attempt for validating/testing Testbed 1 Globus and middleware services WP6 Meeting, December 2001 Flavia Donno, Marco Serra for IT and WPs.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Direct gLExec integration with PanDA Fernando H. Barreiro Megino CERN IT-ES-VOS.
Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Usage of virtualization in gLite certification Andreas Unterkircher.
Alberto Aimar CERN – LCG1 Reliability Reports – May 2007
INFSO-RI Enabling Grids for E-sciencE Enabling Grids for E-sciencE Pre-GDB Storage Classes summary of discussions Flavia Donno Pre-GDB.
Stephen Burke – Data Management - 3/9/02 Partner Logo Data Management Stephen Burke, PPARC/RAL Jeff Templon, NIKHEF.
Grid Deployment Enabling Grids for E-sciencE BDII 2171 LDAP 2172 LDAP 2173 LDAP 2170 Port Fwd Update DB & Modify DB 2170 Port.
Configuring and Troubleshooting Identity and Access Solutions with Windows Server® 2008 Active Directory®
VOMS: Status & Plans Vincenzo Ciaschini, Valerio Venturi MWSG Meeting, CERN, Feb
Shell Interface Shell Interface Functions Data. Graphical Interface Graphical Interface Command-line Interface Command-line Interface Experiments Private.
HLRmon accounting portal DGAS (Distributed Grid Accounting System) sensors collect accounting information at site level. Site data are sent to site or.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Using GStat 2.0 for Information Validation.
INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.
SAM Sensors & Tests Judit Novak CERN IT/GD SAM Review I. 21. May 2007, CERN.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Upcoming Features and Roadmap Ricardo Rocha ( on behalf of the.
Service Availability Monitor tests for ATLAS Current Status Tests in development To Do Alessandro Di Girolamo CERN IT/PSS-ED.
SSH/SSL Attacks not on tests, just for fun. SSH/SSL Should Be Secure Cryptographic operations are secure SSL uses certificates to authenticate servers.
DGC Paris Spitfire A Relational DB Service for the Grid Leanne Guy Peter Z. Kunszt Gavin McCance William Bell European DataGrid Data Management.
Testing Infrastructure Wahid Bhimji Sam Skipsey Intro: what to test Existing testing frameworks A proposal.
The GridPP DIRAC project DIRAC for non-LHC communities.
SAM Status Update Piotr Nyczyk LCG Management Board CERN, 5 June 2007.
Storage Element Security Jens G Jensen, WP5 Barcelona, May 2003.
INFN/IGI contributions Federated Clouds Task Force F2F meeting November 24, 2011, Amsterdam.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks CYFRONET site report Marcin Radecki CYFRONET.
Security recommendations DPM Jean-Philippe Baud CERN/IT.
1 DIRAC Project Status A.Tsaregorodtsev, CPPM-IN2P3-CNRS, Marseille 10 March, DIRAC Developer meeting.
DPM: Future Proof Storage Ricardo Rocha ( on behalf of the DPM team ) EMI INFSO-RI
EMI is partially funded by the European Commission under Grant Agreement RI Future Proof Storage with DPM Oliver Keeble (on behalf of the CERN IT-GT-DMS.
Maria Alandes Pradillo, CERN Training on GLUE 2 information validation EGI Technical Forum September 2013.
CERN IT Department CH-1211 Genève 23 Switzerland t DPM status and plans David Smith CERN, IT-DM-SGT Pre-GDB, Grid Storage Services 11 November.
Virtual Organization Management Registration Service (VOMRS) T. Levshina J. Weigand S. White Co-Authors: L. Bauerdick, G. Carcassi, I. Fisk, A. Heavey,
The Grid Information System Maria Alandes Pradillo IT-SDC White Area Lecture, 4th June 2014.
Jean-Philippe Baud, IT-GD, CERN November 2007
Introduction of load balancers at the RAL Tier-1
EGI Operations Management Board
Lesson 6: Configuring Servers for Remote Management
Ricardo Rocha ( on behalf of the DPM team )
Classic Storage Element
Introduction to CVMFS A way to distribute HEP software on cloud
DPM Python tools Andrea Manzi CERN DPM Workshop 07th December 2015.
Testing for patch certification
Future Test Activities SA3 All Hands Meeting Dublin
DPM releases and platforms status
DPM and SRM-less operation
Data Management cluster summary
UMD update 11 Joao Pina (EGI/LIP).
Operational Tools & Middleware Versions Monitoring
Presentation transcript:

DPM Python tools Ivan Calvet IT/SDC-ID DPM Workshop 10 th October 2014

Introduction  Dmlite Shell  Dpm-listspaces  Webdav interface of DPM Endpoints DPM Python tools 2

Dmlite Shell  Shell to administrate DMLite  Made for admins and developers, not for users  You can manage directories, files, replicas, pools, users, groups,... (41 commands)  A few commands are still to be done:  Filesystem related commands  Space related commands  Replicate and drain commands  Now integrated to the core of dmlite DPM Python tools 3

Dpm-listspaces  Script to populate the GLUE BDIIs (1.3 and 2) with the informations of each DPM site  Lots of improvements in DPM and 1.8.9:  Correction of wrong capacity numbers  Correction of bugs to send correct information  Addition of new elements (version number, some GLUE2EndpointInterface added, …)  Patchwork of fixes which need to be supervised (specially for the capacity numbers) DPM Python tools 4

Webdav interface of DPM Endpoints  128 DPM Endpoints with a Webdav interface are registered in GLUE2.0.  Tests made as a dteam user from CERN:  Reading access: ~40 out of 128 work  Writing access: ~25 out of 128 work  It is necessary to improve this situation. DPM Python tools 5

Testing Webdav access of an endpoint  Nagios probe developed and sent to the SAM instance of EGI  Internal web interface to list all the results from the tests on all the grid storage endpoints  Please, test with a HTTP client from lxplus: davix-put -P grid /etc/services davix-ls -P grid DPM Python tools 6

Common errors to access to an endpoint through Webdav  HTTP server not running or firewall trouble:  Could not connect to server  Connection timed out  Wrong SSL configuration:  Neon error: SSL handshake failed: SSL error: sslv3 alert certificate expired  Neon error: Server certificate verification failed: certificate has expired, issuer is not trusted  Neon error: SSL handshake failed: SSL error: certificate verify failed  Server misconfiguration:  HTTP 500: Unexcepted Server Error.  Non-standard path:  HTTP 404: File not found.  Wrong VOMS configuration:  HTTP 403: Permission refused. DPM Python tools 7

Summary  Dmlite Shell is a powerful tool to manage Dmlite for administrators  Dpm-listspaces got a lot of updates recently and needs to be supervised  ~20% of the DPM Endpoints have a Webdav interface correctly working and we need to improve this DPM Python tools 8