EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Data Management Interface: CDMI for CMF Ilja Livenson PDC KTH

EGI-InSPIRE RI Background & motivation Based on the CDMI-Proxy server developed in FP7 VENUS-C Developing ‘v2’ as part of the EGI mini- project “From scratch” New name – Stoxy (STOrage proXY) Target use case – AWS S3-like storage

EGI-InSPIRE RI Technology Python + Twisted + ZODB + ZCA Buildout packaging Installs with a script Multiplatform 3

EGI-InSPIRE RI Stoxy architecture OMS management framework Stoxy plugin SSH endpoint CDMI endpoint Zope DB (VOMS- enabled) OpenStack Keystone 1. Get CMS token 2. Add it as a HTTP header

EGI-InSPIRE RI Data vs Metadata Sizes are not clear any more But use cases are still there Stoxy’s target is on a smarter server-side processing E.g. filtering or post-processing of the data based, metadata enrichment Depends on the scenarios 5

EGI-InSPIRE RI CDMI Security Different credentials are used for those cases (in the setting of the academic infrastructure) Username/passwords CMS token-based (OpenStack Keystone) CDMI unfortunately is not Keystone token compliant (yet) Abusing standard to get it working

EGI-InSPIRE RI Stoxy authentication HTTP Basic/Digest X509 client certificate based Via Apache HTTPD Keystone authentication Only CMS (crypto) tokens Token = signed blob of information from Keystone

EGI-InSPIRE RI Keystone token example 8 { "access": { "metadata": {....metadata goes here.... }, "serviceCatalog": [....endpoints goes here.... ], "token": { "expires": " T08:52:53Z", "id": "placeholder", "issued_at": " T18:59: ", "tenant": { "description": null, "enabled": true, "id": "925c23eafe1b e08a4c4143f08", "name": "user" } }, "user": {....userdata goes here.... } Generated token

EGI-InSPIRE RI Explorative browsing with SSH Stoxy-specific functionality Comes with OMS framework Not CDMI interface Use putty for operating on the Stoxy objects/containers Useful for debugging/browsing

EGI-InSPIRE RI Supported CDMI feature set Blob CRUD JSON-encoded Streaming Container CRUD ACL – planned to be full At the moment requires admin privileges

EGI-InSPIRE RI Plans Finding scenarios We are happy to provide extensive help with integration Finishing documentation/example cases Integration into Fedcloud’s infrastructure Requires enabling of the PKI tokens in Keystone installations Implementation of the lightweight browser on top

EGI-InSPIRE RI Summary Open Source CDMI server (Apache2 license) Includes python client sdk/cli Accent on metadata manipulation Currently in active development, first production release is planned in Dec 2014 Feedback/contributions are very welcome!

EGI-InSPIRE RI And many thanks to EGI Federated Cloud TF group Andrei Sosnin, OpenNode PDC KTH 13 Thank you for listening!