1. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 Lukasz Dutka RIA-653549 INDIGO-DataCloud is co-founded by the Horizon 2020Framework Programme AMSTERDAM - MEETING 4-5 APRIL 2016 INDIGO-DataCloud DEMO

2. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 AGENDA Short Introduction to onedata Onedata 3.0 – what’s new Demo:  Demo Environment  Dockerized Distribution and automated installation  Example scenarios for distributed data access  CDMI Access  Client in docker based on HBP Image Service story  Summary 1 2 3

3. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 ONEDATA BIG PICTURE

4. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 ONEDATA SPACES User 1User 2User 3Group Community Metadata Index METADATA CHANGE FEED P2P ONEDATA Provider Storage LustreFS CephFS Amazon S3 Each Space might be Supported by many providers Local Network Attached Storage Indigo Cloud Provider 2Indigo Cloud Provider 1 Direct Access

5. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 ONEDATA SYSTEM ARCHITECTURE FUSE Client Oneclient HTTP GUI REST FUSE Client HTTP GUI REST FUSE Client Onezone

6. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 ONEDATA 3.0 – WHAT’S NEW  Redesigned from scratch internal architecture of Onedata 2.x  Access tokens based on macaroons  Support for POSIX, S3 and Ceph storages  Provides CDMI and POSIX access to the data  Support for Zones  Internal Database migrated to Couchbase  Fully dockerized  Batch configuration and depolyment  Many tests at several levels: unit, integration, acceptance, performance, stress Current Status Beta 2

7. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 DOCKERIZED DISTRIBUTION & AUTOMATED INSTALLATION docker-compose- client.yml docker-compose.yml Docker Onezone Docker Oneprovid er Docker Oneclient oz-worker Couchbase DB xyz- worker Couchbase DB FUSE oneclient Samba NFS xyz-worker oz-worker Access Token

8. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 DEMO ENVIRONMENT Amazon S3 DNS: p-aws-useast INFN Italy Docker Oneclient Docker AWS USA Docker Onezone VM onezone Docker Oneclient Docker NFS Server VM oneprovider VM nfs VM oneclient POSIX Volume Docker Oneclient Docker UPV Spain VM: demo-onedata-upv-provider Docker Oneclient Laptop OSX SAMBA Export boot2docker

9. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 SPACE DISTRIBUTION FOR DEMO Space Name Provider INFNProvider UPV Provider AWS USA User Lukasz Dutka User Lukasz Opiola EO SentinelSupported Access Granted HBP ScansSupported Access Granted Lukasz Dutka's spaceSupportedAccess Granted Lukasz Opiola's space Supported Access Granted AMS Sensors Supported Later InvitedAccess Granted

10. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 CDMI Command structure: https://ALIAS.onedata.indigo-datacloud.eu:8443/cdmi ALIAS -> ldutka is my alias in this zone. System automatically do redirects at HTTP or even on DNS level to automatically find the right oneprovider. User can point urls directly to the selected provider, if he wants to be sure which provider he wants to use: https://p-infn.onedata.indigo-datacloud.eu:8443/cdmi Onezone DNS Server (opt.) DNS Server (opt.) DNS Server (opt.)

11. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 CDMI SUPPORTED CAPABILITIES OperationsCapabilities Basic object GET PUT DELETEcdmi_dataobjects, cdmi_read_value, cdmi_modify_value, cdmi_delete_dataobject Basic container GET PUT DELETEcdmi_list_children, cdmi_create_container, cdmi_delete_container Metadata (container&dataobject)cdmi_read_metadata, cdmi_modify_metadata, cdmi_size, cdmi_(atime|mtime|ctime) Access control lists (rwx)cdmi_acl Big folderscdmi_list_children_range File System Export (FUSE client)- Move and copycdmi_(move|copy)_(container|dataobject) Big filescdmi_read_value_range, cdmi_modify_value_range Access by ObjectIDcdmi_object_access_by_ID

12. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 CLIENT IN DOCKER BASED ON HBP IMAGE SERVICE HBP Scans Space 5TB Oneclient HBP Image Viewer /srv/data HBPScans/ HBP Atlas Viewer

13. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 HIGH PERFORMANCE ACCESS

14. INDIGO DATACLOUD MEETING AMSTERDAM 4-5 th APRIL 2016 SUMMARY  Release soon 3.0 beta 2 version  We are finalizing the release 3.0  GUI is a little behind the schedule  Selected New features:  Distribution via dockers and packages (RPM and DEB)  Fully automation of installation using yml configs.  Onezone  Support for Ceph and S3  GUI build as a JS app  Internal architecture build for plugins and storage drivers  Internal authorization based on macaroons