1. Unified Data Access in Distributed hybrid Cloud
Presented by: Lukasz Dutka

2. DATA IN MULTI-CLOUD ENVIRONMENTS

3. ONEDATA for ONEWORLD

4. PROBLEMS ADDRESED BY ONEDATA FOR HYBRIDCLOUD
Multi-protocol transparent access to data “[…] but we want POSIX” Heterogeneity of storage technologies Replica Management Easy Data Sharing without Borders Metadata Management Integrated with Data Management Platform Flexible authentication and authorization Easy integration using API with external services High-throughput data processing 1 2 3 4 5 6 7 8

5. OPPORTUNITIES FOR LEGACY DATA
Space
Meteo
Space
Scans
Metadata
Sync
Support with synching
to existing collection
Metadata
Sync
Supported with synching
to existing collection
Support with local storage cache
Support with local storage cache
P2P Comm.
P2P Comm.
Access to
Meteo and Scans
as local data.
Meteo Cached
Access to
Meteo and Satellite
as local data.
Scans Cached S3
Scans (Lustre)
Meteo
(NFS)
Ceph
Public Cloud
Private Cloud 1
Private Cloud 2
Private Cloud 3

6. QUESTIONS?
Please visit:

7. BACKUP SLIDES

8. ONEDATA SPACES Metadata change feed Metadata change feed
Community Metadata Index
Metadata change feed
Metadata change feed
User 1
User 2
User 3
Group
Is a kind of Virtual Directory with metadata
Is a kind of Virtual Directory with metadata
Each Space might be Supported by manyProviders
P2P
P2P
ONEDATA Provider
ONEDATA Provider
ONEDATA Provider
ONEDATA Provider
Direct Access
Direct Access
Local Network Attached Storage
Storage
LustreFS
CephFS
Amazon S3
Brain Research Facility 1
Active Repository Center 2
Active Repository Center 2
Brain Research Facility 2

9. PROBLEM 8: High-throughput processing
Protocols CDMI
Protocols S3
Protocols POSIX VFS
P2P
P2P
Control,
Remote Data Access
CDMI API
Storage Access
Direct Access
if possible
Parallel Processing Nodes using POSIX oneclient, CDMI or REST
P2P
Ceph S3
SWIFT
Lustre

10. THROUPUT TESTS
55Gbit/s
On single node
5 parallel streams