ONEDATA Way to access to your Data at the global scale Lukasz Dutka, R. Slota, M. Wrzeszcz, D. Krol, L. Opiola, R. Slota, J. Kitowski ACK Cyfronet AGH

2 ONEDATA Data access layer for distributed data: Across multiple sites Across multiple storages Onedata transparently unifies access to files and in the future to databases Open source project Supported by PL-GRID Tested on PL-GRID infrastructure Project currently run by 17 contributors

3 ONEDATA Overview ONEDATA

4 ONEDATA – user point of view Ease of use and unified access from anywhere Use ANY UI to start (and upload data). Process data at ANY worker node. Mount ONEDATA at your PC and use your data as if they were local. Or simply use Internet browser.

5 Onedata– user point of view Ease of use and unified access from anywhere Data publication and sharing within groups Choose data to share Give URL to your coworkers Manage shared files and stop sharing if needed Simple sharing in groups (throw data to group directory)

6 ONEDATA – user point of view Ease of use and unified access from anywhere Data publication and sharing within groups Metadata support Easily manage metadata using Internet browser. Or manage matadata using command line at any node.

7 Low overhead – the data is created always locally Data migration on fly ONEDATA – administrator point of view

8 Performance Choice of efficient technologies Erlang NoSQL

9 ONEDATA – administrator point of view Performance Scalability No bottleneck for scalability - all system components (including DB) are scalable

10 ONEDATA – administrator point of view Performance Scalability High reliability and availability of infrastructure Redundancy of each component of the system Hot-swapping of code Monitoring Automatic load balancing Built-in DNS for load-balancing

11 ONEDATA – administrator point of view Performance Scalability High reliability and availability of infrastructure Safety Use of only two ports, communication via SSL Authentycation via GSI or OpenID Erlang allows creation of safer code

12 ONEDATA– administrator point of view Performance Scalability High reliability and availability of infrastructure Safety Ease of deployment Deployment of all management machinse using simple wizzard Simple instalation using RPM 1 or 2 (for HA) machines needed to start

13 ONEDATA – administrator point of view Performance Scalability High reliability and availability of infrastructure Safety Ease of deployment Accounting (also for groups)

14 Future Work Unified access to document databases – Plan to use BigCouchDB Distributed indexes of data – plan to applicate Lucene or Elasicsearch Access to data without certificates No central point at the global scale – plan to use P2P techniques and DHT Intelligent rules on data migration – implement user interface part Advanced administration panel – implement user interface part

15 ONEDATA, believes in one solution: Accessed without barriers globally Simple like Dropbox Processed at HPC speed level Capacity with sky limit Freedom of data location Having collaboration in its “blood” Scalable when needed Uninterrupted despites bad days Secure no maters earthquake Files and data object in one place