2. Future home directories at CERN
Alberto Pace
CERN, Geneva, Switzerland
With contributions from: German Cancio, Dirk Duellmann, Javier Espinal, Massimo Lamanna, Luca Mascetti, Jakub Moscicki, Andreas Peters.

3. Storage services at CERN
120 PB, 350 million files
70 PB, 500 million files
Storage of physics data
EOS, Castor Tape Archive
Storage of user data
DSF, AFS
Storage of corporate documents
CDS, EDMS, Indico, Alfresco, …
450 TB, >3000 million files
700 TB, >800 million files
Not discussed further

4. Multiple roles Castor Tape Archive High reliability, Low cost
Archives, physics data preservation
High Throughput, High latency
Bulk data transfer, Integrity
Reliable and Cheap
(high latency)
Fast and reliable
(expensive)
Fast and cheap
(Unreliable)

5. Multiple roles EOS High scalability, Arbitrary performance
Arbitrary reliability
Designed to sustain 1 KHz access
Main platform for Physics Analysis at CERN
Reliable
and Cheap
(high latency)
Fast and reliable
(expensive)
Fast and
cheap
(Unreliable)

6. Multiple roles Key requirements AFS, DFS
Storage of user’s and project files
Few PB, many files (billions)
Key requirements
Simple, Scalable, Consistent, Reliable, Available, Manageable, Flexible, Performing, Native, Cheap, and Secure.

7. Requirements of user’s file storage
Native online access
As a mounted file system
Global native access, global http access
/afs/cern.ch/user/…,
Secure, Access Control, Encrypted
Backed up, Journaled,
Cross platform offline access
Access to physics data, unlimited storage

8. Several missing requirements
AFS
DFS
Native online Access
Windows D C
Linux
OS X
(C)
iOS
Android
Global native online Access
Offline Access, syncing (without third party tools)
Web access
Access control
Journaled
Encrypted
Access to Physics data

9. New opportunities with EOS …
Recent development for the physics data storage infrastructure have opened new opportunities

10. Architecture EOS Client Web browser access Owncloud sync client
(web access)
OwnCloud Web Server
(implements sharing)
Owncloud sync client
http OC sync protocol
(Etag, X-OC-Mtime,
If-Match, OC-FileId)
(offline access)
Client
(online access)
http,
webdav,
xroot
EOS

11. New opportunities with EOS …
Recent development for the physics data storage infrastructure have opened new opportunities
Implement OwnCloud http extensions
EOS appearing as a native Owncloud server
Empowered the use of unmodified Owncloud syncing client
Can use the Owncloud web server on top of EOS storage
Easy sharing to unregistered identities
Read/Write sharing
Bulk download

12. Basic functionality Access to the whole storage repository
All Physics (and user) data at CERN
Could implement access to WLCG/EOS/HTTP federated storage worldwide
Mount EOS – Online file access
On Linux, Mac : Use FUSE or WebDAV
On Windows : Webdav (…)
Programmatic, high performance access
Xroot (and http) libraries for all programming languages

13. Syncing functionality
Sync clients available for several platforms
Can chose what to sync among the entire namespace
User / Physics / Project data
DEMO

14. Syncing functionality
Sync clients available for several platforms
Can chose what to sync among the entire namespace
User / Physics / Project data

15. Sharing functionality
Directory-based access control lists
Inherited from EOS service
Supports for CERN Accounts, E-groups, Roles
Sharing functionality added by the Web Server interface
Remote users do not need an account at CERN
Read / Write access, expiry date
DEMO

16. Sharing functionality
Directory-based access control lists
Inherited from EOS service
Supports for CERN Accounts, E-groups, Roles
Sharing functionality added by the Web Server interface
Remote users do not need an account at CERN
Read / Write access, expiry date
DEMO

17. Usage Scenarios EOS Client Web browser access Owncloud sync client
(web access)
OwnCloud Web Server
(implements sharing)
Owncloud sync client
(offline access)
http OC sync protocol
(Etag, X-OC-Mtime,
If-Match, OC-FileId)
Client
(online access)
http,
webdav,
xroot
EOS

18. End-user files EOS Multiplatform access sync client Same user
Sharing with
Third parties
(web access)
(web access)
OwnCloud Web Server
(implements sharing)
sync client
Choose what to sync
(offline access)
Online / offline
File Sharing
(using EOS
Or sync client)
Same user
EOS
(online access)

19. Project space EOS Multiuser access sync client OwnCloud Web Server
Contributions by
Third parties
(web access)
(web access)
OwnCloud Web Server
(remote access using browser)
sync client
(offline access)
(offline user working on cached data)
Contributions by
Third parties
(web access)
(online user working remotely)
EOS
(online access)

20. Physics Analysis EOS Client Web browser access sync client
OwnCloud Web Server
(implements sharing)
Web browser access
(web access)
Analysis results
(Local Physics Data)
sync client
Choose what to sync
(a small subset of data)
prototyping
Client
Analysis results
batch service
http,
webdav,
xroot
EOS

21. A new platform for analysis
Web browser access
Client
OwnCloud Web Server
with Root.js
Virtual CPU container
http,
webdav,
xroot
EOS

22. Future of home directory services
With the availability of CERNBOX + EOS we are offering several new services, but we have duplication of services with AFS
All AFS use cases being currently reviewed
On-line access from desktops / laptop : sync via CERNBOX, mount via WEBDAV (evtl FUSE mount)
World-wide remote access : Xroot, Webdav + HTTP
PLUS + BATCH Access : (permanent) FUSE mount (being investigated). HOMEDIR on EOS
AFS project space, WEB sites hosted in AFS: Moved to CERNBOX / EOS (being investigated)
Other cases (SW build & distribution, …) : local builds, CVMFS
Currently asking the user community to report
use cases for which end-users have concerns
Other use cases not on the list that may require alternate/additional solutions or investigations

23. Questions ?