HEP Data Sharing … … and Web Storage services Alberto Pace Information Technology Division

Htasc meeting, 21 February 2002 Present values of the AFS file sharing services u Native to all UNIX platforms u Fully integrated in the environment, in the shell u Seen by all applications as a “native” file system u It is global on the internet u Files natively shareable across institutes u Good technical implementation (cache) minimizes internet traffic

Htasc meeting, 21 February 2002 AFS is a success in HEP u Lot of information is currently shared across HEP sites using AFS u An AFS file path, identifies universally a file or a directory content u AFS has become “the standard” for file and data sharing in the UNIX/HEP world u HEP users are used to share data across the internet using AFS

Htasc meeting, 21 February 2002 What we would like for the future u Native access, including … u Shell access, Access Control, Source control, versioning, offline files u Global internet sharing u Consolidated standard u Multiple vendors, strong support, interoperability, outsourceable … u Cross platform u Mainly Windows and UNIX u Free u At least the client software u Better if source code is available Difficult to have both

Htasc meeting, 21 February 2002 How to deal with technical difficulties u The Novell Experience u One server (proprietary), multiple protocols u Looks native from each platform (in reality it is not) u Difficult/impossible to map different security models / different namespaces / Case sensitive – insensitive dilemma u Complicated server infrastructure u Cross platform is achieved at the cost of diminished reliability

Htasc meeting, 21 February 2002 u The AFS experience u Unique server type, ad hoc client for each platform u Cannot be native on each platform u Difficult/impossible to map different security models / different namespaces / Case sensitive – insensitive dilemma u Difficult to keep up simultaneously with all clients u Poor NT/2000 support, No good 95/98/Me/Mac support How to deal with technical difficulties

Htasc meeting, 21 February 2002 What we would like for the future u Native access, including … u Shell access, Access Control, Source control, versioning, offline files u Global internet sharing u Consolidated Standard u Multiple vendors, strong support, interoperability, outsourceable … u Cross platform u Mainly Windows and UNIX u Free u At least the client software u Better if source code is available An AFS/DFS-like solution ( ) A web based solution ( ) Technical difficulties

Htasc meeting, 21 February 2002 What we would like for the future u Native access, including … u Shell access, Access Control, Source control, versioning, offline files u Global internet sharing u Consolidated Standard u Multiple vendors, strong support, interoperability, outsourceable … u Cross platform u Mainly Windows and UNIX u Free u At least the client software u Better if source code is available The “Web” can be part of the solution

Htasc meeting, 21 February 2002 The “Web” can be part of the solution u New versions of the HTTP protocol allow managing files on web servers as if these would be part of the local file system u HTTP Extensions for Distributed Authoring (WebDAV) have been released by the IETF as RFC 2518 u Several commercial and public-domain implementations exists already

Htasc meeting, 21 February 2002 What is WebDAV today u File access: u Create / delete files and folders u Read / write files u Copy / Move / Delete / rename files and folders u Document locking u prevent the overwrite problem, where two or more collaborators write to the same resource without first merging changes u Properties u XML properties provide storage for arbitrary metadata (basic features)

Htasc meeting, 21 February 2002 IE 5 on Windows Nautilus- Gnome On Linux

Htasc meeting, 21 February 2002 What could WebDAV implement in a near future u Access control u Set / View / Modify Access Control lists using http u Versioning and Configuration Management u Document check-out, check-in u Retrieval of the history list u Offline files and folders u Other advanced features u Symbolic links u Ordered collections u Aggregated operations (Advanced features)

Htasc meeting, 21 February 2002 For the “advanced features” … u The standard is being discussed now u Proprietary implementations already available

Htasc meeting, 21 February 2002 Is this a solution ? u WebDAV is currently implemented at the “application” level u This means that applications that are WebDAV enabled can see files on web servers, otherwise not This is not a solution. The WebDAV client must be done at the operating system level as a “file redirector”. This means that any application is able access WebDAV servers. This feature is already available on MacOS X, Windows XP and partially Windows There are open source implementations on Linux being worked at. (see news)

Htasc meeting, 21 February 2002 Data sharing in HEP u WebDAV can be part of the solution u WebDAV, at present stage, is not yet THE solution u Every platform needs access to native file system services u File system services offer more than what is in WebDAV u Random Access / direct access u Multi open count / Multiple authors simultaneously on the same file u Record level locking u Notification services u However, if combined with a WebDAV infrastructure, requirements on “file systems” may be reduced u Is cross platform file sharing necessary ? Possible ? u Is global internet file sharing necessary ? Desirable ?

Htasc meeting, 21 February 2002 A possible scenario u Investigate the possibilities to offer HEP-wide web storage services for data sharing matter following internet standards and trends u Is the current technology supported by major vendors really interoperable ? u Can we outsource / insource / switch between vendors and open source as necessary ? u If client WebDAV implementations on LINUX are weak, should the HEP community contribute to open source development ? u In term of file system services, follow the platform specific standards without adding “CERN / HEP enhancements” u We may not fulfill all requirements, but the missing requirements may be fulfilled by standard web storage services u We limit our investment in a technology that is considered less strategic

Htasc meeting, 21 February 2002 CERN Web Services Other WebDAV presentations from the CERN web services are available at the address