1. CASTOR 2.1.15 Giuseppe Lo Presti on behalf of the CASTOR dev team
CASTOR Face-to-face meeting, 22/09/2014

2. Outline 2.1.15 Content (cf. Phoneconf June '14) Stager refactoring
Ceph and its integration in CASTOR
Xroot as the main internal protocol
Movers (protocols) handling
Tape servers
SL(C) 5/6 support
Beyond : future developments and open discussion

3. Stager refactoring 1/2
Up until , the stager daemon goes to the DB multiple times per file request
Using C++ legacy framework
Lots of code that can be simplified In
stagerd goes to DB the minimum number of times
All user requests handling logic is PL/SQL
stagerjob is entirely dropped, along with a number of internal requests
This led to dropping ~24 KLoC and 7 tables from the DB
File close happens in the Stager DB, using the DB Link to the Nameserver DB

4. Stager refactoring 2/2
Advantage: (much) less code, stepping stone to transparently introduce DataPools
Still to be done: support different nbReplicas on the same shared DiskPool according to the different service classes
Again the simplified code allows a more straightforward implementation

5. Movers handling
New interface between movers and the scheduling system (diskmanagerd):
At transfer starting, diskmanagerd spawns the appropiate mover with its environment
At transfer closing, all movers synchronously connect to diskmanagerd to pass the metadata to close the file
Uniformly across all {rfio, xroot, gsiftp} movers
Xroot is not special (any longer)
On the way, dropped support for updates

6. Movers - Xroot 4.x CASTOR plugin ported to Xroot 4
Lots of code refactoring on the way
3rd party copy custom implementation dropped
Use of standard xrdcp --tpc required
Ability to catch client disconnections and notify diskmanagerd about those failed transfers
Removes the need for configuring a long (hours) timeout for xroot transfers – reducing the chance of DoS...
A timeout will still exist for the start of the transfer, because of the intrinsic architecture of xroot, i.e. the inability to call back the clients when a slot is ready. But this is in the order of 5 minutes...

7. Tape server developments
A new daemon called tapeserverd replaces the legacy tape-server daemons tapebridged, rtcpd and taped
First edition: based on RFIO. But ready for XROOT in the future
More details in Eric's presentation
A modification is needed in the xrootd plugin to allow tapeserverd to talk XROOT to a disk server running
A new release is on the pipe to address this (and a few more issues)

8. SL 5/6 support
The bold statement is: Support for SL5 is deprecated in v2.1.15
The reality:
Head nodes software will only support SL6
Tape servers software will only support SL6
Disk servers will be partially supported in SL5 and fully in SL6, provided python 2.6 is installed
A python26 RPM has been ported to SL5
Ceph is not supported in SL5
Stress tests are (going to be) performed only in SL6. We will separately test this specific SL5 disk server configuration at a functional level.

9. A few more loose ends
Full plan at
Most important items before finalizing :
stage_open API implementation: handle update
Implement xroot API for querying running transfers and use it into diskmanagerd to 'readopt' them upon restart
Prevents slots overallocation, fixes listtransfers output
Allows identifying tape transfers (broken in !)
Secure RPyC channel
Modify permission checks logic to allow recalls without Put permission?

10. Beyond 2.1.15 Plans and open discussion
...Let's have a look at last year's whish list ...

11. Discussions to have What we would like to do (easiest first) cmake SRM
StagerJob integration & Stager rework
Xroot refactoring
CEPH integration
Lesser priority (highest priority first)
Secure RH
Nameserver improvements
More Stager improvements
Castor Dev team Dec 9th Post plans

12. Status as of 2.1.15 What we would like to do √ cmake X SRM
√ StagerJob integration & Stager rework
√ Xroot refactoring
√ CEPH integration
Lesser priority
X Secure RH → on the way, need to take GC out
X Nameserver improvements
X More Stager improvements → partially done, remains merging the Stage*Request tables

13. Wish list beyond 2.1.15 VDQM/VMGR merge and rewrite
Incorporating the tape admin side of CUPV
More tomorrow on Tape Developments
Namespace split + refactoring
Refreshing the last bit of the code base dating from mid 1990s...
Merging (the rest of) CUPV
Prevent trivial DoS attacks...
GC metadata traffic to move out of the Request Handler
Replace 'internal requests' with e.g. Google's ProtocolBuffer, already adopted in the new tapeserverd
Secure the Request Handler

14. Wish list beyond 2.1.15, cont'd SRM Still there?
RFIO deprecation – need to provide escape paths
Client-side library that translates RFIO to XROOT?
Pro: XROOT as wire protocol
Cons: library to be maintained for potentially many platforms/OSs
Or server-side xrootd plugin?
Pro: better/easier control on the server side
Cons: wire protocol still RFIO. What about IPv6?
What about RFIO usage at Tier 1s?

15. (More) Questions? What about your wish list?