1. CTA: CERN Tape Archive Rationale, Architecture and Status
Germán Cancio, Daniele Kruse, Eric Cano, Steven Murray
CASTOR face to face meeting, June 2016
CTA Project

2. Rationale & objectives
CTA :CERN Tape Archive
Stand alone tape system
Decoupled from disk storage
First target EOS, potentially others systems
Straightforward architecture with minimum number of elements
Session preemption: optimized use of drives
Target is LHC Run 3:
EOS+CTA as the DAQ interface for all experiments
CASTOR face to face meeting, June 2016
CTA Project

3. Architecture Shared storage concept New queueing system
Only 2 daemons: front end for CLI (xrootd based) & taped.
New queueing system
Based on Ceph
Only for transient data
Each queue (per tape/tapepool) is an independent object
Avoids a single huge queue table
Allows storage of rich objects
Separate file catalogue
Based on usual relational DB
For persistent data
cta-taped an adapted tapeserverd from CASTOR
CASTOR face to face meeting, June 2016
CTA Project

4. Architecture CTA Catalogue CTA tape server CTA front-end CTA
All of the CTA business logic and data management code is in the client interface to metadata
CTA Catalogue
Evolved from CASTOR
Files
Routing
Commands/control
Tape pools
Mount policies
CTA
tape server
CTA
front-end
CTA
tape server
CTA
front-end
CTA
tape server
CTA Queues
Drive status
Queues
Tape
Tape
Tape
Tape
Tape
Tape
Two prototypes
Ceph
Local file system
Data
CTA
command-line
tools
CTA
command-line
tools
CTA
CLI
Disk
Disk
Remote
storage
EOS
w/workflow
Disk
Disk
CASTOR face to face meeting, June 2016
CTA Project

5. EOS integration EOS triggers archivals and retrievals
Handled by workflow engine
Through CLI interface of CTA
EOS maintains a stub for each file
Namespace belongs to EOS
CTA files referenced by unique numeric ID
Path only interpreted in EOS
EOS does not query CTA
CTA maintains Disaster Recovery (DR) data
File metadata stored in the catalogue as a DR blob
CASTOR face to face meeting, June 2016
CTA Project

6. Status “Passive EOS” integration achieved end 2015
CTA transfers file in/out of an EOS instance
Hand triggered via the CLI
Prototype with a full namespace, now removed
Some persistent data on Ceph side
Namespace based on shared filesystem
“Active EOS” integration targeted for 2016
Integration with EOS’s workflow engine
Changes needed on both sides
New catalogue replaces namespace
Queuing system adapted to new catalogue and performance validated
Refactoring of cta-taped code
Longer term targets
Repack
Verify
Session preemption instead of dedication
Saturate the otherwise idle drives with low-priority, high-volume sessions (repack, verify)
Yield to higher priority sessions (user access)
Disaster recovery
CASTOR import (metadata)
cta-taped improvements in parallel (low priority)
Recommended Access Order (drive-recommended read order)
Out of order archiving (writing files in reception from disk order)
CASTOR face to face meeting, June 2016
CTA Project