DDN Web Object Scalar for Big Data Management Shaun de Witt, Roger Downing (STFC) Glenn Wright (DDN)

Topics A brief introduction to Object Stores What is WOS –Additional features Integration with Other Tools Some performance testing metrics Costs Wrap up

Object Stores

What is an Object Store? NOT a traditional file system –No hierarchical namespace No directory hierarchy –No POSIX (by default) No ‘ls’, ‘stat’, ‘open’, etc methods –Not mountable (‘by default’) Mostly limited functionality –Read, write, delete

So why do you want one? Massively scalable –10’s of petabytes –Billions of objects (that’s UK billions!) –Ideal for ‘cloudy’ storage Easier to administer –Survives losses of drives or storage nodes –No central point of control –Faster rebuild times Fast access –Data distributed across many nodes –Fast lookup of object location Simple Interface –Write, Read, Delete (again)

So where can I get one? Kinetic

What is WOS?

WOS Summary DataDirect Networks Web Object Scaler For the geeks, each WOS ‘appliance’ has –Up to 60 drives, –4x10GigE, 40GigE, IB –Up to 128MB RAM –4U chassis

So what makes it different? Global, policy based, federation –Based on ‘zones’ –See next slides Security (kerberos) ‘Heat map’ of network connections –Makes sure you can get to data as fast as possible from wherever you are Policy based self-healing

A Simple Policy Make 2 copies in Zone A Client

A More Complex Policy Write to Zone A Replicate to Zone B Zone A Zone B Client

More Complex Still… Write to Zone A Replicate to Zone B asynchronously Zone A Zone B Client

Synchronously create replicas in zones A and B then async. create replicas in zones C,D and E Zone A Zone B Zone C Zone D Zone E Client And policies can be quite complex……

Summary of Policies Based on ‘zones’ which can overlap Simple to set up web based admin interface But to make the best use of them… –understand your use case –have some knowledge of network latencies –ask if you really need this functionality

Data Integrity (per object) Traditional Replication Local Copy ObjectAssure Replicate ObjectAssure Global ObjectAssure 3x Capacity performance efficiency reliability scalability  1.25x Capacity performance efficiency reliability scalability  2.5x Capacity performance efficiency reliability scalability  <1.88x Capacity performance efficiency reliability scalability 

Integration with Other Tools

IRODS + WOS User search, access, add and manage data & metadata IRODS iRODS Rule Engine Track Policies iRODS Metadata Catalogue Track data WOS Data Locality Manager policy driven replication Object Supervisor Global data mgmt.

GPFS + WOS GPFS NFS/CIFS WOS Bridge WOS Access

Performance Testing

Info and Caveats Box was on loan from OCF –limited testing time Two WOS appliances available –at two separated sites WOS software has been updated since testing –figures are probably better now Not enough repetition

Testing through IRODS Namespace scalability Sequentially add small (100 byte) files with checkpointing every 5k 2.26 Mobjects inserted in 1 week –Performance impacted by erasure encoding security –Latest software much more performant

Number of Objects Time to Write 5k Files (secs)

Scale-out Tests Read/Write/Delete jobs on 1GB file Increasing number of compute nodes used wget and curl for all commands –First two slides shows performance in ‘typical’ compute where jobs are scheduled –Third shows impact of all jobs starting concurrently

More testing Also tested failover between two sites and loss of disks Rebuild times very quick compared with out current RAID6

COSTS

Costs NOT A SALESMAN OR REPRESNTING DDN But most purchasers know you pay a premium for DDN Hardware BUT… WOS is available software only –Can be (and is) run on modern commodity hardware If you want to know more – contact your local salesman

Wrap-Up

Summary WOS is potentially a very useful tool –multi site/ multi domain data sharing –Simple and resilient storage –Flexibility in defining rules –Good for multiple projects –Seems very scalable from initial test For more info, check the DDN web pages – web-object-scaler-wos/

Thanks… To OCF and DDN for supplying the hardware and support Part of this work was performed on behalf of EUDAT ( … and to all of you for listening