Testing the Zambeel Aztera Chris Brew FermilabCD/CSS/SCS Caveat: This is very much a work in progress. The results presented are from jobs run in the last two days and have not been fully checked for validity.

Aim To test the overall performance of the Zambeel Aztera Network Attached Storage System To test the overall performance of the Zambeel Aztera Network Attached Storage System To develop suite of tests that can be used for comparisons between different NAS devices To develop suite of tests that can be used for comparisons between different NAS devices

What is it? From a user point of view it’s just a large “black box” network attached NFS server serving out a 2.8TB filesystem From a user point of view it’s just a large “black box” network attached NFS server serving out a 2.8TB filesystem Behind the scenes it consists of multiple “compute” nodes and “storage” nodes. Compute nodes interface between the storage and the clients making multiple disks across multiple nodes appear as one (or more) filesystems. Load balancers share the load amongst the compute nodes. Very redundant - every component is duplicated, data is mirrored and striped across strorage nodes. Behind the scenes it consists of multiple “compute” nodes and “storage” nodes. Compute nodes interface between the storage and the clients making multiple disks across multiple nodes appear as one (or more) filesystems. Load balancers share the load amongst the compute nodes. Very redundant - every component is duplicated, data is mirrored and striped across strorage nodes.

Tests Manageability Tests: Manageability Tests: –Ease of setup –Ease to Reconfigure Performance Tests: Performance Tests: –Max Throughput: Read and Write –Max Throughput Reading a Single File –Simultaneous Reads and Writes –Creation, Listing and Deletion of Large Numbers of Small Files –Data Integrity Failure Tests: Failure Tests: –Fail various parts of the system and see what happens

Tools Reader/Writer : Reader/Writer : –Fermi Built tools that write and read a file IOZone: IOZone: –Standard Benchmark tool with many options. Used here for the data integrity tests. Bonnie++: Bonnie++: –Another standard benchmark tool. Used for it’s File Creation/Deletion test.

Why Three Tools? All have different Strengths/Weaknesses All have different Strengths/Weaknesses –Reader/Writer: Very Simple and Easy to script around –IOZone Many Many modes –Bonnie++ Only one with file creation/deletion test

Performance Tests So Far All tests use the new hotdog nodes (Dual 1.6 GHz, 1GB Memory, 100Mb/s Network) All tests use the new hotdog nodes (Dual 1.6 GHz, 1GB Memory, 100Mb/s Network) Completed Tests: Completed Tests: –Max Throughput (n clients with 1 or 2 processes writing/reading 1GB files. Rate for all files completed before the first node to complete 5 files are averaged on each node and summed across the nodes) –Single File Throughput (same as previous read test but all nodes/processes read the same file) –Client comparison (Measure the single client read/write speeds for different clients)

Maximum Throughput

Tuning Changes Throughput

Different Clients/Different Performance

Reading a Single File

Failure Testing Whilst the system was up and clients were reading and writing data, removed and replaced one by one: Whilst the system was up and clients were reading and writing data, removed and replaced one by one: –Compute Module –Storage Module –Load Balancer Performance was obviously degraded but the system stayed up and serving it’s clients Performance was obviously degraded but the system stayed up and serving it’s clients

Conclusions For 50 Clients the system gives a total throughput of ~120Mb/s for writes and ~220Mb/s for reads For 50 Clients the system gives a total throughput of ~120Mb/s for writes and ~220Mb/s for reads Reliability seems to be high. It can survive any single component failure Reliability seems to be high. It can survive any single component failure With these tools we have the basis of a test suite and procedure for comparing the different storage technologies that are becoming available With these tools we have the basis of a test suite and procedure for comparing the different storage technologies that are becoming available