http://secret.cis.uab.edu | http://thecenter.uab.edu/ A Comparative Study on I/O Performance between Compute and Storage Optimized Instances of Amazon EC2 Abu Awal Md Shoeb, Ragib Hasan, Md. Haque, and Meng Hu University of Alabama at Birmingham, USA http://secret.cis.uab.edu | http://thecenter.uab.edu/ IEEE CLOUD2014

The Center The UAB Center for Information Assurance and Joint Forensics Research (CIA|JFR) A DHS/NSA Center of Excellence in Information Assurance Research (CAE-R) http://TheCenter.uab.edu/ SECRETLab: http://secret.cis.uab.edu

Outline Introduction Problem of Cloud Computing Motivation Available EC2 Families Benchmark Tool Experimental Setup Results Future Direction Conclusion

What is the Problem? Along with the Security issues, clients face following problems: Clients can not verify whether they really Consume the resources as claimed by the provider Clients ca not ensure whether the cloud provider provides Resources efficiently Moreover, Amazon itself recommends to do load test before selecting an instance for specific application/services (http://aws.amazon.com/ec2/instance- types/#measuring-performance)

Why is it Difficult to Solve? Clients have little access to architectural info due to the black-box nature of clouds Providers are reluctant to share information about actual location of the VM inside a physical machine Clients can not trace resource usage whether it is properly attributed to correct customer or not

Cloud Usability Issues

Our Hypothesis Amazon’s Claim- Storage Optimized instances should have the highest I/O performance with low cost Our Hypothesis- Two instances, one from Compute Optimized and one from Storage Optimized family, have been taken to compare I/O performance. In our experiment, Amazon’s Claim has been disproved almost in 50% cases! Need to maintain a blacklist of phishing sites

Different Families in EC2 General Purpose- provides a balance of compute, memory, and network resources Compute Optimized- provides the highest performing processors and the lowest price/compute performance available in EC2 Memory Optimized- optimized for memory- intensive applications and have the lowest cost per GiB of RAM Storage Optimized- provides very fast SSD- backed storage for very high random I/O performance at a low cost Need to maintain a blacklist of phishing sites

Networking Performance Compute and Storage Optimized Instances Group Instance Type vCPU Memory (GiB)  Storage (GB) Networking Performance Physical Processor Clock Speed (GHz) Compute Optimized c3.large 2 3.75 2 x 16 SSD Moderate Intel Xeon E5-2680 v2 2.8 c3.xlarge 4 7.5 2 x 40 SSD c3.2xlarge 8 15 2 x 80 SSD High c3.4xlarge 16 30 2 x 160 SSD c3.8xlarge 32 60 2 x 320 SSD 10 Gigabit Storage i2.xlarge 30.5 1 x 800 SSD Intel Xeon E5-2670 v2 2.5 i2.2xlarge 61 2 x 800 SSD i2.4xlarge 122 4 x 800 SSD i2.8xlarge 244 8 x 800 SSD hs1.8xlarge 117 24 x 2,048 10 Gibabit Intel Xeon Family

Variables that Affect the I/O Performance Instance Type – Performance varies even two different type instances have same configuration Time – Performance of same instance varies over the time File Size – I/O performance of same instance varies with different file sizes Need to maintain a blacklist of phishing sites

About Benchmark Tool We use standard benchmark tool available in http://www.roylongbottom.org.uk/ We modified the tool (i.e. file size, number of files to be read, write, and delete) for our experiment It measures disk write and read speeds of multiple files at different block sizes Need to maintain a blacklist of phishing sites

Experimental Setup Disk Performance Benchmark Random and Sequential Read, Write, and Delete With different number of files and file sizes (1KB to 160MB) Number of Instances and Location Around 10 instances (5 instances of each group C3 and I2) US East Virginia Number of Benchmark Execution 20 times for each instance Time Frame Through one month

Comparison Summary : Compute VS Storage Optimized Instances Operations Type c3.xlarge i2.xlarge Read Random 46% 54% Sequential 40% 60% Write 50% 51% 49% Delete 92% 8%

Performance Variation – I2 Read Operation

Performance Variation–Random Write

Future Direction Benchmark for other nstances from same group Bechmark for instances located in other regions Benchmark with different file type and storage

Conclusion Our Findings Performance of different instances of same group is not same Performance of an instance varies over time Performance of I2 instances varies with file sizes Instance from appropriate group might not be the best for your application. In our case, Compute Optimized (C3) performed better I/O operation than Storage Optimized instance (I2)

For further queries contact Thank You Q&A For further queries contact shoeb@cis.uab.edu

Why Amazon EC2 Well Known Some instances can be used freely Comparatively less expensive than other clouds Our lab had some funds from Amazon Why we used Amazon for our research/experiment