1. http://secret.cis.uab.edu | http://thecenter.uab.edu/
A Comparative Study on I/O Performance between Compute and Storage Optimized Instances of Amazon EC2
Abu Awal Md Shoeb, Ragib Hasan, Md. Haque, and Meng Hu
University of Alabama at Birmingham, USA |
IEEE CLOUD2014

2. The Center
The UAB Center for Information Assurance and Joint Forensics Research (CIA|JFR)
A DHS/NSA Center of Excellence in Information Assurance Research (CAE-R)
SECRETLab:

3. Outline Introduction Problem of Cloud Computing Motivation
Available EC2 Families
Benchmark Tool
Experimental Setup
Results
Future Direction
Conclusion

4. What is the Problem?
Along with the Security issues, clients face following problems:
Clients can not verify whether they really Consume the resources as claimed by the provider
Clients ca not ensure whether the cloud provider provides Resources efficiently
Moreover, Amazon itself recommends to do load test before selecting an instance for specific application/services ( types/#measuring-performance)

5. Why is it Difficult to Solve?
Clients have little access to architectural info due to the black-box nature of clouds
Providers are reluctant to share information about actual location of the VM inside a physical machine
Clients can not trace resource usage whether it is properly attributed to correct customer or not

6. Cloud Usability Issues

7. Our Hypothesis
Amazon’s Claim- Storage Optimized instances should have the highest I/O performance with low cost
Our Hypothesis- Two instances, one from Compute Optimized and one from Storage Optimized family, have been taken to compare I/O performance.
In our experiment, Amazon’s Claim has been disproved almost in 50% cases!
Need to maintain a blacklist of phishing sites

8. Different Families in EC2
General Purpose- provides a balance of compute, memory, and network resources
Compute Optimized- provides the highest performing processors and the lowest price/compute performance available in EC2
Memory Optimized- optimized for memory- intensive applications and have the lowest cost per GiB of RAM
Storage Optimized- provides very fast SSD- backed storage for very high random I/O performance at a low cost
Need to maintain a blacklist of phishing sites

9. Networking Performance
Compute and Storage Optimized Instances
Group
Instance Type
vCPU
Memory (GiB)
 Storage (GB)
Networking Performance
Physical Processor
Clock Speed (GHz)
Compute
Optimized
c3.large 2
3.75
2 x 16 SSD
Moderate
Intel Xeon E v2
2.8
c3.xlarge 4
7.5
2 x 40 SSD
c3.2xlarge 8 15
2 x 80 SSD
High
c3.4xlarge 16 30
2 x 160 SSD
c3.8xlarge 32 60
2 x 320 SSD
10 Gigabit
Storage
i2.xlarge
30.5
1 x 800 SSD
Intel Xeon E v2
2.5
i2.2xlarge 61
2 x 800 SSD
i2.4xlarge
122
4 x 800 SSD
i2.8xlarge
244
8 x 800 SSD
hs1.8xlarge
117
24 x 2,048
10 Gibabit
Intel Xeon Family

10. Variables that Affect the I/O Performance
Instance Type – Performance varies even two different type instances have same configuration
Time – Performance of same instance varies over the time
File Size – I/O performance of same instance varies with different file sizes
Need to maintain a blacklist of phishing sites

11. About Benchmark Tool
We use standard benchmark tool available in
We modified the tool (i.e. file size, number of files to be read, write, and delete) for our experiment
It measures disk write and read speeds of multiple files at different block sizes
Need to maintain a blacklist of phishing sites

12. Experimental Setup Disk Performance Benchmark Random and Sequential
Read, Write, and Delete
With different number of files and file sizes (1KB to 160MB)
Number of Instances and Location
Around 10 instances (5 instances of each group C3 and I2)
US East Virginia
Number of Benchmark Execution
20 times for each instance
Time Frame
Through one month

13. Comparison Summary : Compute VS Storage Optimized Instances
Operations
Type
c3.xlarge
i2.xlarge
Read
Random
46%
54%
Sequential
40%
60%
Write
50%
51%
49%
Delete
92% 8%

14. Performance Variation – I2 Read Operation

15. Performance Variation–Random Write

16. Future Direction Benchmark for other nstances from same group
Bechmark for instances located in other regions
Benchmark with different file type and storage

17. Conclusion Our Findings
Performance of different instances of same group is not same
Performance of an instance varies over time
Performance of I2 instances varies with file sizes
Instance from appropriate group might not be the best for your application. In our case, Compute Optimized (C3) performed better I/O operation than Storage Optimized instance (I2)

18. For further queries contact
Thank You
Q&A
For further queries contact

19. Why Amazon EC2 Well Known Some instances can be used freely
Comparatively less expensive than other clouds
Our lab had some funds from Amazon
Why we used Amazon for our research/experiment