1. On the Varieties of Clouds for Data Intensive Computing 董耀文 1098308101 碩資工一甲 @ Antslab Robert L. Grossman University of Illinois at Chicago And Open Data Group Yunhong Gu University of Illinois at Chicago

2. Outline  What is Cloud ?  Types of Clouds  Clouds provide on-demand computing capacity  Experimental Studies  Research Questions

3. What is Cloud?

4.  An infrastructure.  At scale and reliability of a data center.  Provides resources or services.  Google  Hadoop  Amazon’s EC2

5. Types of Clouds

6.  Architecural Model  Loosely coupled commodity computers.  Computing instances on demand.  Amazon’s EC2  US $0.10 /1hr  1.0~1.2GHz 2007 Opteron or Xeon processor  1.7 GB memory  160GB disk  Moderate I/O performance

7. Types of Clouds  Tightly Coupled  Loosely coupled

8. Types of Clouds  Architecural Model  Computing capacity on demand  Google’s MapReduce  TeraSort use1800 machines.  2GHz Xeon process  4GB memory  2 * 160GB IDE disks  TeraSort  Sort 100-byte records. ( 1TB data )  Required ~= 891s

9. Types of Clouds  Architecural Model  Open source  Eucalyptus (Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems)  University of California, Santa Barbara  Linux & Xen  Hadoop  Hadoop MapReduce  HDFS  HBase

10. Types of Clouds  Programming Model  On-demand computing support any computing model compatible with loosely coupled clusters.  Amazon EC2  MapReduce :  Map : map each pair into a new pair of  Reduce : merges values with the same key  Sector/Sphere : User Defined Function(UDF)

11. Types of Clouds  Management Model  Internal vs Hosted  Private vs Shared  Combinations(Hybrid)

12. Types of Clouds  Payment Model  Pay as you go.  Buy.  make arrangements with a third party to pay for the exclusive use of cloud resources for a specified period of time.

13. Types of Clouds  What’s New?  New scale.  Hadoop  Google  New simplicity clouds provide.  Amazon’s EC2,S3  AMI(Amazon Machine Image)

14. Clouds provide on-demand computing capacity  Google Cloud  GFS (Google File System)  MapReduce  BigTable

15. Clouds provide on-demand computing capacity  MapReduce m(Bear) m(River)

16. Clouds provide on-demand computing capacity

17.  UDT(UDP-based Data Transfer)  Designed for extremely high speed networks.  Concurrent UDT flows share the available bandwidth fairly.  Resides completely at the application level.  User defined congestion control algorithms.  Easier to traverse the firewall.

18. Experimental Studies

19.  10 Gb/s networks.  30 DELL 1435 computer  4G memory  1TB disk  2.0GHz dual-core ADM Opteron 2212  1Gb/s NIC

20. Experimental Studies hadoopsector

22. Experimental Studies  CreditStone  credit card transactions.  flags some of the transactions.

23. Research Questions  Quite easy to use.  Develop appropriate network protocols, architectures and middleware for wide area clouds.  How different clouds can interoperate.  Develop standards and standards based architectures for cloud services.  Alternate storage  Compute  Table services

24.  THANKS