Distributed Computing Systems Current Issues in DCS Dr. Sunny Jeong. Mr. Colin Zhang With Thanks to Prof. G. Coulouris, Prof. A.S. Tanenbaum and Prof. S.C Joo

Cluster, Grid, Cloud …Now…Hadoop?  To Support hadling Big data  Hadoop HDFS + MapReduce –designed so that any node failures are automatically handled by the framework. –enables applications to work with thousands of computation- independent computers and petabytes of data.  Facebook Yahoo..of course GOOGLE

Overview  Hadoop  Computer Cluster  Grid Computing  Grid Computing Projects 3

Hadoop Architecture  Wikipedia

MapReduce Architecture  Yuan Luo

HDFS Architecture  Dhruba Borthakur

Clustering Computing  Consists of a set of loosely connected computers that work together as a single system.  The components of a cluster are usually connected to each other through fast LAN, each node with own OS  Availability of low cost microprocessors, high speed networks, and software for high performance distributed computing 7

Grid Computing  Computational Grid: –Focus on CPU scavenging/stealing technology for complicated operation  Data Grid): Managing and sharing distributed intensive data  Access Grid: For Cooperation between remote computer using audio or video applications  Equipment Grid : Controlling remote machine and analyzing data from the machine.

Grid Project BOINC in 2011)

Cloud Computing( comparison with Grid)  Service Oriented  Transaction oriented  No standardization Organization  No Standard Technology  More less homogeneous gathering (compare to Grid)  Loosely connected 10

Barriers of the Concepts  Shim young cheol