1. Hadoop Distributed File System (HDFS) implementation in GENI Wei Kou – University of Connecticut Madhav –Missouri University of Science and Technology Sheyda – University of Missouri Kansas City Min Sang Yoon – Iowa State University

2. Introduction of Hadoop Hadoop configuration in GENI(single site) Multiple sites configuration Simulation result Contents

3. Apache Hadoop is an open-source software framework written in Java for distributed storage and distributed processing of very large data sets on computer clusters built from commodity hardware. Composed with a single name node and data node clusters. Use Map-reduce programming model to distribute single file HDFS (Hadoop Distributed File Systems)

4. Hadoop configuration in GENI (single site) -One maser node -5 data nodes -Configured the cluster Master node 172.16.1.1 172.16.1.10 172.16.1.14 172.16.1.12 172.16.1.13172.16.1.12

5. Hadoop configuration in GENI (single site) We configured 128 GB Capacity HDFS.

6. Hadoop configuration in GENI (single site) Each data node allocate 25.6GB for HDFS

7. Hadoop configuration in GENI (single site) File distribution command File list on HDFS Master Worker-0

8. Hadoop configuration in GENI (multiple sites) Purpose : To observe how physical distance affect to performance of network. We generated 4 slices configured in different sites. Master node is located in same site in all scenarios. Two data nodes are assigned in same site with master node and three data nodes are assigned in different sites. All are connected to same subnet. case 1: GPO(master) – Texas A&M case 2: GPO(master) – UC Davis case 3: GPO(master) - Wayne State University case 4: GPO(master) – University of Florida GPO(master) – UC Davis

9. Hadoop configuration in GENI (multiple sites) Wayne State University UC Davis Texas A&M University of Florida

10. Simulation configuration We generated 1Gb dataset for each case. We measure data transmission time of each case. 128 GB HDFS capacity. 25GB from each data node.

11. Simulation Result Distribution time (case 2 result) Distribution time (case 3 result) CaseDistanceDistribution time Single site0 miles19 seconds GPO – Wayne State University717 miles7min GPO – University of Florida1220 miles7min 36 seconds GPO – Texas A&M1862 miles7min 55 seconds GPO – UC Davis3027 miles8 min 30 seconds

12. Simulation Result second

13. Conclusion & future work Hadoop distributed file system can be implemented in GENI successfully. We could observe the relationship between physical distance and network time. However, the affection of physical distance is not very significant than our expectation. We should consider other factors more carefully in deciding load distribution in network.