1. Meng Cao, Xiangqing Sun, Ziyue Chen May 28th, 2014
Wireless Network
that Forms a Simple Cluster
among Computers and Mobile Devices
Meng Cao, Xiangqing Sun, Ziyue Chen
May 28th, 2014

2. Project Overview
Explore the feasibility of utilizing the computation potential of personal mobile devices (laptops, tablets, or mobile phones, etc) to form a local Hadoop cluster.
All devices will be connected through WiFi Ad-hoc network with one master node and multiple slave nodes.
Master node’s responsibilities:
Split a given task into sub-tasks;
Assign sub-tasks to slave nodes;
Merge returned partial results from slave nodes
Monitor network status, slave node connectivity
Slave node’s responsibilities:
Compute the results of a given sub-task
Transmit the partial result back to master slave

3. Motivation Big Data Expensive, Enterprise Level Machines
Unavailable to public (like us!)
Growing Performance of Smartphones
CPU
storage
network OS
Available Open-Source Platforms
Apache Hadoop
MapReduce, Hadoop Distributed File System (HDFS)
Android SDK
Free, Integrated IDE in Eclipse, Emulators

4. Motivation

5. Background-AD HOC What is Ad Hoc?
decentralized wireless network & no pre existing infrastructure
How does it work?
routing by forwarding data for other nodes
decision made dynamically & flooding
The Wireless Routing Protocol(WRP)
table-based protocol
each node maintains 4 tables:
1. distance table
2. routing table
3. link-cost table
4. message-retransmission list table
communicate through update msgs

6. Background
Hadoop Framework

7. Project Setup WiFi Ad-hoc network setup
Enable WiFi Direct on both Debian laptops and Android phones
Hadoop setup on Linux machines
Hadoop single node setup
Hadoop cluster setup among Debian laptops in LAN
Simple task programming and testing
Configure Hadoop cluster on Ad-hoc networks
Porting Hadoop framework onto Android devices
(More details on the next slide)
Testing & Simulation
Few nodes running test (One laptop as master node, 2-3 Android phones as slave node)
More nodes simulation through Android emulator on PC

8. Port Hadoop to Android
Modify & re-compile Hadoop source code for Android compatibility
Modify & re-compile Java libraries required by Hadoop
DataNode & TaskTracker processes are replaced with service processes
Performance tuning
Memory allocation
CPU utilization

9. Thank you!