CSE 548 Advanced Computer Network Security Document Search in MobiCloud using Hadoop Framework Sayan Cole Jaya Chakladar Group No: 1

Project Goal Hadoop MapReduce – software framework to write applications to process huge datasets on large number of computer clusters Apache Hadoop Distributed File System – primary storage systems used by Hadoop applications Project – develop a document search algorithm in MobiCloud based on Hadoop and HDFS

Group project Description Install and configure Hadoop in MobiCloud Develop user interface to enable searching and display result Create and update the HDFS data Mapper function to map users to keywords Reduction function to sort the best matches for a search criteria System testing Delivery and demo

Project Description (cont.) TasksResponsible Install and configure Hadoop in MobiCloud Jaya & Sayan Develop UI web applicationJaya Synchronize phone list with virtual machines Sayan Search mapper algorithmSayan Search reduction algorithmJaya HDFS data store creation and updates Jaya & Sayan Testing and problem resolutionJaya & Sayan Delivery and demoJaya & Sayan

Software and Hardware Requirements Hadoop Database software e.g. MySQL or Apache HDFS 3 or 4 Android phones mapped to virtual machines in 2 different Linux boxes

Network topology Web Application to process Requests Hadoop Master Applicatio n HDFS Data Store VM

Project Time Line Week 1Week 2Week 3Week 4Week 5Week 6Week 7Week 8Week 9 Week 10 Week 11 Week 12 Study and understand Mapreduce and Hadoop Install and configure Hadoop Run simple application and demonstrate correctness of implementation Create Mapreduce algorithm particular to specific problem/application Develop the user interface/frontend Installation on Mobicloud Stress checking and testing Analyze and interpret the results Present the application

Technical Details Install and configure Hadoop in MobiCloud Create a UI web application –The user interface of the application will be web based –The output will be shown back in the same web page Create an application to synchronize contact list with virtual machines

Technical Details Develop an application to search for documents –The user requests a document search using some keywords. The input is handled in a webpage hosted as part of the web application –The web application hands over the request to the Hadoop Master application –The master application accesses the contact list of the requesting user

Technical Details –For each entry in the contact list, it creates a job to map the user to the keywords. –The reduction function creates a sorted list of users based on their degree of match –The result set is returned to the web application to update the web page displayed to the user. Creation and updating of Apache HDFS distributed file storage system

Risks and Benefits Novel aspects of this project: –Distributed computing in mobile clouds –Enable meaningful search in a mobile device Risks/challenges: –Cluster failures, debugging and solution –Handling errors e.g. a mobile is switched off or network issues Potential applications & benefits: –Can be used to search for multimedia objects in mobile environment –Access data handling capabilities in a mobile device