PROJECT

Topics  Theoretical: Error Performance Analysis for Partitioned Sketch Data Structures  Survey: Security and Privacy for Big Data: A Survey and Future Directions  Experiments: Citizen Behavior of 7-21 Storm in Beijing, 2012 Music Knowledge Mining Hadoop for Video Streaming on the Web MapReduce Jobs For Video Conversion  Your proposed one…

1. Error Performance Analysis for Partitioned Sketch Data Structures  We talked about the time complexity already (in terms of update time)  TASK: What about error performance? How to optimally allocate the depth of each sketch (zipfian)?  Start to learn from how CM sketch analyzes its error performance (Theorem 1 and alike) full.pdf  Learn about P(d)-CU

How to determine this?

Result  Analysis (e.g., mathematical derivations)  Some initial simulation (correctness)

2. Survey  Write a good survey in English on Security and Privacy for Big Data: A Survey and Future Directions  Cite at least 40+ references (IEEEXplore and ACM Digital Lib)  Paper organization Classify these works in different categories, from different angles Extensive comparisons Identify future directions (i.e., what are the missing pieces?)

Some Materials    intelligence-driven-security-io.pdf intelligence-driven-security-io.pdf  data/GroupDocuments/Big_Data_Top_Ten_v1.pdf data/GroupDocuments/Big_Data_Top_Ten_v1.pdf  papers/wp_addressing-big-data-security-challenges.pdf papers/wp_addressing-big-data-security-challenges.pdf   Think about: Storage Analysis Applications Cloud, Internet-of-Things

3. Analyze Citizen Behaviors of 7-21 Storm in Beijing, 2012  The Power of Social Networks and Public Crowd   Using social network APIs like Sina Weibo open.weibo.com/wiki  Use the keyword search to retrieve all related data  # 望京人赴机场免费救援 # ， # 双闪车队 # (100+)  菠菜 X6 望京网

4. Music Knowledge Mining  Million Song Dataset  For Example: to calculate music density process-a-million-songs-in-20-minutes/ process-a-million-songs-in-20-minutes/  YOUR TASK: Predict which songs a user will listen to

5. Video Streaming on the Web  Store your video as chunks in HDFS  Case: user suddenly move to a specific part of the video  Seek in the file to position the cursor at a specific location  HDFS can only be accessed through a Hadoop client, Apache server is not.  Apache/FUSE: all file system operations (dir browsing, file opening and content access) are enabled over HDFS content through the FUSE interface.  oop_for_video_streaming/ oop_for_video_streaming/

Result  A demo Choose a least 1 type of video format (e.g., flv) A client to play video A web server (with Apache FUSE) HDFS to store your videos

6. MapReduce For Video Conversion  Convert huge number of video files from one format to another.  using the open source video converter FFMPEG (  Data stored on HDFS  Create an app doing it (running on Google AppEngine)

Mechanism  Working in group: 3-5 students, clear roles  me by this Friday (Nov 22) Team leader, Team members Topic  Deadline: 28 December 2013!  Deliverable: project report in Chinese Introduction (motivation, WHY?) Related Work (What others have done) Your proposal (HOW?) Performance Evaluation Conclusion  Presentation

Suggested Arrangement  Week-1: Define your roles and start literature research  Week-2 and 3: Propose solutions  Week-4 and 5: Implementation and obtain results  Week-6: Write report