1. MID-SEM REVIEW

2. Contents Abstract Introduction Scope Objectives
Existing system & Proposed system
Algorithms
Design
Modules with functionalities
Implementation
Action Plan

3. Abstract
Social Networking sites provide tremendous impetus for Big Data in mining people’s opinion. Public API’s catered by sites such as Twitter provides us with useful data for perusing writer’s attitude with reference to a particular topic, product etc. To discern people’s opinion, tweets are tagged into positive, negative or neutral indicators. This project provides an effective mechanism to perform opinion mining by designing a end to end pipeline with the help of Apache Flume, Apache HDFS, Apache Oozie and Apache Hive. To make this process near real time we study the workaround of ignoring Flume tmp files and removing default wait condition from Oozie job configuration. The underlying architecture employed here is not restricted only to opinion mining but also has a gamut of applications. This paper explores few of the use cases that can be developed into actual working models.

4. Introduction
Twitter data sentiment analysis can be an excellent source of information and can provide insights that can:
Determine marketing strategy
Improve campaign success
Improve product messaging
Improve customer service
Generate leads

5. Scope
The recent emerging area of interest is sentiment analysis of social issues.
Now a day most of the research scholars have been working on Twitter and YouTube comments data set.
To perform sentiment analysis the most and common source of data set are web pages, social web site like face book, twitter, YouTube etc.
There is a vast scope for research scholars to increase the accuracy level up to some extent by using well designed sentence structure.
But, Sarcastic comments are the ones which are very difficult to identify.

6. Objectives
To implement an algorithm for real time classification of twitter data into positive, negative or neutral sets.
To extract the meaning of an input text or tweet using natural language processing.
To determine the attitude of the mass into various objective sets towards the subject of interest.
To improve the accuracy of the analysis using our algorithm.

7. EXISTING SYSTEM
There are many non big data softwares, in the non big data softwares we cannot store large amount of data and we cannot retrieve the data immediately.
DISADVANTAGES OF EXISTING SYSTEM
No immediate data retrieval
Less security
Efficiency is less

8. PROPOSED SYSTEM
We proposed System to extract opinion based on sentiment analysis for dynamic tweet data which is stored in HDFS file system ,by using this HDFS file system data transmission is so fast and by default fault tolerance will be given by Big data.
ADVANTAGES OF PROPOSED SYSTEM
Actionable intelligence can be achieved
Security increases
Speed of transmission
Huge amount of Data Retrieval can be done

9. Algorithms Topic Adaptive Sentiment Classification
Auto Inclusion of Sensitive Word
Keyword Identification

10. Topic Adaptive Sentiment Classification
In social media, a Twitter user may have different opinions on different topics. Thus, topic adaptation is needed for sentiment classification of tweets on emerging and unpredictable topics.
The algorithm focuses on cross-domain sentiment analysis on tweets, and we propose a semisupervised topic-adaptive sentiment classification model (TASC). It transfers an initial common sentiment classifier to a specific one on an emerging topic.

11. TASC has three key components.
The semi-supervised multiclass SVM model is formalized.
We set feature vector in the model into two parts: fixed common feature values and topic-adaptive feature variables
To tackle the content sparsity of tweets, more features are extracted, and split into two views: text and non-text features.
The algorithm iteratively minimizing the margins of two independent objectives separately on text and non-text features to learn coefficient matrices.

12. Auto Inclusion of Sensitive Word
This algorithm would simply look up a word in a dictionary, and if not present there, it was probably misspelled. Unfortunately, not all misspelled words result in an unknown word. Misspelled words resulting in existing words are called context-sensitive spelling errors, since a context is required to detect an error.
The proposed method mitigates the effect of sparse data by preprocessing the corpus and extracting extra information on PoS tags.

13. Keyword Identification
Keyword extraction from text data is a common tool used by search engines and indexes alike to quickly categorize and locate specific data based on explicitly or implicitly supplied keywords.
Various methods of locating and defining keywords have been used, both individually and in concert.
Despite their differences, most methods have the same purpose and attempt to do the same thing: using some heuristic (such as distance between words, frequency of word use, or predetermined word relationships), locate and define a set of words that accurately convey themes or describe information contained in the text.

14. DESIGN

15. Modules with Functionality
Extracting Tweets from twitter based on the query keyword In this module based on the twitter access key and consumer key, we are going to extract tweets based on the query keyword. Storing Tweets to Hadoop In this module extracted tweets are stored into data nodes of hadoop . Based on the Hash tag Retrieve data In this module whatever the data stored in the hadoop we are going to retrieve back in the format of json data, based on the hash tag we will get tweets and finally store it into database.

16. Modules with Functionality
Preprocessing of Tweets
Remove Unnecessary words
Remove Hyperlinks
Remove Special characters
Get filtered data
Sentiment Process
In this module we have keep first initial positive, negative and neutral words ,based on this initial expansion we are going to give positive,negative and neutral count for the words in the tweet and finally we will detect the sentiment of the tweet

17. Implementation

18. Action Plan Storing of tweets into HDFS Preprocessing of tweets
Classification of tweets
Final Result