Big-Data Fundamentals An Introduction Yogesh Kumar

Big Data Definition “Big Data” is data whose scale, diversity, and complexity require new architecture, techniques, algorithms, and analytics to manage it and extract value and hidden knowledge from it…

What is Big-Data Big Data is a collection of large datasets that cannot be processed using traditional computing techniques. It is not a single technique or a tool, rather it involves many areas of business and technology. ‘Big-data’ is similar to ‘Small-data’, but bigger. …but having data bigger consequently requires different approaches: techniques, tools & architectures …to solve: New problems… …and old problems in a better way.

Some numbers of Data How many data in the world? 800 Terabytes, 2000 60 Exabytes 2006 (1EB = 1018B) 4.5 Zettabytes 2013 (1ZB = 1021B) 44 Zettabytes by 2020 How much is a zettabyte? 1,000,000,000,000,000,000,000 bytes A stack of 1TB hard disks that is 25,400 km high How many data in a day? 2.5 Exabytes 8 TB, Twitter 50 TB, Facebook 90% of world's data: Generated over last two years!

What Comes Under Big Data? Big data involves the data produced by different devices and applications. Given below are some of the fields that come under the umbrella of Big Data. Black Box Data: It is a component of helicopter, airplanes, and jets, etc. It captures voices of the flight crew, recordings of microphones and earphones, and the performance information of the aircraft. Social Media Data: Social media such as Facebook and Twitter hold information and the views posted by millions of people across the globe. Stock Exchange Data: The stock exchange data holds information about the ‘buy’ and ‘sell’ decisions made on a share of different companies made by the customers. Power Grid Data: The power grid data holds information consumed by a particular node with respect to a base station. Transport Data: Transport data includes model, capacity, distance and availability of a vehicle.

What Comes Under Big Data? Search Engine Data: Search engines retrieve lots of data from different databases. Healthcare: Big Data has a huge application in healthcare, Big Data has a huge application in healthcare, Retails: The retail sector is built on an understanding of the consumers’ retail habits. Top retailers are mining customer data and using Big Data technologies to help make decisions about their marketing campaigns, merchandising and supply chain management. Education: In the education sector, learners are creating information at the same time as they are consuming knowledge. Students are faced with increasingly demanding curricula where they are no longer expected to regurgitate facts from hard memorizing but are required to learn the subjects with deep understanding. Finance: Big Data plays a significant role in the finance sector, especially with regard to fraud detection with the application of Complex Event Processing (CEP)65.

Benefits of Big Data Using the information kept in the social network like Facebook, the marketing agencies are learning about the response for their campaigns, promotions, and other advertising mediums. Using the information in the social media like preferences and product perception of their consumers, product companies and retail organizations are planning their production. Using the data regarding the previous medical history of patients, hospitals are providing better and quick service.

Big Data Statistics 100 Terabytes of data is uploaded to Facebook every day Facebook Stores, Processes, and Analyzes more than 30 Petabytes of user generated data Twitter generates 12 Terabytes of data every day LinkedIn processes and mines Petabytes of user data to power the "People You May Know" feature YouTube users upload 48 hours of new video content every minute of the day Decoding of the human genome used to take 10 years. Now it can be done in 7 days 500+ new websites are created every minute of the day