Big Data

Big Data – How Important is It? The Economist Intelligence Unit – “A game-changing asset” World Economic Forum – “Big impact” Forbes – “Big market and big business” McKinsey & Company – “The next frontier for innovation, competition, and productivity”

Big Data Other descriptions of “Big Data”? Examples of “Big Data”? What is it? Very large datasets Datasets so large: difficult to process and analyze According to a survey reported by CIO.com, 38% of director-level respondents worldwide don't know what BIG DATA is Source: http://www.itpro.co.uk/strategy/19434/big-facts-about-big-data

Big Data IOPS – Input / Output Processing per Second Source: http://www.esg-global.com/blogs/big-data-a-better-definition/

Big Data How "BIG" is it?

Nearly 3 Billion People will be online by 2015 BIG DATA Comscore was capturing 1.4 trillion digital interactions per month by 2012 Nearly 3 Billion People will be online by 2015 Source: MGI: Big data: The next frontier for innovation, competition, and productivity, McKinsey & Company, 2011.

BIG DATA Walmart stores more than 1 million transactions per hour in its 2.5 petabyte databases 90% of the world's data were created in the last 2 years 750 million Facebook users create an average of 90 pieces of content per month 294 billion e-mails are sent per day Source: The Economist Intelligence Unit; IBM

Business and Big Data Credit card companies checking for fraudulent transactions - is a purchase legitimate? Airlines monitoring the data from a plane traveling through the sky – are there any problems with the engine? Companies monitoring social media for customer complaints and compliments – should we respond to customers on Twitter or Facebook?

Volume Velocity Variety BIG DATA The Three V’s of Big Data Volume Velocity Variety

Volume One petabyte could hold more than 58,000 movies Source: http://www.wipro.com/services/analytics-information-management/

Variety unstructured data text audio video click streams log files Source: http://www.wipro.com/services/analytics-information-management/

Velocity 3 BILLION Estimated number of phone calls on which the U.S. National Security Agency (NSA) collects data per day Source: http://www.wipro.com/services/analytics-information-management/

BIG DATA

Making Sense of BIG DATA CHALLENGES Making Sense of BIG DATA A visualization of thousands of Wikipedia edits that were made by a single software bot. Each color corresponds to a different page. Image: Fernanda B. Viégas, Martin Wattenberg, and Kate Hollenbach

A single version of the truth CHALLENGES A Single Version of the Truth

RESISTANCE Challenges Why do some managers resist dealing with Big Data?

CHALLENGES

Unsupervised Learning Solutions Cluster Analysis Classification Crowdsourcing Simulation Pattern Recognition Supervised Learning Neural Networks Machine Learning Unsupervised Learning

SOLUTIONS Open Source Software for storing and processing large volumes of unstructured and semi-structured data. Can process and analyze huge amounts of unstructured data – video, audio, social media postings, images, etc. Allows large data sets to be processed across clusters of computers. Uses parallel computing power to analyze data with incredible speed, compared to traditional data analysis.

BIG DATA Management SOLUTIONS Evolving Strategic Data Managers Data Wasters companies that collect data but severely underutilize it Data Collectors companies that collect a large amount of data but do not consistently maximize their value Aspiring Data Managers companies that understand the value of data and are marshaling resources to take better advantage of them Strategic Data Managers companies that have well-defined data management strategies that focus resources on collecting and analyzing the most valuable data Source: The Economist Intelligence Unit

Next RFID…