S EEQ C ORPORATION Big Data Oregon Connections Telecommunications Conference Dustin Johnson October 23, 2014
Introduction Dustin Johnson Software architect and founding partner at Seeq Background in computer networks and distributed systems Founded on the premise that companies need better solutions for quickly and easily deriving business insight from their Industrial Process Data Leverages many classical Big Data tools and design patterns 2
What is Big Data? 3 Marketing term English words
What is Big Data? - Problem 4 Economist Intelligence Unit Survey, October 2010 More data all the time The number of sensors is increasing at 30% every year (The Economist – Building with big data, May 26 th 2011) Corporate data is growing by 60% every year (Gartner – Economic Intelligence Unit – Leveling the playing field 2011) Economist Intelligence Unit Survey, October 2010
What is Big Data? - Scalability 5 Scalability – Ability to make the system capability larger or smaller as needed. A typical example is adding or removing computers from the system. Emphasizes the scalability of a software system, often by leveraging distributed computing Scale is not just the volume of data, but can be computational resources, bandwidth, memory, and reliability as well Turns a technical problem into a business solution: More money = more capability
What is Big Data? - Opportunity More data could equal more insight Opportunity to better know your customers Opportunity to substantiate or originate business ideas Predicted $16.1 billion market for big data in 2014 (Forbes 12/12/2013) Opportunity for tool and service providers in the Big Data space 6
Where did Big Data come from? The rise of the Internet-scale company in the 2000s These business needs were challenging since the data was Huge! And Growing! These demands lead to Distributed Computation / Massively Parallel Processing (MapReduce) NoSQL Databases (BigTable, Dynamo) Many Big Data companies shared slightly out of date concepts and even Open Source Software (OSS) 7
What are the key technologies now? Data Processing Platforms Not Only SQL (NoSQL) Databases 8
Where is Big Data going? Deeper into consumers (finer grained market segmentation needs) Predictive analytics (Machine learning / Deep Learning, Collaborative Filtering,...) Across industries (horizontal adoption) Many industries are just now beginning to look at Big Data technologies Seeq is involved in bringing Big Data to the Industrial/Process sector 9
Controversial Topics Personal information Personal tracking (can be creepy) de-anonymizing (through introspection of data) 10 Security - New technology is often paired with new security challenges Government Tug of war between individual usefulness and government usefulness Can resonate as “Big Brother” Precautions can be abused
But wait…what about the cloud? 11 Many Big Data technologies fit well with the cloud (remote datacenter) paradigm Most technologies aren’t exclusive to the cloud Many new markets (industrial process being one of them) are looking at alternatives Private clouds On premises datacenters IT managed solutions
S EEQ C ORPORATION End
References The Economist – Building with big data, May 26 th Gartner – Economic Intelligence Unit – Leveling the playing field Economist Intelligence Unit Survey, October Forbes 12/12/ and-iia/ 13