Download presentation
Presentation is loading. Please wait.
Published byEdmund Boyd Modified over 9 years ago
1
Analytics from 330 million smartphones Sean Byrnes CTO & Co-founder
2
Flurry Overview 60,000 160,000 App Developers: Live Applications: Flurry Analytics Better apps on iOS, Android, BB, WP, HTML5 480M Devices per month: 33B Sessions per month: AppCircle Network Acquisition & Monetization: iOS, Android 6,200 App Developers: 200M Devices per month: 300B Events per month: 3M Daily Completed Views
3
How Flurry Works
4
Flurry’s Scale 1.2 Billions Sessions / Day 900 Servers 1.56 PB
5
Topics 1. Big Data Collection (HDFS) 2. Big Data Processing (Hadoop) 3. Data Mining at Scale (Hbase)
6
BIG DATA COLLECTION
7
Incoming Data Peak Connections per Second: 25,000 Data per day: 1.5 TB
8
Data Collection Reports Load Balancer Data Collector Load Balancer Data Collector Load Balancer Data Collector File HDFS
9
Data Collection Reports HDFS Location A Location B
10
BIG DATA PROCESSING
11
11 Normalization Data Correction Metrics Computation Agent Report De-duplication Portfolio Analysis Benchmarking Clustering Identify Device, Country, Carrier, etc. Bad Phone Clocks Partial Session Reports Handle duplicate reports Flexible calculation Configurable Dimensions Data mining and analysis Audience Segmentation Industry TrendsApplication Analytics Merchandising Analytics Analytics Processing
12
Large-scale Data Processing Input Data NoSQL DataStore Real-Time Batch Collectors Consumer/ Producer Systems MapReduce (jobs) External Action
13
Map/Reduce Management Challenge: Task Starvation Challenge: Task Roadblocking Challenge: Network Connection Waiting
14
Network Topology: Chained Rack 1 Rack 2 Switch 1 Switch 2 Rack 3 Switch 3
15
Network Topology: Star Rack 3 Rack 2 Switch 3 Switch 4 Switch 1 Switch 2 Trunk Rack 1 Rack 2
16
DATA MINING AT SCALE
17
Stages of Data Normalized OLAP Cube Raw Data 80 Billion Rows 160 Billion Rows 500 Billion Records
18
NoSQL Tables 111111111 Data Index Column Family A Column Family B 222222222 Data 333333333 Data
19
NoSQL OLAP metric.dimension Index Column Family A # metric.dimensionA metric.dimensionB metric.dimensionC metric.dimensionA.dimensionB.dimensionC metric.dimensionA.dimensionB metric.dimensionA.dimensionC...
20
Lexicographical Ordering metricdimensionAdimensionBindex 311311 31113111 31113111 metric.dimensionA.dimensionB
21
Lexicographical Ordering metricdimensionAdimensionBindex 3001 3001001 30010113001011 30110013011001 metric.dimensionA.dimensionB
22
NoSQL OLAP metric.dimension.date metric.dimension.1_1_12 metric.dimension.3_1_12 Index Row Scan metric 1/1/12 3/1/12
23
blog.flurry.com
24
Sean Byrnes sean@flurry.com Flurry, Inc. 282 2 nd St. Suite 202 San Francisco, CA 94105 http://www.flurry.com
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.