Download presentation
Presentation is loading. Please wait.
Published byHaylie Fields Modified over 10 years ago
1
The BigFrame Team Duke University, Hong Kong Polytechnic University, and HP Labs
2
Analytics System Landscape
10
What does this mean for Big Data Practitioners?
11
Gives them a lot of power! From: http://animeonly.org/Digital-Wallpapers/Digital-renders/Spiderman-95061p.html
12
Even the mighty may need a little help
13
Challenges for Practitioners Which system to use for the app that I am developing? Features (e.g., graph data) Performance (e.g., claims like System A is 50x faster than B) Resource efficiency Growth and scalability Multi-tenancy App Developers, Data Scientists
14
Different parts of my app have different requirements Compose best of breed systems OR Use one size fits all system? Managing many systems is hard! System Admins Challenges for Practitioners Which system to use for the app that I am developing? App Developers, Data Scientists
15
Managing many systems is hard! Different parts of my app have different requirements Total Cost of Ownership (TCO)? CIO System Admins Challenges for Practitioners Which system to use for the app that I am developing? App Developers, Data Scientists
18
One Approach
19
Useful, But …
24
How a user uses BigFrame BigFrame Interface BigFrame Interface Benchmark Generator Benchmark Generator HBase Hive Map Reduce Benchmark Driver for System Under Test Benchmark Driver for System Under Test
25
bspec: Benchmark Specification HBase Hive Map Reduce 2. Data refresh pattern 3. Query streams 4. Evaluation metrics 1. Data for initial load
26
What does the user (want to) specify? BigFrame Interface BigFrame Interface
27
The 3 Vs
28
bigif: BigFrames InputFormat Data Variety Relational, text, array, graph Small, medium, large Data Volume Query Volume Query concurrency & classes Data Velocity At rest, slow, fast Micro, Macro Query Variety Exploratory, Continuous Query Velocity
29
Benchmark Generation Benchmark Generator Benchmark Generator
30
Application Domain Modeled Currently E-commerce sales, promotions, recommendations Social media sentiment & influence Social media sentiment & influence
31
Application Domain Modeled Currently Item Customer Web_sales Promotion Tweets Relationships
32
Application Domain Modeled Currently Item Web_sales Promotion
33
Application Domain Modeled Currently
34
Benchmark Generation Benchmark Generator Benchmark Generator
36
Use Case I: Exploratory BI Large volumes of relational data Mostly aggregation and few joins Can Sparks performance match that of an MPP DB?
37
Use Case II: Complex BI Large volumes of relational data Even larger volumes of text data Combined analytics
38
Large volume and velocity of relational and text data Use Case III: Dashboards Continuously-updated Dashboards
39
Use Case IV: Does One Size Fit All? Growing set of applications have to process relational, text, & graph data Compose best of breed systems or use a one size fits all system?
40
Use Case V: Multi-tenancy and SLAs Big data deployments are increasingly multi-tenant and need to meet SLAs
41
Working with the Community First release of BigFrame planned for August 2013 With feedback from benchmark developers (BigBench) Open-source with extensibility APIs Benchmark Drivers for more systems Utilities (accessed through the Benchmark Driver to drill down into system behavior during benchmarking) Instantiate the BigFrame pipeline for more app domains
42
Benchmarks shape a field (for better or worse) … -- David Patterson, Univ. of California, Berkeley Benchmarks meet different needs for different people End customers, application developers, system designers, system administrators, researchers, CIOs BigFrame helps users generate benchmarks that best meet their needs
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.