1. Herodotos Herodotou, Harold Lim, Fei Dong, Shivnath Babu Duke University

2. Analysis in the Big Data Era 9/26/20112 Massive Data Data Analysis Insight Key to Success = Timely and Cost-Effective Analysis Starfish

3. Hadoop MapReduce Ecosystem Popular solution to Big Data Analytics 9/26/20113 MapReduce Execution Engine Distributed File System Hadoop Java / C++ / R / Python OozieHivePig Elastic MapReduce Jaql HBase Starfish

4. Practitioners of Big Data Analytics Who are the users? Data analysts, statisticians, computational scientists… Researchers, developers, testers… You! Who performs setup and tuning? The users! Usually lack expertise to tune the system 9/26/20114Starfish

5. Tuning Challenges Heavy use of programming languages for MapReduce programs (e.g., Java/python) Data loaded/accessed as opaque files Large space of tuning choices Elasticity is wonderful, but hard to achieve (Hadoop has many useful mechanisms, but policies are lacking) Terabyte-scale data cycles 9/26/20115Starfish

6. Our goal: Provide good performance automatically Starfish: Self-tuning System 9/26/20116 MapReduce Execution Engine Distributed File System Hadoop Java / C++ / R / Python OozieHivePig Elastic MapReduce Jaql HBase Starfish Analytics System Starfish

7. What are the Tuning Problems? 9/26/20117 Job-level MapReduce configuration Workload management Data layout tuning Cluster sizing Workflow optimization J1J1 J2J2 J3J3 J4J4 Starfish

8. Starfishs Core Approach to Tuning 9/26/20118 1) if Δ(conf. parameters) then what …? 2) if Δ(data properties) then what …? 3) if Δ(cluster properties) then what …? Profiler Collects concise summaries of execution What-if Engine Estimates impact of hypothetical changes on execution Optimizers Search through space of tuning choices Job Workflow Workload Data layout Cluster Starfish

9. Starfish Architecture 9/26/20119 Profiler What-if Engine Workflow Optimizer Workload OptimizerElastisizer Job Optimizer Data Manager Metadata Mgr. Intermediate Data Mgr. Data Layout & Storage Mgr. Starfish

10. MapReduce Job Execution 9/26/201110 split 0 map out 0 reduce Two Map WavesOne Reduce Wave split 2 map split 1 map split 3 map Out 1 reduce job j = Starfish

11. What Controls MR Job Execution? Space of configuration choices: Number of map tasks Number of reduce tasks Partitioning of map outputs to reduce tasks Memory allocation to task-level buffers Multiphase external sorting in the tasks Whether output data from tasks should be compressed Whether combine function should be used 9/26/201111 job j = Starfish

12. Effect of Configuration Settings Use defaults or set manually (rules-of-thumb) Rules-of-thumb may not suffice 9/26/201112 Two-dimensional projection of a multi- dimensional surface (Word Co-occurrence MapReduce Program) Rules-of-thumb settings Starfish

13. MapReduce Job Tuning in a Nutshell Goal: Challenges: p is an arbitrary MapReduce program; c is high-dimensional; … 9/26/201113 Profiler What-if Engine Optimizer Runs p to collect a job profile (concise execution summary) of Given profile of, estimates virtual profile for Enumerates and searches through the optimization space S efficiently Starfish

14. Job Profile Concise representation of program execution as a job Records information at the level of task phases Generated by Profiler through measurement or by the What-if Engine through estimation 9/26/201114 Memory Buffer Merge Sort, [Combine], [Compress] Serialize, Partition map Merge split DFS SpillCollectMapRead Starfish

15. Job Profile Fields Dataflow: amount of data flowing through task phases Map output bytes Number of spills Number of records in buffer per spill 9/26/201115 Costs: execution times at the level of task phases Read phase time in the map task Map phase time in the map task Spill phase time in the map task Dataflow Statistics: statistical information about dataflow Width of input key-value pairs Map selectivity in terms of records Map output compression ratio Cost Statistics: statistical information about resource costs I/O cost for reading from local disk per byte CPU cost for executing the Mapper per record CPU cost for uncompressing the input per byte Starfish

16. Generating Profiles by Measurement Goals Have zero overhead when profiling is turned off Require no modifications to Hadoop Support unmodified MapReduce programs written in Java or Hadoop Streaming/Pipes (Python/Ruby/C++) Approach: Dynamic (on-demand) instrumentation Event-condition-action rules are specified (in Java) Leads to run-time instrumentation of Hadoop internals Monitors task phases of MapReduce job execution We currently use Btrace (Hadoop internals are in Java) 9/26/201116Starfish

17. Generating Profiles by Measurement 9/26/201117 split 0 map out 0 reduce split 1 map raw data map profile reduce profile job profile Use of Sampling Profile fewer tasks Execute fewer tasks JVM = Java Virtual Machine, ECA = Event-Condition-Action JVM Enable Profiling ECA rules Starfish

18. What-if Engine Job Oracle Virtual Job Profile for What-if Engine 9/26/201118 Task Scheduler Simulator Job Profile Properties of Hypothetical job Input Data Properties Cluster Resources Configuration Settings Possibly Hypothetical Starfish

19. Virtual Profile Estimation 9/26/201119 Given profile for job j = estimate profile for job j' = (Virtual) Profile for j' Dataflow Statistics Dataflow Cost Statistics Costs Profile for j Input Data d 2 Confi- guration c 2 Resources r 2 Costs White-box Models Cost Statistics Relative Black-box Models Dataflow White-box Models Dataflow Statistics Cardinality Models Starfish

20. Job Optimizer 9/26/201120 Best Configuration Settings for Subspace Enumeration Recursive Random Search Just-in-Time Optimizer Job Profile Input Data Properties Cluster Resources What-if calls Starfish

21. Workflow Optimization Space 9/26/201121 Job-level Configuration Dataset-level Configuration Physical Optimization Space Logical Join Selection Partition Function Selection Vertical Packing Inter-job Starfish

22. Optimizations on TF-IDF Workflow 9/26/201122 Logical Optimization M1 R1 … D0 J1 D1 D2 D4 M2 R2 … J2 … J3, J4 … Partition:{D} Sort: {D,W} M1 R1 M2 R2 … D0 J1, J2 D2 D4 M3 R3 M4 … J3, J4 … Physical Optimization Reducers= 50 Compress = off Memory = 400 … … … … Reducers= 20 Compress = on Memory = 300 … Legend D = docname f = frequency W = word c = count t = TF-IDF M3 R3 M4 Starfish

23. New Challenges What-if challenges: Support concurrent job execution Estimate intermediate data properties Optimization challenges Interactions across jobs Extended optimization space Find good configuration settings for individual jobs 9/26/201123 J1J2 J3 J4 Workflow Starfish

24. Cluster Sizing Problem Use-cases for cluster sizing Tuning the cluster size for elastic workloads Workload transitioning from development cluster to production cluster Multi-objective cluster provisioning Goal Determine cluster resources & job-level configuration parameters to meet workload requirements 9/26/201124Starfish

25. Multi-objective Cluster Provisioning 9/26/201125 Cloud enables users to provision clusters in minutes Starfish

26. Experimental Evaluation 9/26/201126 Starfish (versions 0.1, 0.2) to manage Hadoop on EC2 Different scenarios: Cluster × Workload × Data EC2 Node Type CPU: EC2 units MemI/O Perf.Cost /hour #Maps /node #Reds /node MaxMem /task m1.small1 (1 x 1)1.7 GBmoderate$0.08521300 MB m1.large4 (2 x 2)7.5 GBhigh$0.34321024 MB m1.xlarge8 (4 x 2)15 GBhigh$0.68441536 MB c1.medium5 (2 x 2.5)1.7 GBmoderate$0.1722300 MB c1.xlarge20 (8 x 2.5)7 GBhigh$0.6886400 MB cc1.4xlarge33.5 (8)23 GBvery high$1.60861536 MB Starfish

27. Experimental Evaluation 9/26/201127 Starfish (versions 0.1, 0.2) to manage Hadoop on EC2 Different scenarios: Cluster × Workload × Data Abbr.MapReduce ProgramDomainDataset COWord Co-occurrenceNatural Lang Proc.Wikipedia (10GB – 22GB) WCWordCountText AnalyticsWikipedia (30GB – 1TB) TSTeraSortBusiness AnalyticsTeraGen (30GB – 1TB) LGLinkGraphGraph ProcessingWikipedia (compressed ~6x) JOJoinBusiness AnalyticsTPC-H (30GB – 1TB) TFTerm Freq. - Inverse Document Freq. Information RetrievalWikipedia (30GB – 1TB) Starfish

28. Job Optimizer Evaluation 9/26/201128 Hadoop cluster: 30 nodes, m1.xlarge Data sizes: 60-180 GB Starfish

29. Estimates from the What-if Engine 9/26/201129 Hadoop cluster: 16 nodes, c1.medium MapReduce Program: Word Co-occurrence Data set: 10 GB Wikipedia True surfaceEstimated surface Starfish

30. Profiling Overhead Vs. Benefit 9/26/201130 Hadoop cluster: 16 nodes, c1.medium MapReduce Program: Word Co-occurrence Data set: 10 GB Wikipedia Starfish

31. Multi-objective Cluster Provisioning 9/26/201131 Instance Type for Source Cluster: m1.large Starfish

32. More info: www.cs.duke.edu/starfish 9/26/201132 Job-level MapReduce configuration Workflow optimization Workload management Data layout tuning Cluster sizing J1J1 J2J2 J3J3 J4J4 Starfish