1. PACT 2010 Big Data Workloads An Architect’s Perspective Lizy K. John University of Texas at Austin BPOE 2014

2. PACT 2010 The Buzz with Big Data 3/1/2014 BPOE 2014 Lizy K. John

3. PACT 2010 BIG DATA - Seeing things we could not see before 3/1/2014 BPOE 2014 Lizy K. John Analyze massive amounts of data Derive Insights Business Medicine World Economy

4. PACT 2010 3/1/2014 BPOE 2014 An Architect would like to know Lizy K. John  What kind of cores, memory organizations and clustering support needed to support big data  Performance metrics to guide workload partitioning strategies other than use available/affordable nodes  Partitioning considering performance, power, energy  Scaling of computation and communication depending on partitions Becomes important to understand big data workloads

5. PACT 2010 Common Definition “Data that is too large and complex to classify using traditional relational database methods” -Wikipedia 3/1/2014 BPOE 2014 What is “Big Data”? Lizy K. John  1 Terabyte? –Yesterdays “Big Data”  Petabytes? Exabytes? –Today’s “Big Data”  Zettabytes? –Tomorrow’s “Big Data”  What does complex mean??  Need a more complete definition

6. PACT 2010 3/1/2014 BPOE 2014 Some examples Lizy K. John  Combined Space of all hard drives in 2006 –160 exabytes  All hard drives sold by Seagate in 2011 –300 exabytes  The world wide web in 2013 –4 zettabytes  NSA Utah Data Center in Snowden leaks –5 zettabytes (some claimed it to be 1 YB) Exa = 2^60 Zetta = 2^70 Yotta = 2^80

7. PACT 2010 3/1/2014 BPOE 2014 Characteristics of Big Data Lizy K. John * Not always included in taxonomy

8. PACT 2010 3/1/2014 BPOE 2014 Big Data Analytics = I got this in the mail the very same week my son turned 16 Lizy K. John

9. PACT 2010 3/1/2014 BPOE 2014 What’s the Problem? Lizy K. John  Deriving insights from data NOT a new problem –Traditional relational databases that contain carefully pruned and organized data  But storage is relatively cheap these days –Possible to store more data in unstructured form  Need intelligent ways to distill large amounts of data in different formats to actionable KNOWLEDGE  Many different levels to approach this problem…..

10. PACT 2010 3/1/2014 BPOE 2014 Big Data Stack Lizy K. John  Algorithms –PageRank, Genetic Algorithms, SVM, etc.  Frameworks and Implementations –Map/Reduce (Hadoop), MySQL, NoSQL (Cassandra), etc  Hardware –SMT, Accelerator Nodes (Intel Phi, GPU), etc  How does workload analysis fit in? –EVERYONE BENEFITS FROM A DEEP UNDERSTANDING OF A WORKLOAD AND ITS CHARACTERISTICS!

11. PACT 2010 3/1/2014 BPOE 2014 Are New Benchmarks Needed? Lizy K. John  Already have industry standard benchmarks!  Critical Question –Do Big Data workloads have different characteristics than these “traditional” Benchmarks? –Yes they do! TLB Behavior [Wang et al] I-Cache Behavior [Ferdman et al, Zhen et al, Wang et al] SMT [Ferdman et al] Operation Intensity [Wang et al] Data Volume [Wang et al]

12. PACT 2010 3/1/2014 BPOE 2014 Why New Benchmarks? Lizy K. John  I-Cache behavior from Cloudsuite [Ferdman et al] –Much higher miss rate than traditional benchmarks –Significant OS contribution to cache behavior

13. PACT 2010 3/1/2014 BPOE 2014 Why New Benchmarks? Lizy K. John  OS Activity [Zhen et al] –Shows percentage of instructions –Significant variation in kernel/application dynamic instructions

14. PACT 2010 3/1/2014 BPOE 2014 Why New Benchmarks? Lizy K. John  I-TLB Behavior from BigDataBench [Wang et al] –Once again, more misses than traditional benchmarks

15. PACT 2010 3/1/2014 BPOE 2014 Big Data Characterization Challenges Lizy K. John INPUT GENERATION  Input data is critical!  Couple of approaches –Synthetic data generation Questionable Veracity –Grab data from industry Not always possible CAIDA-like  How much data? –Feasibility vs accuracy

16. PACT 2010 3/1/2014 BPOE 2014 Common Big Data Domains Lizy K. John  Databases –Structured Typically relational data SQL databases –Unstructured Example: document oriented Generally no fixed table schema –Semi-structured

17. PACT 2010 3/1/2014 BPOE 2014 Common Big Data Domains Lizy K. John  Common NoSQL Databases –Cassandra Industry leading, ultra scalable –HBase Database built on top of Hadoop and HDFS –MongoDB JSON- database with dynamic schema

18. PACT 2010 3/1/2014 BPOE 2014 Common Big Data Domains Lizy K. John Map/Reduce - Hadoop Key/ Value computation –Map and Reduce phase

19. PACT 2010 3/1/2014 BPOE 2014 Common Big Data Domains Lizy K. John  Graph Algorithms –Important for Data Mining and Machine Learning –Graphlab – essentially Hadoop over large graphs –GraphChi – web scale graph computation –Streaming graph changes –asynchronous changes to the graph (i.e changes written to edges are immediately visible to subsequent computation) –Partitioning Challenges

20. PACT 2010 3/1/2014 BPOE 2014 Hierarchical Decomposition of Workloads Lizy K. John  By dividing into functional blocks - e.g. front end, back end, and database.  By subdividing into tasks, task groups, processes, threads, etc.  By dividing considering hardware modules at microarchitectural level – memory subsystem, CPU, disk, etc. eg: consider AMD APUs  Group together tasks in an application that use data from the same rack.

21. PACT 2010 3/1/2014 BPOE 2014 Entropy Guided Optimizations Lizy K. John  Partitioning Graph Workloads –How do we assign work to nodes?  Important Factors –Data Locality –Minimize Communication –Maximize Resource Utilization  Bisection bandwidth  Entropy Guided Optimization Entropy = (memory-in, memory out, #computations, …other attributes)

22. PACT 2010 3/1/2014 BPOE 2014 In-Memory Map/Reduce Lizy K. John  IBM Main Memory Map Reduce (M3R) –Eliminates intermediate disk writes for Hadoop Map/Reduce Jobs –Pros Significantly speeds up some workloads –45x on sparse matrix mult –Cons Data must fit in cluster memory No failure resilience

23. PACT 2010 3/1/2014 BPOE 2014 Big Data Benchmarking Challenges Lizy K. John WORKLOAD VARIETY  Ton of software stacks required –Configuration of software platform sometimes more important than workload (see next slide)  A comprehensive benchmark should feature –Offline (Batch Style Analytics) –Online (Real Time Analytics)  Seeing positive momentum here!  TPC-* -> Cloudsuite, BigDataBench, etc

24. PACT 2010 3/1/2014 BPOE 2014 Hadoop Case Study – Optimal Settings Lizy K. John  What are the optimal framework settings? –Workload Dependent? –Hardware Dependent? –Just set everything to the maximum value?? –Does it matter?  How do engineers setup clusters for new platforms? –Some “rules of thumb” available, but imprecise

25. PACT 2010 3/1/2014 BPOE 2014 Hadoop Case Study Lizy K. John Standard Hadoop configuration algorithm ):- hadoop_options = Google(“Best Hadoop Configuration”) launch_cluster() if (!cluster_boots || !clients_happy) { hadoop_options = Permute(hadoop_options) launch_cluster() if(!cluster_boots || !clients_happy) { options = Lookup_Options(Buddy_at_Other_Company) launch_cluster() if(!cluster_boots || !clients_happy) { options = default_options launch_cluster() }

26. PACT 2010 3/1/2014 BPOE 2014 Hadoop Case Study (Mapper-Reducer Slots) Lizy K. John 16m4r 2m2r 32m4r 8m8r  CPU Occupancy of TeraSort for different mapper-reducer slots –Simple app, but different very different execution profile depending on configuration 64m4r

27. PACT 2010 3/1/2014 BPOE 2014 Hadoop Case Study (Mapper-Reducer Slots) Lizy K. John  Memory Utilization of TeraSort for different mapper-reducer slots –Simple app, but different very different execution profile depending on configuration 16m4r 2m2r 32m4r 8m8r 64m4r

28. PACT 2010 3/1/2014 BPOE 2014 Hadoop Case Study (Block Size) Lizy K. John  TeraSort – Higher block size reduces total number of maps –Simple app, but different very different execution profile depending on configuration 32MB 64MB 128MB256MB512MB

29. PACT 2010 3/1/2014 BPOE 2014 Big Data Benchmarking Frameworks Lizy K. John  Management frameworks and harnesses essential  Example: AMD SWAT –Software platform for automating the….. creation, deployment, provisioning, execution, and data gathering of synthetic workloads on scalable clusters  Several benchmarks available –Cloudsuite –Hadoop –Graphlab –Anything you want to plugin!

30. PACT 2010 Lizy K. John3/1/2014

31. PACT 2010 3/1/2014 BPOE 2014 Big Data Benchmarking Challenges Lizy K. John Big Cluster  Lots of cores, lots of memory and disk space –Hard for non-industry researchers  Prohibitively long runtimes  Can we simulate Big Data? Requires full system simulation Cloudsuite on Flexus (EPFL)

32. PACT 2010 3/1/2014 BPOE 2014 Adaptable Scalable Futuristic Benchmark Proxies Lizy K. John  Generate Clones by setting knobs to appropriate values  Adaptable  Scalable  Futuristic

33. PACT 2010 3/1/2014 BPOE 2014 Lizy K. John Abstract Workload Model No.MetricCategory 1 Basic block size Control flow predictability 2 Branch taken rate for each branch 3 Branch transition rate 4 Proportion of INT ALU, INT MUL, INT DIV, FP ADD, FP MUL, FP DIV, FP MOV, FP SQRT, LOAD & STORE Instruction mix 5 Dependency distance distribution Instruction level parallelism 6 Private stride value per static load/store Data locality 7 Data Footprint of the workload 8 Mean and standard deviation of the MLP Memory Level Parallelism (MLP) 9 MLP frequency 10 Number of threadsThread level parallelism 11 Thread class and processor assignment Shared data access pattern and communication characteristics 12 Percentage loads to private data 13 Percentage loads to read-only data 14 Percentage migratory loads 15 Percentage consumer loads 16 Percentage irregular loads 17 Percentage stores to private data 18 Percentage producer stores 19 Percentage irregular stores 20 Shared stride value per static load/store 21 Data pool distribution based on sharing patterns 22 Number of lock/unlock pairs and Synchronization Characteristics 23 Number of mutex objects 24 Number of Instructions between lock and unlock

34. PACT 2010 3/1/2014 BPOE 2014 Big Data Synthetics? A Possibility? Lizy K. John  Given challenges in Big Data workloads, this would be useful  But what are the knob settings for “Big Data” –Need detailed characterization

35. PACT 2010 3/1/2014 BPOE 2014 Big Data Workload Clones Lizy K. John CLONES WILL AVOID COMPLEX SOFTWARE STACKS:  Clones for Hadoop  Clones for Graph Processing  Clones for DSS  Clones for OLAP  Clones for DSS with materialized views  Need detailed characterization

36. PACT 2010 3/1/2014 BPOE 2014 Tricks from the Old Treasure Chest Lizy K. John  Search and Sort – –age old computer science problems –new issues raised by scale but  Old OLTP, OLAP and DSS  Combination of HPC and Database Ideas  Old Scatter-Gather  Piece-wise modeling

37. PACT 2010 3/1/2014 BPOE 2014 Conclusion Lizy K. John  Big Data is here to stay  Increasingly important  Cloud and Big Data will take the world in unprecedented ways  Appropriate hardware and software need to be developed  Workload metrics to guide partitioning  Need to act now to develop intelligent benchmarks and workload analysis methodology

38. PACT 2010 3/1/2014 BPOE 2014 Thank You! Questions? Laboratory for Computer Architecture (LCA) The University of Texas at Austin lca.ece.utexas.edu Lizy K. John

39. PACT 2010 [1] M. Ferdman, et. al.. 2012. Clearing the clouds: a study of emerging scale-out workloads on modern hardware.SIGARCH Comput. Archit. News 40, 1 (March 2012), 37-48. [2] Zhen Jia, Lei Wang, Jianfeng Zhan Lixin Zhang, Chunjie Luo. Characterizing Data Analysis Workloads in Data Centers. In Workload Characterization (IISWC), 2013 IEEE International Symposium on. IEEE. [3] Lei Wang, Jianfeng Zhan, Chunjie Luo, Yuqing Zhu, Qiang Yang, Yongqiang He, Wanling Gao, Zhen Jia, Yingjie Shi, Shujie Zhang, Cheng Zhen, Gang Lu, Kent Zhan, Xiaona Li, and Bizhu Qiu. The 20th IEEE International Symposium On High Performance Computer Architecture (HPCA-2014), February 15- 19, 2014, Orlando, Florida, USA. [4] Huang, Shengsheng, et al. "The HiBench benchmark suite: Characterization of the MapReduce-based data analysis." Data Engineering Workshops (ICDEW), 2010 IEEE 26th International Conference on. IEEE, 2010. [5] Cooper, Brian F., et al. "Benchmarking cloud serving systems with YCSB."Proceedings of the 1st ACM symposium on Cloud computing. ACM, 2010. [6] GridMix [Online]. Available: https://hadoop.apache.org/docs/r1.2.1/gridmix.html. (21.10.2013). [7] PigMix [Online]. Available: https://cwiki.apache.org/confluence/display/PIG/PigMix.(21.10.2013).https://cwiki.apache.org/confluence/display/PIG/PigMix.(21.10.2013) [8] PAVLO, A., PAULSON, E., RASIN, A., ABADI, D.J., DEWITT, D.J., MADDEN, S., and STONEBRAKER, M., 2009. A comparison of approaches to large- scale data analysis. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data ACM, 165-178. [9] Transaction Processing Performance Council (Online) http://www.tpc.org/default.asp (02-13-2013)http://www.tpc.org/default.asp [10] GHAZAL, A., RABL, T., HU, M., RAAB, F., POESS, M., CROLOTTE, A., and JACOBSEN, H.-A., 2013. BigBench: Towards an Industry Standard Benchmark for Big Data Analytics. In SIGMOD ACM, New York, New York, 2013, 197-1208. [11] SUMBALY, R., KREPS, J., and SHAH, S., 2013. Linkbench: a database benchmark based on the Facebook social graph In Proceedings of the SIGMOD (New York, New Youk, USA2013), ACM, 1185-1196. [12] Cloudsuite on Flexus[Online]. http://parsa.epfl.ch/cloudsuite/isca12-tutorial.html (02-13-2013). ISCA 2012 Tutorialhttp://parsa.epfl.ch/cloudsuite/isca12-tutorial.html [13] Graphlab [Online]. Available: http://graphlab.com/).http://graphlab.com/) [14] Shinnar, A., Cunningham, D., Saraswat, V., & Herta, B. (2012). M3R: increased performance for in-memory Hadoop jobs. Proceedings of the VLDB Endowment,5(12), 1736-1747. [15] Nambiar, Raghunath Othayoth, and Meikel Poess. "The making of TPC-DS."Proceedings of the 32nd international conference on Very large data bases. VLDB Endowment, 2006. [16] Breternitz, Mauricio, et al. "Cloud Workload Analysis with SWAT." Computer Architecture and High Performance Computing (SBAC-PAD), 2012 IEEE 24th International Symposium on. IEEE, 2012. Lizy K. John3/1/2014 References