1. Berkeley Data Analytics Stack Prof. Chi (Harold) Liu November 2015

2. Data Processing Goals Low latency (interactive) queries on historical data: enable faster decisions –e.g., identify why a site is slow and fix it Low latency queries on live data (streaming): enable decisions on real-time data –e.g., detect & block worms in real-time (a worm may infect 1mil hosts in 1.3sec) Sophisticated data processing: enable “better” decisions –e.g., anomaly detection, trend analysis

3. Today’s Open Analytics Stack…..mostly focused on large on-disk datasets: great for batch but slow Application Storage Data Processing Infrastructure

4. Goals Batch Interactive Streaming One stack to rule them all!  Easy to combine batch, streaming, and interactive computations  Easy to develop sophisticated algorithms  Compatible with existing open source ecosystem (Hadoop/HDFS)

5. Support Interactive and Streaming Comp. Aggressive use of memory Why? 1. Memory transfer rates >> disk or SSDs 2. Many datasets already fit into memory Inputs of over 90% of jobs in Facebook, Yahoo!, and Bing clusters fit into memory e.g., 1TB = 1 billion records @ 1KB each 3. Memory density (still) grows with Moore’s law RAM/SSD hybrid memories at horizon High end datacenter node 16 cores 10-30TB 128- 512GB 1-4TB 10Gbps 0.2- 1GB/s (x10 disks) 1- 4GB/s (x4 disks) 40-60GB/s

6. Support Interactive and Streaming Comp. Increase parallelism Why? –Reduce work per node  improve latency Techniques: –Low latency parallel scheduler that achieve high locality –Optimized parallel communication patterns (e.g., shuffle, broadcast) –Efficient recovery from failures and straggler mitigation result T T new (< T)

7. Trade between result accuracy and response times Why? –In-memory processing does not guarantee interactive query processing e.g., ~10’s sec just to scan 512 GB RAM! Gap between memory capacity and transfer rate increasing Challenges: –accurately estimate error and running time for… –… arbitrary computations 128- 512GB 16 cores 40-60GB/s doubles every 18 months doubles every 36 months Support Interactive and Streaming Comp.

8. Berkeley Data Analytics Stack (BDAS) Infrastructure Storage Data Processing Application Resource Management Data Management Share infrastructure across frameworks (multi-programming for datacenters) Efficient data sharing across frameworks Data Processing in-memory processing trade between time, quality, and cost Application New apps: AMP-Genomics, Carat, …

9. Berkeley AMPLab  “Launched” January 2011: 6 Year Plan –8 CS Faculty –~40 students –3 software engineers Organized for collaboration:

10. Berkeley Funding: – XData, CISE Expedition Grant –Industrial, founding sponsors –18 other sponsors, including Goal: Next Generation of Analytics Data Stack for Industry & Research: Berkeley Data Analytics Stack (BDAS) Release as Open Source

11. Berkeley Data Analytics Stack (BDAS) Existing stack components…. HDFS MPI … Resource Mgmnt. Data Mgmnt. Data Processing Hadoop HIVEPig HBaseStorm Data Management Data Processing Resource Management

12. Mesos Management platform that allows multiple framework to share cluster Compatible with existing open analytics stack Deployed in production at Twitter on 3,500+ servers HDFS MPI … Resource Mgmnt. Data Mgmnt. Data Processing Hadoop HIVEPig HBaseStorm Mesos

13. Spark In-memory framework for interactive and iterative computations –Resilient Distributed Dataset (RDD): fault-tolerance, in- memory storage abstraction Scala interface, Java and Python APIs HDFS Mesos MPI Resource Mgmnt. Data Mgmnt. Data Processing Storm … Spark Hadoop HIVE Pig

14. Spark Streaming [Alpha Release] Large scale streaming computation Ensure exactly one semantics Integrated with Spark  unifies batch, interactive, and streaming computations! HDFS Mesos MP I Resource Mgmnt. Data Mgmnt. Data Processing Hadoop HIVE Pig Stor m … Spark Streamin g

15. Shark  Spark SQL HIVE over Spark: SQL-like interface (supports Hive 0.9) –up to 100x faster for in-memory data, and 5-10x for disk In tests on hundreds node cluster at HDFS Mesos MP I Resource Mgmnt. Data Mgmnt. Data Processing Stor m … Spark Streamin g Shark Hadoop HIVE Pig

16. Tachyon High-throughput, fault-tolerant in-memory storage Interface compatible to HDFS Support for Spark and Hadoop HDFS Mesos MP I Resource Mgmnt. Data Mgmnt. Data Processing Hadoop HIVE Pig Stor m … Spark Streamin g SharkTachyon

17. BlinkDB Large scale approximate query engine Allow users to specify error or time bounds Preliminary prototype starting being tested at Facebook Mesos MP I Resource Mgmnt. Data Processing Stor m … Spark Streamin g SharkBlinkDB HDFS Data Mgmnt. Tachyon Hadoop Pig HIVE

18. SparkGraph GraphLab API and Toolkits on top of Spark Fault tolerance by leveraging Spark Mesos MP I Resource Mgmnt. Data Processing Stor m … Spark Streamin g SharkBlinkDB HDFS Data Mgmnt. Tachyon HadoopHIVE Pig Spark Graph

19. MLlib Declarative approach to ML Develop scalable ML algorithms Make ML accessible to non-experts Mesos MP I Resource Mgmnt. Data Processing Stor m … Spark Streamin g SharkBlinkDB HDFS Data Mgmnt. Tachyon HadoopHIVE Pig Spark Graph MLbas e

20. Compatible with Open Source Ecosystem Support existing interfaces whenever possible Mesos MP I Resource Mgmnt. Data Processing Stor m … Spark Streamin g Shark BlinkDB HDFS Data Mgmnt. Tachyon Hadoop HIVE Pig Spark Graph MLbas e GraphLab API Hive Interface and Shell HDFS API Compatibility layer for Hadoop, Storm, MPI, etc to run over Mesos

21. Compatible with Open Source Ecosystem Use existing interfaces whenever possible Mesos MP I Resource Mgmnt. Data Processing Stor m … Spark Streamin g Shark BlinkDB HDFS Data Mgmnt. Tachyon Hadoop HIVE Pig Spark Graph MLbas e Support HDFS API, S3 API, and Hive metadata Support Hive API Accept inputs from Kafka, Flume, Twitter, TCP Sockets, …

22. Summary Support interactive and streaming computations –In-memory, fault-tolerant storage abstraction, low-latency scheduling,... Easy to combine batch, streaming, and interactive computations –Spark execution engine supports all comp. models Easy to develop sophisticated algorithms –Scala interface, APIs for Java, Python, Hive QL, … –New frameworks targeted to graph based and ML algorithms Compatible with existing open source ecosystem Open source (Apache/BSD) and fully committed to release high quality software –Three-person software engineering team lead by Matt Massie (creator of Ganglia, 5 th Cloudera engineer) Batch Interacti ve Streami ng Spark

23. In-Memory Cluster Computing for Iterative and Interactive Applications UC Berkeley

24. Background Commodity clusters have become an important computing platform for a variety of applications –In industry: search, machine translation, ad targeting, … –In research: bioinformatics, NLP, climate simulation, … High-level cluster programming models like MapReduce power many of these apps Theme of this work: provide similarly powerful abstractions for a broader class of applications

25. Motivation Current popular programming models for clusters transform data flowing from stable storage to stable storage e.g., MapReduce: Map Reduce InputOutput

26. Motivation Acyclic data flow is a powerful abstraction, but is not efficient for applications that repeatedly reuse a working set of data: –Iterative algorithms (many in machine learning) –Interactive data mining tools (R, Excel, Python) Spark makes working sets a first-class concept to efficiently support these apps

27. Spark Goal Provide distributed memory abstractions for clusters to support apps with working sets Retain the attractive properties of MapReduce: –Fault tolerance (for crashes & stragglers) –Data locality –Scalability Solution: augment data flow model with “resilient distributed datasets” (RDDs)

28. Example: Log Mining Load error messages from a log into memory, then interactively search for various patterns lines = spark.textFile(“hdfs://...”) errors = lines.filter(_.startsWith(“ERROR”)) messages = errors.map(_.split(‘\t’)(2)) cachedMsgs = messages.cache() Block 1 Block 2 Block 3 Worke r Driver cachedMsgs.filter(_.contains(“foo”)).count cachedMsgs.filter(_.contains(“bar”)).count... tasks results Cache 1 Cache 2 Cache 3 Base RDD Transformed RDD Cached RDD Parallel operation Result: full-text search of Wikipedia in <1 sec (vs 20 sec for on-disk data)

29. Spark Components

31. Programming Model by RDD Resilient distributed datasets (RDDs) –Immutable collections partitioned across cluster that can be rebuilt if a partition is lost –Created by transforming data in stable storage using data flow operators (map, filter, group-by, …) –Can be cached across parallel operations Parallel operations on RDDs –Reduce, collect, count, save, …

32. RDDs in More Detail An RDD is an immutable, partitioned, logical collection of records –Need not be materialized, but rather contains information to rebuild a dataset from stable storage Partitioning can be based on a key in each record (using hash or range partitioning) Built using bulk transformations on other RDDs Can be cached for future reuse

34. RDD Operations Transformations (define a new RDD) map filter sample union groupByKey reduceByKey join cache … Parallel operations (Actions) (return a result to driver) reduce collect count save lookupKey …

40. RDD Fault Tolerance RDDs maintain lineage information that can be used to reconstruct lost partitions e.g.: cachedMsgs = textFile(...).filter(_.contains(“error”)).map(_.split(‘\t’)(2)).cache() HdfsRDD path: hdfs://… HdfsRDD path: hdfs://… FilteredRDD func: contains(...) FilteredRDD func: contains(...) MappedRDD func: split(…) MappedRDD func: split(…) CachedRDD

41. Example 1: Logistic Regression Goal: find best line separating two sets of points + – + + + + + + + + – – – – – – – – + target – random initial line

42. Logistic Regression Code val data = spark.textFile(...).map(readPoint).cache() var w = Vector.random(D) for (i <- 1 to ITERATIONS) { val gradient = data.map(p => (1 / (1 + exp(-p.y*(w dot p.x))) - 1) * p.y * p.x ).reduce(_ + _) w -= gradient } println("Final w: " + w)

43. Logistic Regression Performance 127 s / iteration first iteration 174 s further iterations 6 s

44. Example 2: MapReduce MapReduce data flow can be expressed using RDD transformations res = data.flatMap(rec => myMapFunc(rec)).groupByKey().map((key, vals) => myReduceFunc(key, vals)) Or with combiners: res = data.flatMap(rec => myMapFunc(rec)).reduceByKey(myCombiner).map((key, val) => myReduceFunc(key, val))

45. Example 3

48. RDD Graph

49. RDD Dependency Types RDD Partition

50. Scheduling

51. Scheduler Optimization

52. Event Flow (Direct Acycle Graph)

53. Conclusion