1. MapReduce

2. Outline MapReduce overview
Note: These notes are based on notes provided by Google

3. What is a Cloud?
Cloud = Lots of storage + compute cycles nearby

4. Data-Intensive Computing
Typically store data at datacenters
Use compute nodes nearby
Compute nodes run computation services
In data-intensive computing, the focus is on the data: problem areas include
Storage
Communication bottleneck
Moving tasks to data (rather than vice-versa)
Security
Availability of Data
Scalability

5. Computation Services Google → MapReduce, Sawzall
Yahoo → Hadoop, Pig Latin
Microsoft → Dryad, DryadLINQ

6. Motivation: Large Scale Data Processing
Want to process lots of data ( > 1 TB)
Want to parallelize across hundreds/thousands of CPUs
How to parallelize
How to distribute
How to handle failures
Want to make this easy

7. What is MapReduce?
MapReduce is an abstraction that allows programmers to specify computations that can be done in parallel
MapReduce hides the messy details needed to support the computations e.g.,
Distribution and synchronization
Machine failures
Data distribution
Load balancing
This is widely used at Google

8. Programming Model
MapReduce simplifies programming through its library.
The user of the MapReduce library expresses the computation as two functions: Map, Reduce

9. Programming Model
Map
Takes an input pair and produces a set of intermediate key/value pairs e.g.,
Map: (key1, value1)  list(key2,value2)
The MapReduce library groups together all intermediate values associated with the same intermediate key
Reduce
This function accepts an intermediate key and a set of values for that key
Reduce: (key2,list(key2,value2))  value3

10. Example: Word Frequencies in Web Pages
Determine the count of each word that appears in a document (or a set of documents)
Each file is associated with a document URL
Map function
Key = document URL
Value = document contents
Output of map function is (potentially many) key/value pairs
Output (word, “1”) once per word in the document

11. Example: Word Frequencies in Web Pages
Pseudo code for map
Map(String key, String value):
// input_key: document name
// input_value: document contents
for each word w in value:
EmitIntermediate(w, "1");

12. Example: Word Frequencies in Web Pages
Example key, value pair:
“document_example”, “to be or not to be”
Result of applying the map function
“to”, 1
“be”, 1
“or”, 1
“not”, 1

13. Example: Word Frequencies in Web Pages
Pseudo-code for Reduce
Reduce(String key, values):
// key: a word, same for input and output
// values: a list of counts
int result = 0;
for each v in values:
result = result + value;
Emit(result);
The function sums together all counts emitted for a particular word

14. Example: Word Frequencies in Web Pages
The MapReduce framework sorts all pairs with the same key
(be,1), (be,1), (not,1), (or, 1), (to, 1), (to,1)
The pairs are then grouped
(be, 1,1), (not, 1), (or, 1), (to, 1, 1)
The reduce function combines (sums) the values for a key
Example: Applying reduce to (be, 1, 1) = 2

15. Example: Distributed Grep
Find all occurrences of a given pattern in a a file (or set of files)
Input consists of (url+offset, line)
map(key=url+offset, val=line):
If contents match specified pattern, emit (line, “1)
reduce(key=line, values=uniq_counts):
Example of input to reduce is essentially (line, [1,1,1,1])
Don’t do anything; just emit line

16. Example: Count of URL Access Frequency
Map function
Input: <log of web page requests, content of log>
Outputs: <URL, 1>
Reduce function adds together all values for the same URL

17. Example:Web structure
Simple representation of WWW link graph
Map
Input: (URL, page-contents)
Output: (URL, list-of-URLs)
Who maps to me?
Input: (URL, list-of-URLS)
Output: For each u in list-of-URLS output <u,URL>
Reduce: Concatenates the list of all source URLs associated with u and emits (<u, list(URL))

18. The Infrastructure
Large clusters of commodity PCs and networking hardware
Clusters consists of 100/1000s of machines (failures are common)
GFS (Google File System).
Distributed file system.
Provides replication of the data.

19. The Infrastructure Users submit jobs to a scheduling system
Possible partitions of data can be based on files, databases, file lines, database records etc;

20. Execution
Map invocations are distributed across multiple machines by automatically partitioning the input data into a set of M splits.
The input splits can be processed in parallel by different machines
Reduce invocations are distributed by partitioning the intermediate key space into R pieces using a hash function: hash(key) mod R
R and the partitioning function are specified by the programmer.

21. Execution Workers are assigned work by the master
The master is started by the MapReduce Framework

22. Execution
Workers assigned map tasks read the input, parse it and invoke
the user’s Map() method.

23. Execution Intermediate key/value pairs are buffered in memory
Periodically, buffered data is written to local disk (R files)
Pseudo random partitioning function (e.g., (hash(k) mod R)

24. Execution Locations are passed back to the master who forwards these
locations to workers executing the reduce function.

25. Execution Reduce runs after all mappers are done
Workers executing Reduce are notified by the master about
location of intermediate data

26. Execution
Reduce workers use remote procedure calls to read the data from
local disks of map works
Sorts all intermediate data by intermediate key

27. Execution
Reduce worker iterates over the sorted intermediate data and for
each key encountered it passes the key and the corresponding set
of intermediate values to the Reduce function

28. Execution
The output of the Reduce function is appended to a final output
file

29. Data flow Input, final output are stored on a distributed file system
Scheduler tries to schedule map tasks “close” to physical storage location of input data
Intermediate results are stored on local file system of map and reduce workers
Output can be input to another map reduce task

30. Execution

31. Parallel Execution

32. Coordination Master data structures
Task status: (idle, in-progress, completed)
Idle tasks get scheduled as workers become available
When a map task completes, it sends the master the location and sizes of its R intermediate files, one for each reducer
Master pushes this info to reducers
Master pings workers periodically to detect failures

33. Failures Map worker failure Reduce worker failure Master failure
Map tasks completed or in-progress at worker are reset to idle
Reduce workers are notified when task is rescheduled on another worker
Reduce worker failure
Only in-progress tasks are reset to idle
Master failure
MapReduce task is aborted and client is notified

34. Locality
MapReduce master takes the location information of input files into account and attempts to schedule a map task on a machine that contains a replica of the corresponding input data
Schedule a map task near a replica of that task’s input data
The goal is to read most input data locally and thus reduce the consumption of network bandwidth

35. Task Granularity
M and R should be much larger than the number of available machines.
Dynamic load balancing.
Speeds up recovery in case of failures.
R determines the number of output files
Often constrained by users.

36. Backup Tasks Stragglers - A common reason for long computations.
Schedule backups for remaining jobs (in progress jobs) when map or reduce phases near completion.
Slightly increases needed computational resources.
Does not increase running time, but has the potential to improve it significantly.

37. Combiners
Often a map task will produce many pairs of the form (k,v1), (k,v2), … for the same key k
E.g., popular words in Word Count
Can save network time by pre-aggregating at mapper
combine(k1, list(v1))  v2
Usually same as reduce function
Works only if reduce function is commutative and associative

38. Partition Function
Inputs to map tasks are created by contiguous splits of input file
For reduce, we need to ensure that records with the same intermediate key end up at the same worker
System uses a default partition function e.g., hash(key) mod R
Sometimes useful to override; What if all output keys are URLS and we want all entries for a single host to end up in the same output file?
Use hash(hostname(URL)) mod R ensures URLs from a host end up in the same output file

39. Summary MapReduce – a framework for distributed computing.
Distributed programs are easy to write and understand.
Provides fault tolerance
Program execution can be easily monitored.
It works for Google!!