0. MapReduce-Simulator: Matchmaking and Scheduling Algorithm
SYSC 5807: Resource Management on Distributed Systems
MapReduce-Simulator: Matchmaking and Scheduling Algorithm
Norman Lim
Dept. of Systems and Computer Engineering
Carleton University
Ottawa, ON, Canada

1. Overview of Matchmaking and Scheduling (Mapping) Algorithm
1. When a job j arrives, it is placed in a queue depending on the scheduling policy selected.
FIFO: uses first-in first-out queue
EDF: uses earliest deadline first queue
2. Job/Task Mapping algorithm is invoked.
Retrieves the first job in the queue and maps each of the job’s tasks onto the resources.
Maps each task t onto a resource r that can execute t at its earliest possible time (at or after its earliest start time).
For a particular job, the tasks in that job with higher execution time are mapped first.

2. Overview of Matchmaking and Scheduling (Mapping) Algorithm Cont.
EDF Policy Only
3. If a job j is not able to be scheduled to complete executing before its deadline, then the Job/Task Re-mapping algorithm is invoked.
Remaps j and the set of jobs (called S) that have caused j to miss its deadline.
S includes jobs that are scheduled to start at, or complete within the interval: [start time of job j, deadline of job j].
j and the jobs in S are re-ordered according to earliest deadline first.
The Job/Task Mapping algorithm (see previous slide) is invoked to map j and the jobs in S.

3. Example: Jobs and Resources
Job 1 (j1)
Arrival time = 0s
EST = 2s
Deadline = 30s t1 2 t2 5 t3 8 t4 10 t5 11
Job 2 (j2)
Arrival time = 1s
EST = 1s
Deadline = 20s t1 4 t2 5 t3 10
Resource 1 (r1)
Map Capacity = 1
Reduce Capacity = 1
Resource 2 (r2)
Map Capacity = 1
Reduce Capacity = 1
Two jobs: j1 and j2. DAG shows the jobs’ tasks.
- Blue circles: Map tasks
- Red circles: Reduce tasks
The task id and its required execution time are shown in the
Two resources: r1 and r2.
EST = earliest start time

4. Example: FIFO Job 1 (j1) Arrival time = 0s EST = 2s Deadline = 30s
FIFO Queue
map
j1, t2
j1, t1
j2, t2
j1, t4
j2, t3
reduce
j2 will miss its deadline.
Job 1 arrives
Job 2 arrives
Job 1 gets scheduled
Job 2 gets scheduled.
map
j1, t3
j2, t1
j1, t5
reduce

5. Example: EDF (Case 1) Job 1 (j1) Arrival time = 0s EST = 2s
Deadline = 30s
Job 2 (j2)
Arrival time = 1s
EST = 1s
Deadline = 20s
EDF Queue
map
j2, t1
j1, t3
j1, t5
reduce
Job 1 arrives.
Job 2 arrives, and goes to the start of the queue.
Job 1 and 2 are scheduled.
Note: In this case, Job 1 is not scheduled immediately because the scheduler was busy.
map
j2, t2
j1, t2
j1, t1
j2, t3
j1, t4
reduce

6. Example: EDF (Case 2) Job 2 (j2) Arrival time = 1s EST = 1s
Deadline = 20s
Job 1 (j1)
Arrival time = 0s
EST = 2s
Deadline = 30s
Job 1 (j1)
Arrival time = 0s
EST = 2s
Deadline = 30s
Job 2 (j2)
Arrival time = 1s
EST = 1s
Deadline = 20s
EDF Queue
map
j2, t1
j1, t2
j1, t3
j1, t1
j2, t2
j1, t5
j1, t4
j2, t3
reduce
j2 will miss its deadline.
Job 1 arrives, and gets scheduled.
Note: In this case, Job 1 is scheduled immediately because the scheduler was not busy.
Job 2 arrives, and gets scheduled.
Job 2 is not able to finish its execution before its deadline so the Job/Task Re-mapping algorithm is executed to remap the jobs j1 and j2.
This time, Job 2 (earlier deadline) is scheduled first, followed by Job 1.
map
j2, t2
j1, t3
j1, t2
j2, t1
j1, t1
j2, t3
j1, t5
j1, t4
reduce