1. Parallel and Distributed Simulation Deadlock Detection & Recovery: Performance Barrier Mechanisms

2. Outline Deadlock Detection and Recovery Algorithm –Empirical performance measurements Synchronous Algorithms –Barrier mechanisms –Centralized Barriers –Tree Barrier –Butterfly Barrier

3. Performance T = arrival time of job Q = waiting time in queue S = service time Example: Tandem first-come-first-serve queues “Classical” approach: lookahead? LP 1 LP 2 arrival event departure event arrival event T T+Q T+Q+S begin service Optimized to exploit lookahead LP 1 LP 2 arrival event arrival event T T+Q T+Q+S Maintain variable indicating departure time of previous job

4. Efficiency of Queueing Network Simulation Parallel Simulation of a Central Server Queueing Network Deadlock Detection and Recovery Algorithm (5 processors) mergefork

5. Speedup of Queueing Network Simulation Deadlock Detection and Recovery Algorithm (5 processors) Exploiting lookahead is essential to obtain good performance

6. Synchronous Execution Basic idea: each process cycles through the following steps: Determine the events that are safe to process Process events, exchange messages Global synchronization (barrier) Messages generated in one cycle are not eligible for processing until the next cycle Issues Barrier mechanism, transient messages Determining safe events

7. Barrier Synchronization Barrier Synchronization: when a process invokes the barrier primitive, it will block until all other processors have also invoked the barrier primitive. When the last process invokes the barrier, all processes can execute forward - barrier - wait - barrier - process 1process 2process 3process 4 wallclock time

8. Barrier Implementation Centralized Message-Passing Approach Central controller used to implement barrier 2 step process –Determine when barrier reached –Broadcast message to release processes from the barrier Barrier primitive for non-controller processes: –Send a message to central controller –Wait for a reply Barrier primitive for controller process –Receive barrier messages from other processes –When a message is received from each process, broadcast message to release barrier Performance –Controller must send and receive N-1 messages –Potential bottleneck

9. Broadcast Barrier 1 step approach Each process broadcasts message when it reaches barrier Wait until a message is received from each other process N (N-1) messages 0123 0123

10. Tree Barrier Organize processes into a tree A process sends a message to its parent process when –The process has reached the barrier point, and –A message has been received from each of its children processes Root detects completion of barrier, broadcast message to release processes (e.g., send messages down tree) 2 log N time if all processes reach barrier at same time 3 87 4 109 5 1211 6 13 12 0

11. Butterfly Barrier N processes (here, assume N is a power of 2) Sequence of log 2 N pairwise barriers (let k = log 2 N) Pairwise barrier: –Send message to partner process –Wait until message is received from that process Process p: b k b k-1 … b 1 = binary representation of p Step i: perform barrier with process b k … b i ’ … b 1 (complement ith bit of the binary representation) Example: Process 3 (011) –Step 1: pairwise barrier with process 2 (010) –Step 2: pairwise barrier with process 1 (001) –Step 3: pairwise barrier with process 7 (111)

12. Butterfly Barrier Example 0 1 2 3 4 5 6 7 01234567 012 3 4567 012 3 456 7 step 1 step 2 step 3 Wallclock time 01234567 0,12,34,56,7 0-34-7 0-7 step 1 step 2 step 3 The communication pattern forms a tree from the perspective of any process

13. 01234567 0246 04 0 1357 15 1 26 2 37 3 4567 Butterfly: Superimpose Trees After log 2 N steps each process is notified that the barrier operation has completed

14. Summary Deadlock detection and recovery algorithm –Performance critically dependent on lookahead Barrier mechanisms –Simple barriers using broadcast or central controller OK for small number of processors –Tree or butterfly give more scalable performance