1. Correcting Error in Message Passing Systems
By Jan Bækgaard Pedersen
A Tool in Millipede Interactive Parallel Debugger

2. Overview Multi Level Interactive Parallel Debugging
Millipede
The problem – Deadlocks
Deadlock detection module
The idea
The Algorithm
Theoretical justification

3. Multi Level Interactive Parallel Debugging
Use a tool that is tailored to the specific debugging task
Debugging
straight line
code
Message debugging
Protocol debugging
Visualization
Sequential tool to debug sequential
code.
Other tools to debug
Message passing errors
Message contents
Protocol errors
Protocol verification
Deadlock correction

4. Millipede Communication Visualization Module
Graphical view of the message
passing / protocol.
Deadlock
Detection &
Correction
Module
Detect and analyze deadlocks
And report the cause and fix
Comm.
Protocol
Verification
Module
Online verification of the
comm. protocol while running
Message Debugging
Module
Inspect, control and change
Contents of messages
Sequential Debugging
Module
Debugging of the sequential
code of the parallel program

5. The Problem
Message passing programs can deadlock:

6. The Idea S = {s0, s1,…,sn-1} list of deadlocked* senders.
R = {r0, r1,…,rn-1} list of deadlocked* receivers.
si = (a,b), a and b are process identifiers, with a fixed by the sender.
ri = (a,b), a and b are process identifiers, with b fixed by the receiver.
si = (ai,bi) matches rj = (aj,bj) if (ai = aj) and (bi = bj)

7. The Idea Find permutations S of S and R of R:
Number of mismatches is minimal, i.e.
Minimal number of fields must be changed for the deadlock to disappear.
Report needed changes to user.
Compute Hamming distances between all
possible permutations of S and R, and pick
the ones with the minimal distance.

8. The Algorithm Let G = (V,E) be a directed graph
V=VsVr (Vs senders, Vr receivers)
E is constructed in the following way
For all messages m left in message queues:
If m=(s,r) is an outstanding send (sVs, rVr) add edge (s,r) to E with capacity 2.
If m=(r,s) is an outstanding receive (sVs, rVr) add edge (r,s) to E with capacity 2.
Iterate backwards through all delivered messages and add edge (u,v) and (v,u) to E with capacity 2 if no other node exist in E with u or v as source or destination.
Add edges with capacity 1 to E to make G complete.
Run maximum bipartite graph matching to get a matching.

9. Theoretical Justification
How accurate is this algorithm?
How is accuracy defined?
Given a working system without deadlocks.
Introduce a number of errors.
Run the algorithm.
If the algorithm often suggests the original working program as a fix, the accuracy is good.

10. 4 moves
3 moves
7 moves

11. Example

12. Example

13. Examples
3 moves