Distributed Deadlocks

Model resource vs communication deadlocks wait-for-graph (WFG; TWG in database parla) strategies: prevention, avoidance, detection & resolution issues in detection & resolution all existing deadlocks must be detected in finite time no false (phantom) deadlocks should be detected selecting victims and cleaning up WFG after resolution fundamental causes of difficulty: consistent global state and lack of global clocks deadlock detection algorithms centralized, distributed, hierarchical

Completely Centralized Deadlock Detection Algorithm every process sends its request/release resource requests to a central control site which performs deadlock detection and resolution advantages simple and easy to implement disadvantages inefficient (requests for local resources are sent to control site) large communication delays congestion fault-tolerance phantom deadlocks

Ho-Ramamoorthy’s One-Phase Centralized deadlock Detection Algorithm a control site periodically requests a status report from all sites each site maintains resource status table which lists for each resource the processes that hold/request local resources process status table which keeps track of resources held/requested by all local processes WFG is constructed by adding a request/assignment edge between P and R if only if P and R appear consistently in both the relevant process status and resource status tables cons: storage and larger messages Ho-Ramamoorthy’s two-phase algorithm, which only uses process status tables, detects phantom deadlocks

Distributed Deadlock Detection Algorithms Various strategies for constructing global WFG path-pushing which disseminate wait-for dependencies in the form of paths (e.g. Obermarck’s algorithm) edge-chasing which circulate a “probe” to detect a cycle (e.g. Chandy-Misra-Haas’s algorithm) diffusion computation which use “echo messages” (i.e. query/reply messages) global state based which are based on the fact that deadlock is a persistent (stable) system property

Obermack’s Deadlock Detection Algorithm A path-pushing algorithm Transactions consist of a number of subtransactions and at each point in time only one subtransaction is active transactions are totally ordered special node “External” (Ex) abstracts remote sites in the WFG upon receiving deadlock detection information, a site constructs received info with its local WFG detects and resolves all local deadlocks (which do not contain node Ex) for all cycles (Ex, T1, T2, …, Tk, Ex) it sends this path to all other sites where a subtransaction of T1, T2, .., Tk may be waiting to receive a message from a local subtransaction (and remote subtransaction at that site has higher priority than local subtransaction)

Obermack’s Deadlock Detection Algorithm Detects phantom deadlocks (cased by the asynchronous snapshots taken by the various sites) performance #messages is n(n-1)/2 for a deadlock involving n sites message size is O(n) detection delay is O(n)

Chandy-Misra-Haas’s Edge-Chasing Deadlock Detection Algorithm Uses a probe message (i,j, k): deadlock detection initiated for process Pi and is sent by the site of Pj to the site of Pk boolean vector dependent(i,j) is kept at the site of Pi, which is true only if Pi knows that Pj is dependent on Pi

Chandy-Misra-Haas’s Edge-Chasing Algorithm Deadlock initiation at Pi if Pi is locally dependent on itself, then “deadlock” else for all Pj and Pk send a probe (i, j, k) to the site of Pk if Pi is locally dependent on Pj Pj is waiting for Pk Pk runs on another site

Chandy-Misra-Haas’s Edge-Chasing Algorithm Upon receipt of a probe (i,j,k) at the site of Pk if Pk is blocked, and dependent(k,i) is false, and Pk has not replied to all the requests of Pj, then set dependent(k,i) = true if k=i then “deadlock” else send a probe (i, m, n) to the site of Pn, if Pk is dependent on Pm, Pm waits for Pn, and Pm and Pn are on different sites

Chandy-Misra-Haas’s Edge-Chasing Algorithm Performance #messages is m(n-1)/2, where m is #processes and n is #sites involved in the deadlock message size is very small detection delay is O(n)

Chandy-Misra-Haas’s Diffusion Deadlock Detection Algorithm Deadlock detection computation is diffused through the WFG uses two types of messages: query(i,j,k) and reply(i,j,k) processes are active or blocked a blocked process Pi sends query messages to its dependent set DSi of processes engaging query: 1st query(i,j,k) message received by Pk num(k,i) is #query messages sent by Pk for deadlock initiated by Pi wait(k,i) is true if Pk was continuously blocked since the receipt of engaging query by Pi Pi detects a deadlock is it receives replies to all query messages it sent

Chandy-Misra-Haas’s Diffusion Algorithm Initiate deadlock detection at Pi send query(i,i,j) to all prtocesses Pj in Dsi set num(i,i) = sizeof(DSi) set wait(i,i) to true Upon receipt of query(i,j,k) at Pk if not engaging query and wait(k,i) then send reply(i,k,j) to Pj else if engaging query then send query(i,k,m) to all Pm in DSk set num(k,i) = sizeof(DSk) set wait(k,i) to true set engage(k,i) = j

Chandy-Misra-Haas’s Diffusion Algorithm Upon receipt of reply(i,j,k) at Pk if wait(k,i) then decrement num(k,i) by 1 if num(k,i) = 0 then if I=k then ‘deadlock” else send reply(i,k,m) to Pm where m = engage(k,i) Note that only one diffusion computation can be active at a time multiple diffusion computations may be active clocks can be used to distinguise old/new diffusion computations for each process

Hierarchical Deadlock Detection Algorithms Menasce-Muntz controllers are organized in a tree leaf controllers manage resources other controllers detect deadlocks each controller maintains a WFG for its subtree each controller updates (continuously or periodically) its parent whenever there is a change in its WFG Ho-Ramamoorthy partition sites into clusters, each having a control site designate a central control site control sites run Ho-Ramamoorthy’s one-phase algorithm central control splices cluster-WFG to global WFG

Performance Metrics Trade-off between storage overhead Communication overhead deadlock persistence time storage overhead computation overhead resolution overhead how process behavior and detection/resolution algorithm used affect these measures is open

Deadlock Resolution Efficient resolution of deadlock requires knowledge of all processes and resources involved in the deadlock in many algorithms process detecting the deadlock does not know all processes/resources involved in the deadlock more than one processes may detect the same deadlock use of process priorities can help to address these issues resolving a deadlock requires that victim is aborted and resources restored to a proper state detection info for victim flushed out from the system quickly since otherwise phantom deadlocks may be detected aborting a single victim may resolve all deadlocks detection and resolution interact extensively