1. A Log(n) Multi-Mode Locking Protocol for Distributed Systems
Nirmit Desai, Frank Mueller
Department of Computer Science
North Carolina State University

2. Background Problem Motivation Approach Contribution
Large parallel and distributed systems
Scalability of synchronization
Motivation
Increasing need to share resources
Inadequate scalability of current protocols
Approach
Leveraging hierarchical locking in distributed synchronization
Contribution
Highly scalable protocol
Supports hierarchical locking
May 6, 2019
NC State University

3. Large Distributed Environments
Cluster Environments
Shared Resource 1
Shared Resource 2
Site 1
Site 2
Site 3
Network
Site N
May 6, 2019
NC State University

4. Distributed Synchronization (Naimi et al.)
Request from A
Request from C
T is done
Single locking mode - Read and Write are not distinguished
Single level of locking granularity (either entire table or a tuple)
May 6, 2019
NC State University

5. Hierarchical Locking (a.k.a. Multi-granularity Locking)
How is a lock associated with data items ?
Data
Fine grain
Data
Coarse grain
Data
Multi grain
- High overhead
- High concurreny
- Low overhead
- No concurrency
- Low overhead
- High concurrency
May 6, 2019
NC State University

6. Multi-Mode Locking Protocol
1. Intention Read (IR) /Intention Write (IW) on all upper level resources
2. Read (R) locks and Write (W) locks on the resource itself
A : Reads tuple 2
t0:Acquire IR on the table
t1:Acquire R on tuple #2
t2:Read tuple #2
t3:Release R
t4:Release IR
Data table
B : Writes tuple 3
t0:Acquire IW on the table
t1:Acquire W on tuple #3
t2:Write tuple #3
t3:Release W
t4:Release IW
May 6, 2019
NC State University

7. Multi-Mode Locking Protocol (cont.)
Modes incompatible?  Conflict  Block
A: reads entire table
t0:Acquire R on the table
t1:Read table
t2:Release R on the table
Data table
B : Writes a tuple
t0:Request IW on the table
t1: wait for lock
t2:Acquires IW
t3:Acquire W on tuple
t4:... X
May 6, 2019
NC State University

8. IR < R < U = IW < W
Compatibility Matrix
‘X’ indicates a conflict
Upgrade: Read followed by write to same data
assures consistent data
Lock Mode Strength: Less compatible  Stronger lock mode
IR < R < U = IW < W
May 6, 2019
NC State University

9. Our Protocol - Conventions
Held Mode: Mh = Mode of held lock
Owned Mode: Mo = Strongest held mode in tree (rooted=owner)
Pending Mode: Mp = await this mode
Node state: (Mo, Mh, Mp)
Request arrives  test compatibility  grant/queue/forward based on result
Actions: rules & transition tables
Requester becomes a child of granter
A(0,0,0)
C(IR,IR,0)
D(IR,IR,0)
B(R,R,0)
E(U,U,0)
May 6, 2019
NC State University

10. A Granting/Forwarding Example
Invariant : “Node cannot grant request for stronger mode than it owns”  has to forward it to parent
A(0,0,0)
C(0,0,0)
D(0,0,IR)
B(0,0,0)
E(IR,IR,0)
A(0,0,R)
C(0,0,0)
D(IR,IR,0)
B(0,0,0)
E(IR,IR,0)
A(R,R,0)
C(0,0,0)
D(IR,IR,0)
B(0,0,0)
E(IR,IR,0)
A requests R
Token transfer
(D,IR)
(A,R)
Grant
(A,R)
If Mr compatible with Mo  Mr compatible w/ all modes in subtree
May 6, 2019
NC State University

11. A Release Example Invariant: “Node knows owned mode of its children”
C(IR,IR,0)
D(IR,IR,0)
B(IR,IR,0)
E(R,R,0)
A(0,0,0)
C(IR,IR,0)
D(IR,IR,0)
B(IR,0,0)
E(R,R,0)
B(IR,0,0)
A(0,0,0)
C(0,0,0)
D(IR,IR,0)
E(R,R,0)
B releases IR
C releases IR
(B,IR)
B(0,0,0)
(C,IR)
Notice: If a child releases a mode but owned mode is unchanged, no need to notify our parent
May 6, 2019
NC State University

12. A Queuing Example
Invariants: “Queue if request is conflicting. Freeze modes if it is a) incompatible with queued request and b) grantable otherwise”
A(0,0,IR)
C(0,0,W)
D(R,R,0)
B(IR,IR,0)
E(R,R,0) IR
IR, R
(C,W)
IR, R, U
A(0,0,0)
C(0,0,W)
D(R,R,0)
B(IR,IR,0)
E(R,R,0)
IR, R, U
A(0,0,0)
C(0,0,W)
D(R,R,0)
B(IR,IR,0)
E(R,R,0)
(C,W)
IR, R, U
Send freeze
A requests IR
(C,W)
(C,W)(A,IR)
Freeze
(IR)
(IR, R)
(C,W)
(A,IR) IR
IR, R
(C,W)
(A,IR)
Starvation of queued request is avoided, FIFO is ensured
May 6, 2019
NC State University

13. Measurements Metrics of interest: Scalability of
Message overhead per lock request
Request latency time
Effect of changing concurrency level on scalability
higher ratio  lower concurency
Compare with Naimi’s protocol
Coarse: Lock the entire table
Fine: Lock each entry
Multi-airline reservation system
IBM SP Power3
May 6, 2019
NC State University

14. Comparison with Naimi’s protocols
Message overhead per Request
Response time per Request
Naimi coarse has different functionality
May 6, 2019
NC State University

15. Effect of concurrency level
Concurrency level increases as # of nodes increase
May 6, 2019
NC State University

16. Applications State management of replicated data
Distributed databases (Oracle, MS-SQL)
Distributed transaction processing
Middleware concurrency services (CORBA)
Coordinated, distributed Server Farms
Fault tolerance in clusters
State replication for redundant computing
Distributed agreement
State coherence
May 6, 2019
NC State University

17. Related Work Naimi et. al. (JPDC ‘96) Ramamritham et. Al. (SIGMOD ‘90)
Distributed mutual exclusion, single level of granularity
Ramamritham et. Al. (SIGMOD ‘90)
Concurrency protocols, centralized transaction coordinator
Other approaches: complexity > O(log (n))
Hierarchical locking: only in context of distributed synchronization problems in past
Scalability: had not been the focus
May 6, 2019
NC State University

18. Conclusions Novel peer-to-peer distributed concurrency protocol
Compact formulation: rules and tables  simplicity intriguing
Highly scalable: O(log n) msgs, asymptote 3-5 msgs
High degree of concurrency common
Scalability unaffected by concurrency level
Response time: linear increasing w/ concurrency level
Best scalable protocol for hierarchical locking known
Wide applications in parallel and distributed world
May 6, 2019
NC State University

19. Breakup of Messages Request propagation is major component C
May 6, 2019
NC State University

20. Request Message Concurrency level Request forwarding Request
Nodes Tree height Path length Request
more request msgs
propagation paths longer
May 6, 2019
NC State University

21. Token Message Concurrency level Granting Token transfers
Nodes Tree height Granting Token transfers
more contention
lower chance to grant token
May 6, 2019
NC State University

22. Grant Message Concurrency level Granting Nodes Tree height Granting
more contention
more child grants
May 6, 2019
NC State University

23. Release Message Concurrency level Granting Release
Nodes Tree height Granting Release
higher concurrency
more trees & release msgs
May 6, 2019
NC State University

24. Freeze Message Concurrency level Conflicts Freeze
Nodes Requests Conflicts Freeze
higher concurrency
more freezes (be fair)
May 6, 2019
NC State University

25. Rule for Granting at Non-root
X = cannot grant
May 6, 2019
NC State University

26. Rule for Queuing/Forwarding
Queue
May 6, 2019
NC State University

27. Rule for Freezing Modes
Modes to freeze
May 6, 2019
NC State University