Performance of Token- based Distributed Mutual Exclusion Algorithms Scott J. McCallen Kent State University November

Token-based Algorithms A single token circulates through the system A single token circulates through the system A process wants to enter CS, it requests token A process wants to enter CS, it requests token A process gets the token, it can enter CS (in general) A process gets the token, it can enter CS (in general) Message Types Message Types Token Token Request Request

Raymond’s Token DMX Based on a tree structure Based on a tree structure Each node holds a pointer to a node that either holds the token or points to where the token is Each node holds a pointer to a node that either holds the token or points to where the token is A request is held locally and is sent through the tree until it reaches the token A request is held locally and is sent through the tree until it reaches the token Token traverses the request path until it reaches the requester Token traverses the request path until it reaches the requester

Suzuki-Kasami Token DMX Fully connected network Fully connected network Requester broadcasts requests to all other nodes with sequence number of request Requester broadcasts requests to all other nodes with sequence number of request Before sending the token, holder calculates which requests have not been satisfied and enters those into the token request queue Before sending the token, holder calculates which requests have not been satisfied and enters those into the token request queue Send token to the request at the top of the queue Send token to the request at the top of the queue

Initial Verification Experiments Setup Done to ensure correctness and begin experimentation Done to ensure correctness and begin experimentation Attributes Attributes Synchronization Delay Synchronization Delay Number of messages between each CS entry Number of messages between each CS entry Message Complexity Message Complexity Total number of messages per CS entry Total number of messages per CS entry Variables Variables Network size Network size Range from 5 to 30 for Suzuki-Kasami (why?) Range from 5 to 30 for Suzuki-Kasami (why?) Range from 5 to 60 for Raymond (why?) Range from 5 to 60 for Raymond (why?) Contention level Contention level low (sequential request/release) and high (all request/all release) low (sequential request/release) and high (all request/all release)

Suzuki-Kasami Results (Low Load)

Suzuki-Kasami Results (High Load)

Raymond Results (Low Load)

Raymond Results (High Load)

Results Summary Raymond Raymond Low Load Low Load Message Complexity : log(n)/log(2) [not log(n)/2] Message Complexity : log(n)/log(2) [not log(n)/2] Synchronization Delay : 2*log(n)/log(2) [not log(n)] Synchronization Delay : 2*log(n)/log(2) [not log(n)] High Load High Load Message Complexity : 4 Message Complexity : 4 Synchronization Delay : 2 Synchronization Delay : 2 Suzuki-Kasami Suzuki-Kasami Low Load Low Load Message Complexity : N Message Complexity : N Synchronization Delay : 1 Synchronization Delay : 1 High Load High Load Message Complexity : N Message Complexity : N Synchronization Delay: 1 Synchronization Delay: 1

Raymond’s Children Message Complexity in low load was shown to be similar to log(n)/log(2) Message Complexity in low load was shown to be similar to log(n)/log(2) In initial experiments, each node had at most 2 children In initial experiments, each node had at most 2 children How does the number of children affect the synchronization delay for Raymond’s DMX? How does the number of children affect the synchronization delay for Raymond’s DMX?

Raymond’s Children Experiment Setup Attributes Attributes Synchronization Delay Synchronization Delay Variables Variables Number of children per node Number of children per node Range from 1 to 8 (why?) Range from 1 to 8 (why?) Parameters Parameters 50 nodes in the network (why?) 50 nodes in the network (why?) 40 sequential request/releases (low contention) 40 sequential request/releases (low contention)

Results Increasing the number of children per node decreases the message complexity Increasing the number of children per node decreases the message complexity

Suzuki-Kasami Average Token Forwarding In high contention, nodes are put on the queue based on node identification In high contention, nodes are put on the queue based on node identification A token is then released to the top request on the queue A token is then released to the top request on the queue What happens when a node has the last request and what is the average of message complexity and number of token forwards before CS is allow? What happens when a node has the last request and what is the average of message complexity and number of token forwards before CS is allow?

Forwarding Experiments Setup Attributes Attributes Message Complexity + Token Forwards until CS Message Complexity + Token Forwards until CS Variables Variables Network size Network size Range from 5 to 30 by 5 Range from 5 to 30 by 5 Parameters Parameters Nodes – 1 requests Nodes – 1 requests

Suzuki-Kasami Results Based on the number of nodes in the network, the result is proportional to [(nodes*requests)+sum(1:requests)]/nodes Based on the number of nodes in the network, the result is proportional to [(nodes*requests)+sum(1:requests)]/nodes

Conclusions Raymond Raymond Better performance with many children Better performance with many children Log(n)/log(c) message complexity and sync delay Log(n)/log(c) message complexity and sync delay Suzuki Kasami Suzuki Kasami Very low sync delay Very low sync delay Steady increase in total token forwards for high contention system Steady increase in total token forwards for high contention system Neither algorithm guarantees granting CS in timestamp order Neither algorithm guarantees granting CS in timestamp order

Future Work Test systems in continuous high-contention Test systems in continuous high-contention Current experiments are all-request/all-release, continuous high- contention may produce alternate results Current experiments are all-request/all-release, continuous high- contention may produce alternate results Test Raymond’s algorithm with random tree Test Raymond’s algorithm with random tree It may not be possible to create a tree with specific children count, what results will be produced in the random case It may not be possible to create a tree with specific children count, what results will be produced in the random case Test algorithms in a real (non-random) situation Test algorithms in a real (non-random) situation Perhaps there are tree algorithms which require high-contention critical section access within a small subgraph Perhaps there are tree algorithms which require high-contention critical section access within a small subgraph Implement message piggy-backing as suggested by Raymond Implement message piggy-backing as suggested by Raymond Could reduce costs of duplicate messages for high-contention system, but should not affect low contention Could reduce costs of duplicate messages for high-contention system, but should not affect low contention

Coding the Project Problems Problems Message Size Message Size Limited the number of nodes for Suzuki-Kasami Limited the number of nodes for Suzuki-Kasami Request Queue Request Queue Incrementing length Incrementing length Added methods to check contents of queue Added methods to check contents of queue copyTokenToOut didn’t copy all information copyTokenToOut didn’t copy all information Knowing which node had the token Knowing which node had the token Required sending messages to all nodes to release Required sending messages to all nodes to release Token loss Token loss Token was lost in some tests making that test erroneous Token was lost in some tests making that test erroneous No token loss check algorithm in place for these tests No token loss check algorithm in place for these tests

Coding the Project Successes Successes Algorithms Algorithms Implementation was mostly successful Implementation was mostly successful Tython scripts Tython scripts Learning Python rather quickly Learning Python rather quickly Scripts can run arbitrary number of trials Scripts can run arbitrary number of trials Outputting results to file for access later Outputting results to file for access later Scripts can be ran unattended (except changing network size) Scripts can be ran unattended (except changing network size) MATLAB MATLAB Utilized to create graphics Utilized to create graphics

References Brewer, E., Culler, D., Gay, D., and Levis, P., nesC 1.1 Language Reference Manual, May Brewer, E., Culler, D., Gay, D., and Levis, P., nesC 1.1 Language Reference Manual, May Gay, D., Levis, P., von Behren, R., Welsh, M., Brewer, E., and Culler, D The nesC language: A holistic approach to networked embedded systems. SIGPLAN Not. 38, 5 (May. 2003), Gay, D., Levis, P., von Behren, R., Welsh, M., Brewer, E., and Culler, D The nesC language: A holistic approach to networked embedded systems. SIGPLAN Not. 38, 5 (May. 2003), Hill, J., Szewczyk, R., Woo, A., Hollar, S., Culler, D., and Pister, K System architecture directions for networked sensors. SIGOPS Oper. Syst. Rev. 34, 5 (Dec. 2000), Hill, J., Szewczyk, R., Woo, A., Hollar, S., Culler, D., and Pister, K System architecture directions for networked sensors. SIGOPS Oper. Syst. Rev. 34, 5 (Dec. 2000), Levis, P., Lee, N., Welsh, M., and Culler, D TOSSIM: accurate and scalable simulation of entire tinyOS applications. In Proceedings of the 1st international Conference on Embedded Networked Sensor Systems (Los Angeles, California, USA, November , 2003). SenSys '03. ACM Press, New York, NY, Levis, P., Lee, N., Welsh, M., and Culler, D TOSSIM: accurate and scalable simulation of entire tinyOS applications. In Proceedings of the 1st international Conference on Embedded Networked Sensor Systems (Los Angeles, California, USA, November , 2003). SenSys '03. ACM Press, New York, NY, Raymond, K A tree-based algorithm for distributed mutual exclusion. ACM Trans. Comput. Syst. 7, 1 (Jan. 1989), Raymond, K A tree-based algorithm for distributed mutual exclusion. ACM Trans. Comput. Syst. 7, 1 (Jan. 1989), Suzuki, I. and Kasami, T A distributed mutual exclusion algorithm. ACM Trans. Comput. Syst. 3, 4 (Nov. 1985), Suzuki, I. and Kasami, T A distributed mutual exclusion algorithm. ACM Trans. Comput. Syst. 3, 4 (Nov. 1985),