CS Multiprocessor Synchronization

CS 0368-4061-01 Multiprocessor Synchronization
Nir Shavit Tel-Aviv University

Requirements Required Reading of Handouts 5-6 problem sets
35% of grade Typeset in LaTeX/Word in English! Late loses 20%/day Closed book final exam 14-Jan-19 © 2003 Herlihy and Shavit

Overview Fundamentals Real-World programming
Models, algorithms, impossibility Real-World programming Architectures techniques 14-Jan-19 © 2003 Herlihy and Shavit

Sequential Computation
thread memory object object 14-Jan-19 © 2003 Herlihy and Shavit

Concurrent Computation
threads memory object object 14-Jan-19 © 2003 Herlihy and Shavit

Asynchrony Sudden unpredictable delays Cache misses (short)
Page faults (long) Scheduling quantum used up (really long) 14-Jan-19 © 2003 Herlihy and Shavit

Model Summary Multiple threads Single shared memory
Sometimes called processes Single shared memory Objects live in memory Unpredictable asynchronous delays 14-Jan-19 © 2003 Herlihy and Shavit

Road Map We are going to focus on principles first, then performance
Start with idealized models Look at simplistic problems Emphasize correctness over pragmatism “Correctness may be theoretical, but incorrectness has practical impact” 14-Jan-19 © 2003 Herlihy and Shavit

I’m no theory weenie - why all the theorems and proofs?
You may ask yourself … I’m no theory weenie - why all the theorems and proofs? 14-Jan-19 © 2003 Herlihy and Shavit

Fundamentalism Distributed & concurrent systems are hard
Failures Concurrency Easier to go from theory to practice than vice-versa 14-Jan-19 © 2003 Herlihy and Shavit

If both sides attack together
The Two Generals Red army wins If both sides attack together The red army is divided on two sides of a valley with the Blue army between them. If both sides attack simultaneously, the Red army wins, but if they attack at different times, the Blue army wins. 14-Jan-19 © 2003 Herlihy and Shavit

Communications Red armies send messengers across valley 14-Jan-19
© 2003 Herlihy and Shavit

Communications Messengers don’t always make it 14-Jan-19

Design a protocol to ensure that red armies attack simultaneously
Your Mission Design a protocol to ensure that red armies attack simultaneously 14-Jan-19 © 2003 Herlihy and Shavit

Proof Strategy Assume a protocol exists Reason about its properties
Derive a contradiction 14-Jan-19 © 2003 Herlihy and Shavit

Proof Consider the protocol that sends fewest messages
It still works if last message lost So just don’t send it Messengers’ union happy But now we have a shorter protocol! Contradicting #1 14-Jan-19 © 2003 Herlihy and Shavit

Fundamental Limitation
Need an unbounded number of messages Or possible that no attack takes place 14-Jan-19 © 2003 Herlihy and Shavit

You might say I want a real-time ROTFL- compliant Two Generals protocol using UDP datagrams running on our enterprise-level fiber tachyion network ... Yes, Ma’am, right away! 14-Jan-19 © 2003 Herlihy and Shavit

You might say Advantage: Buys time to find another job
No one expects software to work anyway I want a real-time dot-net compliant Two Generals protocol using UDP datagrams running on our enterprise-level fiber tachyion network ... Yes, Ma’am, right away! 14-Jan-19 © 2003 Herlihy and Shavit

You might say Advantage: Disadvantage: Buys time to find another job
No one expects software to work anyway I want a real-time dot-net compliant Two Generals protocol using UDP datagrams running on our enterprise-level fiber tachyion network ... Yes, Ma’am, right away! Disadvantage: You’re doomed Without CS , you may not even know you’re doomed 14-Jan-19 © 2003 Herlihy and Shavit

You might say I want a real-time ROTFL- compliant Two Generals protocol using UDP datagrams running on our enterprise-level fiber tachyion network ... I can’t find a fault-tolerant algorithm, I guess I’m just a pathetic loser. 14-Jan-19 © 2003 Herlihy and Shavit

You might say Advantage: No need to take CS 3068-4061-01
I want a real-time ROTFL- compliant Two Generals protocol using UDP datagrams running on our enterprise-level fiber tachyion network ... I can’t find a fault-tolerant algorithm, I guess I’m just a pathetic loser 14-Jan-19 © 2003 Herlihy and Shavit

You might say Advantage: Disadvantage: No need to take CS 3068-4061-01
I want a real-time ROTFL- compliant Two Generals protocol using UDP datagrams running on our enterprise-level fiber tachyion network ... I can’t find a fault-tolerant algorithm, I guess I’m just a pathetic loser Disadvantage: Boss fires you, hires graduate of Michlelet Tel-Hai 14-Jan-19 © 2003 Herlihy and Shavit

You might say I want a real-time ROTFL- compliant Two Generals protocol using UDP datagrams running on our enterprise-level fiber tachyion network ... Using skills honed in CS , I can avert certain disaster! Rethink problem spec, or Weaken requirements, or Build on different platform 14-Jan-19 © 2003 Herlihy and Shavit

Important Distinction
Safety Property Nothing bad happens ever Liveness Property Something good happens eventually 14-Jan-19 © 2003 Herlihy and Shavit

Concurrency Jargon Hardware Software
Processors Software Threads, processes Sometimes OK to confuse them, sometimes not. 14-Jan-19 © 2003 Herlihy and Shavit

Parallel Primality Testing
Challenge Print primes from 1 to 1010 Given Ten-processor multiprocessor One thread per processor Goal Get ten-fold speedup (or close) 14-Jan-19 © 2003 Herlihy and Shavit

Load Balancing Split the work evenly Each thread tests range of 109 …
14-Jan-19 © 2003 Herlihy and Shavit

Procedure for Thread i void run(int i) {
for (j = i*109+1, j < (i+1)*109; j++) { if (isPrime(j)) print(j); } 14-Jan-19 © 2003 Herlihy and Shavit

Issues rejected Larger Num ranges have fewer primes
Larger numbers harder to test Thread workloads Uneven Hard to predict Need dynamic load balancing rejected 14-Jan-19 © 2003 Herlihy and Shavit

each thread takes a number
Shared Counter 17 18 19 each thread takes a number 14-Jan-19 © 2003 Herlihy and Shavit

Procedure for Thread i int counter = new Counter(1);
void thread(int i) { int j = 0; while (j < 1010) { j = counter.inc(); if (isPrime(j)) print(j); } 14-Jan-19 © 2003 Herlihy and Shavit

Procedure for Thread i Shared counter object
int counter = new Counter(1); void thread(int i) { int j = 0; while (j < 1010) { j = counter.inc(); if (isPrime(j)) print(j); } Stop when every value taken Increment & return each new value 14-Jan-19 © 2003 Herlihy and Shavit

Counter Implementation
public class Counter { private long value; public long inc() { return value++; } OK for uniprocessor, not for multiprocessor 14-Jan-19 © 2003 Herlihy and Shavit

What It Means public class Counter { private long value;
public long inc() { return value++; } temp = value; value = value + 1; return temp; 14-Jan-19 © 2003 Herlihy and Shavit

Uh-Oh Value… 2 3 2 read 1 read 2 write 2 write 3 read 1 write 2 time

FLP: Facts of Life for Processors
write read read write If we could only glue reads and writes… 14-Jan-19 © 2003 Herlihy and Shavit

Make these steps atomic (indivisible)
Challenge public class Counter { private long value; public long inc() { temp = value; value = temp + 1; return temp; } Make these steps atomic (indivisible) 14-Jan-19 © 2003 Herlihy and Shavit

An Aside: Java™ Critical section Synchronized block
public class Counter { private long value; public long inc() { synchronized { temp = value; value = temp + 1; } return temp; Critical section Synchronized block 14-Jan-19 © 2003 Herlihy and Shavit

Correctness Mutual Exclusion No Lockout (lockout-freedom)
Never two or more in critical section This is a safety property No Lockout (lockout-freedom) If someone wants in, someone gets in This is a liveness property 14-Jan-19 © 2003 Herlihy and Shavit

The Problem A B The pets don’t get along 14-Jan-19

Correctness Mutual Exclusion No Deadlock
Both pets never in pond simultaneously No Deadlock if only one wants in, it gets in if both want in, one gets in. 14-Jan-19 © 2003 Herlihy and Shavit

Simple Protocol Idea Gotcha Just look at the pond
Trees obscure the view 14-Jan-19 © 2003 Herlihy and Shavit

Interpretation Threads can’t “see” what other threads are doing
Explicit communication required for coordination 14-Jan-19 © 2003 Herlihy and Shavit

Cell Phone Protocol Idea Gotcha Bob calls Alice (or vice-versa)
Bob takes shower Alice recharges battery Bob out shopping for pet food … 14-Jan-19 © 2003 Herlihy and Shavit

Interpretation Message-passing doesn’t work Recipient might not be
Listening There at all Communication must be Persistent (like writing) Not transient (like speaking) 14-Jan-19 © 2003 Herlihy and Shavit

Can Protocol Idea Gotcha Cans on Alice’s windowsill
Strings lead to Bob’s house Bob pulls strings, knocks over cans Gotcha Cans cannot be reused Bob runs out of cans 14-Jan-19 © 2003 Herlihy and Shavit

Interpretation Cannot solve mutual exclusion with interrupts
Sender sets fixed bit in receiver’s space Receiver resets bit when ready 14-Jan-19 © 2003 Herlihy and Shavit

Alice’s Protocol (sort of)
B 14-Jan-19 © 2003 Herlihy and Shavit

Bob’s Protocol (sort of)
A B 14-Jan-19 © 2003 Herlihy and Shavit

Alice’s Protocol Raise flag Wait until Bob’s flag is down Unleash pet
Lower flag when pet returns 14-Jan-19 © 2003 Herlihy and Shavit

Bob’s Protocol Bob defers to Alice Raise flag While Alice’s flag is up
Lower flag Wait for Alice’s flag to go down Unleash pet Lower flag when pet returns 14-Jan-19 © 2003 Herlihy and Shavit

The Flag Principle Raise the flag Look at other’s flag Flag Principle:
If each raises and looks, then Last to look must see both flags up This intuitively explains implies why at least one of them will not enter the critical section if both are trying at the same time. The following proof of mutual exclusion will not be presented in class, but we provide it just to give you some intuition about how one reasons about concurrent programs. Lets prove that if they follow the algorithm the dogs will never be together in the yard. Assume by way contradiction that this is not the case. We are assuming that both dogs are in the yard. Therefore both Alice and Bob had a last ``looking'' action before they let their dog enter the yard. Lets take a look at the one who finished this looking action first. When he (she) looked, he (she) saw that the other one's flag was down. Without loss of generality let's assume it was Bob, so he had {\tt (= Alice-flag 'down)} as true, otherwise he couldn't have entered the critical section. So it follows that Alice's flag was up {\em after} Bob finished his looking action. Therefore, Alice's looking was {\em completely after} the end of Bob's raising of his flag, so Alice must have seen this flag up and could not have entered the critical section, a contradiction. 14-Jan-19 © 2003 Herlihy and Shavit

Proof of Mutual Exclusion
Assume both pets in pond Derive a contradiction By reasoning backwards 14-Jan-19 © 2003 Herlihy and Shavit

Alice raised her flag after Bob looked
Proof Bob raises flag Alice raised her flag after Bob looked Alice’s last look Bob’s last look time 14-Jan-19 © 2003 Herlihy and Shavit

Contradiction QED Contradiction: Alice didn’t see Bob’s flag
Except that she did QED 14-Jan-19 © 2003 Herlihy and Shavit

Proof of No Deadlock QED If only one pet wants in, it gets in.
Deadlock requires both continually trying to get in. If Bob sees Alice’s flag, he gives her priority (gentleman or hegemonistic patriarch?) QED 14-Jan-19 © 2003 Herlihy and Shavit

Remarks Protocol is unfair Protocol uses waiting
Bob’s pet might never get in Protocol uses waiting If Bob is eaten by his pet, Alice’s pet might never get in 14-Jan-19 © 2003 Herlihy and Shavit

Moral of Story Mutual Exclusion cannot be solved by
transient communication (cell phones) interrupts (cans) It can be solved by one-bit shared variables that can be read or written During the course we will devote quite a bit of effort to understanding the tradeoffs that have to do with the use of mutual exclusion. 14-Jan-19 © 2003 Herlihy and Shavit

The Fable Continues Alice and Bob fall in love & marry
Then they fall out of love & divorce She gets the pets He has to feed them Leading to a new coordination problem 14-Jan-19 © 2003 Herlihy and Shavit

Producer/Consumer Alice and Bob can’t meet Avoid
Each has restraining order on other So he puts food in the pond And later, she releases the pets Avoid Releasing pets when there’s no food Putting out food if uneaten food remains 14-Jan-19 © 2003 Herlihy and Shavit

Producer/Consumer Need a mechanism so that
Bob lets Alice know when food has been put out Alice lets Bob know when to put out more food 14-Jan-19 © 2003 Herlihy and Shavit

Pseudocode Bob’s code Alice’s code while (true) {
while (can.isUp()){}; pet.release(); pet.recapture(); can.reset(); } while (true) { while (can.isDown()){}; pond.stockWithFood(); can.knockOver(); } Bob’s code Alice’s code 14-Jan-19 © 2003 Herlihy and Shavit

Correctness safety liveness liveness Mutual Exclusion No Starvation
Pets and Bob never together in pond No Starvation if Bob is always willing to feed, and the pets are always famished, then the pets will eat infinitely often. Producer/Consumer: The pets will not enter the yard unless there is food, and Bob will never provide more food if there is unconsumed food. liveness liveness \item \emph{Mutual Exclusion}: Bob and the pets are never in the yard together. \item \emph{No-Starvation}: if Bob is always willing to feed, and the pets are always famished, then the pets will eat infinitely often. \item \emph{Producer/Consumer}: The pets will not enter the yard unless there is food, and Bob will never provide more food if there is unconsumed food. 14-Jan-19 © 2003 Herlihy and Shavit

Waiting Both solutions use waiting Waiting is problematic
while(mumble){} Waiting is problematic If one participant is delayed So is everyone else But delays are common & unpredictable 14-Jan-19 © 2003 Herlihy and Shavit

The Fable drags on … Bob and Alice still have issues
So they need to communicate So they agree to use billboards … 14-Jan-19 © 2003 Herlihy and Shavit

Billboards are Large D B A C E Letter Tiles From Scrabble™ box 2 3 1 3

To post a message W A S H T H E A C R whew 4 1 1 4 1 3 14-Jan-19

Uh-Oh S E L T H E A C R L OK 1 1 4 1 3 1 14-Jan-19

Readers/Writers Devise a protocol so that
Writer writes one letter at a time Reader reads one letter at a time Reader sees Old message or new message No mixed messages 14-Jan-19 © 2003 Herlihy and Shavit

Readers/Writers (continued)
Easy with mutual exclusion But mutual excluson requires waiting We prefer not to wait Delay one, delay all … Remarkably We can solve R/W without waiting Stay tuned … 14-Jan-19 © 2003 Herlihy and Shavit

CS Multiprocessor Synchronization

Similar presentations

Presentation on theme: "CS Multiprocessor Synchronization"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

CS Multiprocessor Synchronization

Similar presentations

Presentation on theme: "CS Multiprocessor Synchronization"— Presentation transcript:

Similar presentations

About project

Feedback