1. Mary Jane Irwin ( www.cse.psu.edu/~mji )
CSE 431 Computer Architecture Fall Lecture 26. Bus Connected Multi’s
Mary Jane Irwin ( )
[Adapted from Computer Organization and Design, Patterson & Hennessy, © 2005]
Other handouts
To handout next time

2. Review: Where are We Now?
Processor
Processor
Output
Output
Control
Control
Memory
Memory
Datapath
Input
Input
Datapath
Multiprocessor – multiple processors with a single shared address space
Cluster – multiple computers (each with their own address space) connected over a local area network (LAN) functioning as a single system

3. Multiprocessor Basics
Q1 – How do they share data?
Q2 – How do they coordinate?
Q3 – How scalable is the architecture? How many processors?
# of Proc
Communication model
Message passing
8 to 2048
Shared address
NUMA
8 to 256
UMA
2 to 64
Physical connection
Network
Bus
2 to 36

4. Single Bus (Shared Address UMA) Multi’s
Proc1
Proc2
Proc3
Proc4
Caches
Caches
Caches
Caches
Single Bus
Memory
I/O
Caches are used to reduce latency and to lower bus traffic
Write-back caches used to keep bus traffic at a minimum
Must provide hardware to ensure that caches and memory are consistent (cache coherency)
Must provide a hardware mechanism to support process synchronization

5. Multiprocessor Cache Coherency
Cache coherency protocols
Bus snooping – cache controllers monitor shared bus traffic with duplicate address tag hardware (so they don’t interfere with processor’s access to the cache)
Proc1
Proc2
ProcN
DCache
Single Bus
Memory
I/O
Snoop

6. Bus Snooping Protocols
Multiple copies are not a problem when reading
Processor must have exclusive access to write a word
What happens if two processors try to write to the same shared data word in the same clock cycle? The bus arbiter decides which processor gets the bus first (and this will be the processor with the first exclusive access). Then the second processor will get exclusive access. Thus, bus arbitration forces sequential behavior.
This sequential consistency is the most conservative of the memory consistency models. With it, the result of any execution is the same as if the accesses of each processor were kept in order and the accesses among different processors were interleaved.
All other processors sharing that data must be informed of writes

7. Handling Writes
Ensuring that all other processors sharing data are informed of writes can be handled two ways:
Write-update (write-broadcast) – writing processor broadcasts new data over the bus, all copies are updated
All writes go to the bus  higher bus traffic
Since new values appear in caches sooner, can reduce latency
Write-invalidate – writing processor issues invalidation signal on bus, cache snoops check to see if they have a copy of the data, if so they invalidate their cache block containing the word (this allows multiple readers but only one writer)
Uses the bus only on the first write  lower bus traffic, so better use of bus bandwidth
All commercial machines use write-invalidate as their standard coherence protocol

8. A Write-Invalidate CC Protocol
read (hit or miss)
read (miss)
Invalid
Shared
(clean)
receives invalidate
(write by another processor
to this block)
write-back due to read (miss) by another processor to this block
write (miss)
send invalidate
send invalidate
write (hit or miss)
For lecture
A read miss causes the cache to acquire the bus and write back the victim block (if it was in the Modified (dirty) state). All the other caches monitor the read miss to see if this block is in their cache. If one has a copy and it is in the Modified state, then the block is written back and its state is changed to Invalid. The read miss is then satisfied by reading from the block memory, and the state of the block is set to Shared.
Read hits do not change the cache state.
A write miss to an Invalid block causes the cache to acquire the bus, read the block, modify the portion of the block being written and changing the block’s state to Modified. A write miss to a Shared block causes the cache to acquire the bus, send an invalidate signal to invalidate any other existing copies in other caches, modify the portion of the block being written and chang the block’s state to Modified.
A write hit to a Modified block causes no action. A write hit to a Shared block causes the cache to acquire the bus, send an invalidate signal to invalidate any other existing copies in other caches, modify the part of the block being written, and change the state to Modified.
write-back caching protocol in black
Modified
(dirty)
signals from the processor coherence additions in red
signals from the bus coherence additions in blue
read (hit) or write (hit)

9. Write-Invalidate CC Examples
I = invalid (many), S = shared (many), M = modified (only one)
3. snoop sees read request for A & lets MM supply A
1. read miss for A
1. write miss for A
Proc 1
A S
Main Mem A
Proc 2
A I
Proc 1
A S
Main Mem A
Proc 2
A I
4. gets A from MM & changes its state to S
2. writes A & changes its state to M
4. change A state to I
2. read request for A
3. P2 sends invalidate for A
1. write miss for A
3. snoop sees read request for A, writes-back A to MM
1. read miss for A
Proc 1
A M
Main Mem A
Proc 2
A I
Proc 1
A M
Main Mem A
Proc 2
A I
4. gets A from MM & changes its state to M
2. writes A & changes its state to M
For lecture
1st example – supplying A from Proc 1 cache would be faster than getting it from MM
2nd and 4th example – assumes a Write allocate cache policy
5. change A state to I
4. change A state to I
5. P2 sends invalidate for A
2. read request for A
3. P2 sends invalidate for A

10. SMP Data Miss Rates
Shared data has lower spatial and temporal locality
Share data misses often dominate cache behavior even though they may only be 10% to 40% of the data accesses
64KB 2-way set associative data cache with 32B blocks
Hennessy & Patterson, Computer Architecture: A Quantitative Approach
Compulsory misses are all very small and are included in capacity misses. The x axis is processor count

11. Block Size Effects Writes to one word in a multi-word block mean
either the full block is invalidated (write-invalidate)
or the full block is exchanged between processors (write-update)
Alternatively, could broadcast only the written word
Multi-word blocks can also result in false sharing: when two processors are writing to two different variables in the same cache block
With write-invalidate false sharing increases cache miss rates A B
Proc1
Proc2
4 word cache block
Compilers can help reduce false sharing by allocating highly correlated data to the same cache block

12. Other Coherence Protocols
There are many variations on cache coherence protocols
Another write-invalidate protocol used in the Pentium 4 (and many other micro’s) is MESI with four states:
Modified – same
Exclusive – only one copy of the shared data is allowed to be cached; memory has an up-to-date copy
Since there is only one copy of the block, write hits don’t need to send invalidate signal
Shared – multiple copies of the shared data may be cached (i.e., data permitted to be cached with more than one processor); memory has an up-to-date copy
Invalid – same
For E and S book states that memory has an up-to-date version so is this not a write-back cache ???

13. Process Synchronization
Need to be able to coordinate processes working on a common task
Lock variables (semaphores) are used to coordinate or synchronize processes
Need an architecture-supported arbitration mechanism to decide which processor gets access to the lock variable
Single bus provides arbitration mechanism, since the bus is the only path to memory – the processor that gets the bus wins
Need an architecture-supported operation that locks the variable
Locking can be done via an atomic swap operation (processor can both read a location and set it to the locked state – test-and-set – in the same bus operation)

14. Spin Lock Synchronization
Read lock
variable
Spin
unlock variable:
set lock variable
to 0 No
Unlocked?
(=0?)
Yes
Finish update of
shared data
atomic
operation
Try to lock variable using swap:
read lock variable and set it
to locked value (1) .
The losers will continue to write the variable with the locked value of 1, but that is okay! No
Yes
Begin update of
shared data
Succeed?
(=0?)
The single winning processor will read a 0 - all others processors will read the 1 set by the winning processor

15. Review: Summing Numbers on a SMP
Pn is the processor’s number, vectors A and sum are shared variables, i is a private variable, half is a private variable initialized to the number of processors
sum[Pn] = 0;
for (i = 1000*Pn; i< 1000*(Pn+1); i = i + 1)
sum[Pn] = sum[Pn] + A[i];
/* each processor sums its
/* subset of vector A
repeat /* adding together the
/* partial sums
synch(); /*synchronize first
if (half%2 != 0 && Pn == 0)
sum[0] = sum[0] + sum[half-1];
half = half/2
if (Pn<half) sum[Pn] = sum[Pn] + sum[Pn+half];
until (half == 1); /*final sum in sum[0]
Divide and conquer summing – half of the processors add pairs of partial sums, the a quarter add pairs of the new partial sums, and so on.

16. An Example with 10 Processors
synch(): Processors must synchronize before the “consumer” processor tries to read the results from the memory location written by the “producer” processor
Barrier synchronization – a synchronization scheme where processors wait at the barrier, not proceeding until every processor has reached it P0 P1 P2 P3 P4 P5 P6 P7 P8 P9
sum[P0]
sum[P1]
sum[P2]
sum[P3]
sum[P4]
sum[P5]
sum[P6]
sum[P7]
sum[P8]
sum[P9]
For lecture

17. Barrier Implemented with Spin-Locks
n is a shared variable initialized to the number of processors,count is a shared variable initialized to 0, arrive and depart are shared spin-lock variables where arrive is initially unlocked and depart is initially locked
procedure synch()
lock(arrive);
count := count + 1; /* count the processors as
if count < n /* they arrive at barrier
then unlock(arrive)
else unlock(depart);
As each arriving process executes the Barrier procedure, it tests and increments a global counting variable. If the count has not yet reached n, then the processor goes into a suspended state within the Barrier procedure. As the processors arrive at the Barrier, they each become suspended and count grows. Finally, the last process to arrive finds that the count has reached n, and then releases all of the suspended processors.
lock(depart);
count := count - 1; /* count the processors as
if count > 0 /* they leave barrier
then unlock(depart)
else unlock(arrive);

18. Spin-Locks on Bus Connected ccUMAs
With bus based cache coherency, spin-locks allow processors to wait on a local copy of the lock in their caches
Reduces bus traffic – once the processor with the lock releases the lock (writes a 0) all other caches see that write and invalidate their old copy of the lock variable. Unlocking restarts the race to get the lock. The winner gets the bus and writes the lock back to 1. The other caches then invalidate their copy of the lock and on the next lock read fetch the new lock value (1) from memory.
This scheme has problems scaling up to many processors because of the communication traffic when the lock is released and contested

19. Cache Coherence Bus Traffic
Proc P0
Proc P1
Proc P2
Bus activity
Memory 1
Has lock
Spins
None 2
Releases lock (0)
Bus services P0’s invalidate 3
Cache miss
Bus services P2’s cache miss 4
Sends lock (0)
Waits
Reads lock (0)
Response to P2’s cache miss
Update memory from P0 5
Swaps lock
Bus services P1’s cache miss 6
Swap succeeds
Response to P1’s cache miss
Sends lock variable to P1 7
Swap fails
Bus services P2’s invalidate 8
Probably don’t want to cover this in detail in class – ask the class to look at it outside lecture.

20. Commercial Single Backplane Multiprocessors
MHz
BW/ system
Compaq PL
Pentium Pro 4
200
540
IBM R40
PowerPC 8
112
1800
AlphaServer
Alpha 21164 12
440
2150
SGI Pow Chal
MIPS R10000 36
195
1200
Sun 6000
UltraSPARC 30
167
2600

21. Summary
Key questions
Q1 - How do processors share data?
Q2 - How do processors coordinate their activity?
Q3 - How scalable is the architecture (what is the maximum number of processors)?
Bus connected (shared address UMA’s(SMP’s)) multi’s
Cache coherency hardware to ensure data consistency
Synchronization primitives for synchronization
Scalability of bus connected UMAs limited (< ~ 36 processors) because the three desirable bus characteristics
high bandwidth
low latency
long length
are incompatible
Network connected NUMAs are more scalable

22. Next Lecture and Reminders
Reading assignment – PH
Reminders
HW5 (and last) due Dec 1st (Part 2), Dec 6th (Part 1)
Check grade posting on-line (by your midterm exam number) for correctness
Final exam (tentatively) schedule
Tuesday, December 13th, 2:30-4:20, 22 Deike