1. Spring 2006 1 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Advanced Computer Architecture Lecture 21 MSP shared cached MSI protocol MESI protocol

2. Spring 2006 2 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Symmetric multiprocessor (SMP) MemoryMemory... CPU Memory Controller CPU I/O Hub/Bridge Key Board Mouse Monitor BIOS EtherNet Power Supply Cooling Fan One address space, uniform access time

3. Spring 2006 3 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering A simpler model Each processor has a local cache, one main memory P 1 transactionsP n transactions Bus transactions

4. Spring 2006 4 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Coherency through snooping Controllers monitor bus to manage local cache Single shared memory Local controller

5. Spring 2006 5 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Coherency requirements 1.Memory operations occur in the order they were issued 2.All reads return the most current value

6. Spring 2006 6 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Cache coherency solution Monitor bus to see when things change Must maintain the “state” of each cache line –Modifed (as in write-back) –Others

7. Spring 2006 7 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Two solutions Both write-back invalidation: snooped write (another writer) invalidates local copy MSI protocol –Three states –Simpler, uses bus a bit more MESI protocol: most popular –Four states –Slightly more complex, uses bus less

8. Spring 2006 8 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering WB invalidation protocol Cache states –Modified: dirty, memory inconsistent, local cache has only valid copy, only one CPU in this state –Shared: clean, one or more copies, memory consistent –Invalid: local data is not current, stale State transitions –Determined by local controller –States may vary across caches MSI

9. Spring 2006 9 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Processor transactions Read (PrRd): read instruction Write (PrWr): write instruction Misses: if modified, must write back data to memory

10. Spring 2006 10 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Bus transactions Read (BusRd): memory read Write (BusWr): memory write Read Exclusive (BusRdX) –Used to request an exclusive copy of data –Generated by a PrWr if data Invalid or Shared –Data returned may be ignored Flush: Modified cache performs a WB, resolves inconsistency

11. Spring 2006 11 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Processor hits and misses ActionProcessorBus Read hitPrRdnone Write hitPrWrnone Read missPrRdBusRd Write missPrWrBusRdX Invalidates other local copies

12. Spring 2006 12 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Bus snooping BusAction BusRdAnother processor wants to read line BusRdXAnother processor wants to write line

13. Spring 2006 13 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering MSI protocol Bus/snoop generatedProcessor generated Controller Observes/Action write miss read miss write hit read hit Another reader Another writer

14. Spring 2006 14 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Find the coherency? ActionP 1 StateP 2 StateP 3 StateBus Data source P 1 read u SIIBusRdmem P 3 read u SISBusRd C 1 or mem P 3 writes u IIM BusRdX /Flush P3P3 P 1 read u SISBusRdC3C3 P 2 read u SSSBusRdC3C3

15. Spring 2006 15 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering MSI performance Consider single read-write –Read results in Shared state, BusRd –Write results in Modified state, BusRdX Unfortunate Result –Two bus actions –Second, BusRdX, not necessary if line is not shared with other processors –Suggest a new state, exclusive-clean

16. Spring 2006 16 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering WB invalidation protocol Improved performance vs. MSI States –Modified: dirty, memory inconsistent, local cache has only valid copy –Exclusive (clean) : local cache owns, but not written, one CPU in this state –Shared: clean, one or more copies, memory consistent –Invalid: data is not current, stale MESI

17. Spring 2006 17 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Processor transactions Read (PrRd): read instruction Write (PrWr): write instruction Misses: if modified, must write back data to memory

18. Spring 2006 18 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Bus transactions Read (BusRd): memory read Write (BusWr): memory write Read Exclusive (BusRdX) Flush: only one need provide WB data Shared (S): new signal –Determines if data already shared –Used with BusRd(S) or BusRd(S#)

19. Spring 2006 19 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering MESI protocol Controller Observes/Action Bus/snoop generatedProcessor generated

20. Spring 2006 20 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering MESI protocol Controller Observes/Action Bus/snoop generatedProcessor generated write hit read hit read miss one cache does WB Another writer Another reader no bus cycle write miss

21. Spring 2006 21 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Find the coherency? ActionP 1 StateP 2 StateP 3 StateBus Data source P 1 read u EIIBusRd(S#)Mem P 3 read u SISBusRd(S) C 1 or mem P 3 writes u IIMBusRdX/Flush C 3 or mem P 1 read u SISBusRd(S) P 2 read u SSS

22. Spring 2006 22 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering

23. Spring 2006 23 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Find the coherency? ActionP 1 StateP 2 StateP 3 StateBus Data source P 1 read u SIIBusRdMemory P 3 read u SISBusRdMemory P 3 writes u IIMBusRdXP3P3 P 1 read u SIS BusRd/F lush P 3 cache P 2 read u SSSBusRdMemory

24. Spring 2006 24 EE 437 Lillevik 437s06-l21 University of Portland School of Engineering Find the coherency? ActionP 1 StateP 2 State P 3 State Bus Data source P 1 read u EIIBusRd(S#)Memory P 3 read u SISBusRd(S)/FlushP 1 cache P 3 writes u IIMBusRdX/Flush’P3P3 P 1 read u SISBusRd(S)/FlushP 3 cache P 2 read u SSSBusRd(S)/Flush’ P 1 or P 3 cache