DDM – A Cache Only Memory Architecture Hagersten, Landin, and Haridi (1991) Presented by Patrick Eibl

Outline Basics of Cache-Only Memory Architectures The Data Diffusion Machine (DDM) DDM Coherence Protocol Examples of Replacement, Reading, Writing Memory Overhead Simulated Performance Strengths and Weaknesses Alternatives to DDM Architecture

The Big Idea: UMA→NUMA →COMA Centralized shared memory feeds data through network to individual caches Uniform access time to all memory Shared memory is distributed among processors (DASH) Data can move from home memory to other caches as needed No notion of “home” for data; moves to wherever it is needed Individual memories behave like caches

COMA: The Basics Individual memories are called Attraction Memories (AM) – each processor “attracts” its working data set AM also contains data that has never been accessed (+/-?) Uses shared memory programming model, but with no pressure to optimize static partitioning Limited duplication of shared memory The Data Diffusion Machine is the specific COMA presented here

Data Diffusion Machine Directory hierarchy allows scaling to arbitrary number of processors Branch factors and bottlenecks a consideration – Hierarchy can be split into different address domains to improve bandwidth

Coherence Protocol Transient states support split-transaction bus Fairly standard protocol with important exception of replacement, which must be managed carefully (example to come) Sequential consistency is guaranteed, but with cost that writes must wait for acknowledge before continuing Item States I: Invalid E: Exclusive S: Shared R: Reading W: Waiting RW: Reading and Waiting Bus Transactions e: erase x: exclusive r: read d: data i: inject o: out

I: Invalid S: Shared Processors Caches Directories o: out i: inject Replacement Example o o oi i id 1. A block needs to be brought into a full AM, necessitating a replacement and an out transaction 2. Out propagates up until it finds another copy of block in S, R, W, or RW 3. Out reaches top and is converted to inject, meaning this is the last copy of the data and it needs a new home 4. Inject finds space in new AM 5. Data is transferred to new home 6. States change accordingly

r r r r r r rd d d d d d d I: Invalid R: Reading A: Answering S: Shared Processors Caches Directories r: read d: data Multilevel Read Example 1. First cache issues read request 2. Read propagates up hierarchy 4. Directories change to answering state while waiting for data 3. Read reaches directory with block in shared state 5. Data moves back along same path, changing states to shared as it goes 2. Second cache issues request for same block 3. Request for same block encountered; directory simply waits for data reply from other request

e I: Invalid R: Reading W: Waiting E: Exclusive S: Shared Processors Caches Directories e: erase x: exclusive e ee e e e x x Multilevel Write Example 1. Cache issues write request 2. Erase propagates up hierarchy and back down, invalidating all other copies 5. ACK propagates back down, changing states from Waiting to Exclusive 4. Top of hierarchy responds with acknowledge 2. Second cache issues write to same block 3. Second exclusive request encounters other write to same block; first one won because it arrived first; other erase is propagated back down 4. State of second cache changed to RW, and will issue a read request before another erase (not shown)

Memory Overhead Inclusion is necessary for directories, but not for data Directories only need state bits and address tags For two sample configurations given, overheads were 6% for one-level 32-processor and 16% for two-level 256-processor Larger item size reduces overhead

Simulated Performance Minimal success on programs for which each processor operates on entire shared data MP3D was rewritten to improve performance by exploiting fact that data has no home OS, hardware, and emulator in development at the time Different DDM topology for each program (-)

Strengths Each processor attracts the data it’s using into its own memory space Data doesn’t need to be duplicated at a home node Ordinary shared memory programming model No need to optimize static partitioning (there is none) Directory hierarchy scales reasonably well Good when data is moved around in smaller chunks

Weaknesses Attraction memories hold data that isn’t being used, making them bigger and slower Different DDM hierarchy topology was used for each program in simulations Does not fully exploit large spatial locality; software wins in that case (S-COMA) Branching hierarchy is prone to bottlenecks and hotspots No way to know where data is but with expensive tree traversal (NUMA wins here)

Alternatives to COMA/DDM Flat-COMA – Blocks are free to migrate, but have home nodes with directories corresponding to physical address Simple-COMA – Allocation managed by OS and done at page granularity Reactive-NUMA – Switches between S-COMA and NUMA with remote cache on per-page basis Good summary of COMAs: