1. Distributed Shared Memory
Yabstega Dubale, Cooper Scott, and Rawlin Peters
CS 455: Introduction to Distributed Systems
Computer Science Department, Colorado State University

2. Why is this problem important?
Processor speeds have hit a wall
Throw more cores at the problem
Multithreaded and multi-process programming is hard
But becoming the new norm
How can we make it easier?
Abstraction…at a cost
This led to research on distributed shared memory

3. Problem characterization
Shared global address space among a cluster
Distributed framework for the shared memory paradigm
In contrast to the message-passing interface (MPI)
Abstracts away the explicit message passing
But also introduces data replication, cache coherency, and consistency problems
Extra overhead due to cache coherency
Greatly reduces overall scalability

4. Trade-off space for solutions in this area
Granularity of data
Page-based or object-based
Too large can cause thrashing
Too small can cause extra overhead
Level of implementation
Software, Hardware, or Hybrid
Consistency models
Looser typically means better parallelism and higher throughput but less overall consistency

5. Dominant approaches to the problem (1/2)
Software-based:
Linda
Uses a shared tuple space
3-step modification (remove, change, put back)
IVY
Page-based
Master node contains info about all shared pages
Mirage
Implemented on Locus OS (UNIX-based env.)
Central queue of DSM pages for all requests

6. Dominant approaches to the problem (2/2)
Hardware-based:
SCI: Scalable Coherent Interface
Cache coherency protocols in DSM multiprocessors
Uses directories as chained lists
Hybrid-based:
PLUS
Global memory mapping, atomic operations, and coherence in hardware
Virtual address space tracked in software
Lynx
Caches global addresses in memory, and all page requests go to local memory or across the network

7. Insights Gleaned Best approach is to reduce network congestion
Reduce the total number of messages
Pass references to data instead of the actual data
Also want to reduce data contention
Smaller pages sizes
Reduce unnecessary thrashing (constantly page- transferring back and forth between nodes)
But this can lead to more metadata

8. What the problem space in the future would look like
Due to the slowing increase of processor speeds, the degree of multicores and parallelism is going up
This leads to much higher data contention and higher overhead for cache consistency
Exponential growth of data generation
Much higher overhead for data consistency
DSM will have a hard time keeping up with these changes
Will it stay relevant?

9. Trade-off space and solutions in the future
DSM will likely be phased out (might already be)
The abstraction it provides is not scalable enough
More scalable systems like MapReduce and MPI will be necessary to handle Big Data
We all experienced explicit message-passing (marshalling, TCP, etc) in HW1 and HW2
You survived
DSM is most likely not the best solution to make distributed parallel programming easy

10. THANK YOU!