1. Using Multiphase Shared Arrays
Jayant DeSouza
Parallel Programming Lab
University of Illinois, Urbana

2. Motivation No shared-memory facility in Charm++
General shared memory is expensive
Page faults and fetching of data
Cache coherence
Observation: data is accessed differently in phases of the program
A restricted form of shared memory could
Provide shared memory paradigm
And good performance
3/11/2004

3. Phases Read-only: page fetch, no coherence
Write-only: no page fetch, no coherence
Accumulate: no page fetch, no coherence
Prefetch, Waitall, getKnownLocal, setKnownLocal, release
3/11/2004

4. API Create Init: arr1->enroll(numWorkers); Access:
arr1= new MSA2D<double, ITEMS_PER_PAGE, ROW_MAJOR>(ROWS1, COLS1, NUM_WORKERS, bytes);
new MSA2D<…>(rows,cols,arr1->getCacheGroup());
Init: arr1->enroll(numWorkers);
Access:
arr1->get(i, j)
arr1->set(i,j) = 1.0;
arr1->accumulate(i, j, value);
Sync: arr1->sync();
3/11/2004

5. Usage Details “build LIBS”, or #include “msa/msa.h” Compile with:
cd charm/src/libs/ck-libs/multiphaseSharedArrays
make
#include “msa/msa.h”
Compile with:
charmc –module msa
Documentation: charm/doc/libraries/
Examples: charm/pgms/charm++/multiphaseSharedArrays.
3/11/2004

6. Performance Issues Every access does page-table lookup.
getKnownLocal, setKnownLocal
Size of page table.
1000 CPUs, 10 MB/CPU, 1KB/page implies 10M pointers, i.e. 40 MB page table per CPU.
We are investigating a 2-level table.
3/11/2004

7. Comparison with Global Arrays
Array data is distributed blockwise across processes. Each block is local to one process. Home can change.
No caching, no fetch.
GA has (remote) operations get, put, float accumulate, int read-increment.
No phases.
3/11/2004

8. Conclusion and Future Work
Multiphase shared arrays have been designed and implemented.
Performance studies needed.
Add support for MSA in Jade.
Extend accumulate to set-theoretic union.
Choose a nice name: MiSa ?
3/11/2004