Parallel Computer Architecture: Essentials for Both Computer Scientists and Engineers Edward F. Gehringer †* Yan Solihin* † Dept. of Computer Science * Department of Electrical & Computer Engineering North Carolina State University

2/12/ Outline  Why parallel computer architecture?  The students  Course organization  Example animations  Conclusion

Computer architecture courses  Most popular—processor architecture For both CE & CS  Most relevant—parallel architecture Esp. for CS 9/23/2010

Our course and curriculum 9/23/ ECE 521, Computer Design & Technology CSC/ECE 506: Arch. of Parallel Computers ECE 721, Advanced Computer Architecture CSC 706: Adv. Parallel Computer Architecture ECE 743, High Perf. Computer Architecture ECE 463, Adv. Micro- processor Design

The textbook 9/23/2010

2/12/ Outline  Why parallel computer architecture?  The students  Course organization  Example animations  Conclusion

The students  Majority are masters, but …  Both masters programs are large  Core course for CE students  Core course for CS students 9/23/2010

2/12/ Outline  Why parallel computer architecture?  The students  Course organization  Example animations  Conclusion

Syllabus 1. Concurrency 2. Bus-based multiprocessors 3. Memory consistency 4. Distributed multiprocessors 5. Interconnection networks 9/23/ classes 6 classes 2 classes 4 classes 3 classes

Concurrency  Taxonomy of parallel processors Nodes Interconnect  Models of parallel computation Shared memory Message-passing Data parallel  Parallelization constructs DOALLDOACROSSDOPIPE Function parallelism  Data-parallel programming SCANREDUCE  Parallelizing linked data structures 9/23/2010

Bus-Based Multiprocessors  The need for synchronization  Invalidation coherence protocols  Update coherence protocols  Performance aspects 9/23/2010

Memory Consistency  Contrast with cache coherence  Sequential consistency  Improving performance Prefetching Speculation  Relaxed consistency models Processor consistency Weak ordering Release consistency 9/23/2010

Distributed multiprocessors  Scalable multiprocessors  Coherence without a shared bus  Directory organization  Directory-based coherence protocols Full bit vector SCI (Scalable Coherent Interface)  Case study: Origin /23/2010

Interconnection networks  Topologies Ring Mesh Hypercube Fat tree  Routing E-cube Turn model Adaptive 9/23/2010

2/12/ Outline  Why parallel computer architecture?  The students  Course organization  Example animations  Conclusion

2/12/ Outline  Why parallel computer architecture?  The students  Course organization  Example animations  Conclusion

Conclusion  Important concept: Parallel programming Parallel architecture affect each other  Need to know this to … Write efficient programs Design an efficient architecture  This course covers “nuts and bolts”  We’d love to share … 9/23/2010