Programming with CUDA WS 08/09 Lecture 13 Thu, 04 Dec, 2008

Today Last lecture of this half Last lecture of this half Analysis of parallel algorithms Analysis of parallel algorithms Final projects Final projects

Analyzing parallel algorithms Tightly coupled with the parallel architecture Tightly coupled with the parallel architecture Serial programs have a standard reference model, the von Neumann model Serial programs have a standard reference model, the von Neumann model

Analyzing parallel algorithms von Neumann model von Neumann model –Single CPU reads and writes to an attached memory

Analyzing parallel algorithms A parallel model supporting analysis has to be A parallel model supporting analysis has to be –Simple: understandability –Realistic: applicability

Analyzing parallel algorithms The multicomputer The multicomputer –A node is a von Neumann machine –Each node has its own program –Messages: read/write remote memory –Message cost depends on size

Analyzing parallel algorithms Distributed memory MIMD Distributed memory MIMD –Distributed memory –Similar to multicomputer, but message sending cost depends on node location and network traffic

Analyzing parallel algorithms Multiprocessor Multiprocessor –PRAM (Parallel Random Access Machine) model: all processors access memory with same cost –Special case: SIMD (CUDA)‏ Simplified hardware & software Simplified hardware & software Specialized problems Specialized problems

Analyzing parallel algorithms Network Network –Potentially high remote-access costs –New issues: reliability, security

Analyzing parallel algorithms Useful analysis tools Useful analysis tools –Brent's theorem (algorithm only)‏ –Amdahl's law –Iso-efficiency

Final Projects General guideline General guideline –Read up on the problem and possible solutions –Design your solution –Implement your solution –Optimize your solution –Analyze your solution A couple of lectures in the second half A couple of lectures in the second half

Final Exam When? When? –End of semester Submit project early Submit project early Get done with the course Get done with the course –First week of March More time to work on the project More time to work on the project Need to worry about exam during holidays Need to worry about exam during holidays –Might forget about projects in the meanwhile

See you next year? Happy holidays