1. Some key aspects of NVIDIA GPUs and CUDA

2. Silicon Usage

3. Performance and Bandwidth

4. PCIe Connected Separate memory space on CPU (host) and GPU (device) Must malloc space on GPU and copy data from CPU memory to GPU memory Need means to copy executable to GPU and start it (kernel launch)

5. Hardware Architecture

7. CUDA Streaming Multiprocessor 32 cores assigned to 32 threads (a warp) Limited number of SFUs In-order execution Latency masked by swamping between warps of threads

8. Warp Processing (hide latency)

9. Memory Model

10. Blocks and Grids