1. GPU Programming Overview Spring 2011 류승택

2. What is a GPU? GPU stands for Graphics Processing Unit Simply – It is the processor that resides on your graphics card. GPUs allow us to achieve the unprecedented graphics capabilities now available in games (Demo: NVIDIA GTX 400)NVIDIA GTX 400

3. Introduction ■ GPGPU (General-Purpose Computation on GPUs)  The first commodity, programmable parallel architecture  GPU evolution driven by computer game market  Advantage of data-parallelism GPUs are >10x faster than CPU for appropriate problems  Advantage of commodity GPUs are inexpensive GPUs are Ubiquitous Desktops, laptops, PDAs, cell phones  Achieving this speedup Requires a large amount of GPU-specific knowledge

4. Motivation ■ Challenge Statement  GPGPU signifies the dawn of the desktop parallel computing age

5. Why Program on the GPU ? Graph from: http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/docs/CUDA_C_Programming_Guide.pdf

6. Why Program on the GPU ? ■ Compute  Intel Core i7 – 4 cores – 100 GFLOP  NVIDIA GTX280 – 240 cores – 1 TFLOP ■ Memory Bandwidth  System Memory – 60 GB/s  NVIDIA GT200 – 150 GB/s ■ Install Base  Over 200 million NVIDIA G80s shipped

7. How did this happen? ■ Games demand advanced shading ■ Fast GPUs = better shading ■ Need for speed = continued innovation ■ The gaming industry has overtaken the defense, finance, oil and healthcare industries as the main driving factor for high performance processors.

8. NVIDIA GPU Evolution Slide from David Luebke: http://s08.idav.ucdavis.edu/luebke-nvidia-gpu-architecture.pdf

9. Real-time Rendering ■ Realtime Rendering  Graphics hardware enables real-time rendering  Real-time means display rate at more than 10 images per second 3D Scene = Collection of 3D primitives (triangles, lines, points) Image = Array of pixels

10. Graphics Review ■ Modeling ■ Rendering ■ Animation

11. Graphics Review: Modeling ■ Modeling  Polygons vs Triangles How do you store a triangle mesh?  Implicit Surfaces  Height maps  …

12. Triangles Image courtesy of A K Peters, Ltd. www.virtualglobebook.com

13. Triangles Image courtesy of A K Peters, Ltd. www.virtualglobebook.com. Imagery from NASA Visible Earth: visibleearth.nasa.gov.

14. Triangles

16. Implicit Surfaces Images from GPU Gems 3: http://http.developer.nvidia.com/GPUGems3/gpugems3_ch01.html

17. Height Maps Image courtesy of A K Peters, Ltd. www.virtualglobebook.com

18. Graphics Review: Rendering ■ Rendering  Goal: Assign color to pixels ■ Two Parts  Visible surfaces What is in front of what for a given view  Shading Simulate the interaction of material and light to produce a pixel color

19. Rasterization What about ray tracing?

20. Visible Surfaces Image courtesy of A K Peters, Ltd. www.virtualglobebook.com

21. Visible Surfaces Z-Buffer / Depth Buffer Fragment vs Pixel Image courtesy of A K Peters, Ltd. www.virtualglobebook.com

22. Shading Images courtesy of A K Peters, Ltd. www.virtualglobebook.com

23. Shading Image from GPU Gems 3: http://http.developer.nvidia.com/GPUGems3/gpugems3_ch14.html

24. Graphics Pipeline Primitive Assembly Primitive Assembly Vertex Transforms Vertex Transforms Frame Buffer Frame Buffer Raster Operations Rasterization and Interpolation Scissor Test Stencil Test Depth Test Blending

25. Graphics Pipeline Images courtesy of A K Peters, Ltd. http://www.realtimerendering.com/

26. Graphics Pipeline Images courtesy of A K Peters, Ltd. http://www.realtimerendering.com/

27. Graphics Pipeline Images courtesy of A K Peters, Ltd. http://www.realtimerendering.com/

28. Graphics Pipeline Images courtesy of A K Peters, Ltd. http://www.realtimerendering.com/

29. Graphics Review: Animation ■ Move the camera and/or agents, and re-render the scene  In less than 16.6 ms (60 fps)

30. Evolution of the Programmable Graphics Pipeline ■ Pre GPU ■ Fixed function GPU ■ Programmable GPU ■ Unified Shader Processors

31. Early 90s – Pre GPU Slide from Mike Houston: http://s09.idav.ucdavis.edu/talks/01-BPS-SIGGRAPH09-mhouston.pdf

32. OpenGL Pipeline

34. GPU Shader ■ Fixed functionalities ■ Programmable functionalities ■ Flexible memory access

35. Stream Program => GPU ■ A stream is a sequence of data (could be numbers, colors, RGBA vectors, … )

36. Vertex Shader ■ Vertex transformation ■ Once per vertex ■ Input attributes  Normal  Texture coordinates  Colors

37. Geometry Shader ■ Geometry composition ■ Once per geometry ■ Input primitives  Points, lines, triangles  Lines and triangles with adjacency ■ Output primitives  Points, line strips or triangle strips  [0, n] primitives outputted

38. Fragment Shader ■ Pre-pixel (or fragment) composition ■ Once per fragment ■ Operations on interpolated values  Vertex attributes  User-defined varying variables

39. GPU Shader

40. Programming Graphics Hardware

41. PC Architecture

42. Bus Interface ■ ISA (Industry Standard Architecture)  버스 인터페이스  90 년대 초반의 XT, AT 시절부터 사용  이론적으로 최대 16Mbps 의 속도  주변기기에서의 병목현상은 심각 처리속도가 크게 문제되지 않는 사운드카드나 모뎀등을 연결하 는 정도로 쓰이고 있음 ■ PCI (Peripheral Component Interconnect)  parallel connection  ISA 후속으로 주변장치 연결을 위해 사용되고 있는 인터페이스  ISA 슬롯보다 크기가 작고 IRQ 공유  일반적인 32 비트 33MHz 는 133Mbps 의 속도, 64 비트 66MHz 는 524Mbps 속도  주변 장치 대부분이 PCI 인터페이스를 사용 ISA PCI AGP

43. Bus Interface ■ AGP (Accelerated Graphics Port)  Serial Connection (cheap, scalable)  인텔에 의해 개발  PCI 에 기반을 두고 있으나 전송 속도는 PCI 보다 두배 이상 빠름  기본적으로 66MHz 로 작동  AGP = 2 x PCI (AGP 2x = 2 x AGP) AGP 1x 방식일 경우는 최고 264Mbps AGP 2x 방식에서는 최고 533Mbps  3D 그래픽 카드용 ■ PCIe (PCI Express)  Serial Connection  최대 8.0 GB/s 의 대역폭 (PCIe = 2 x AGP x 8)  전 세계 그래픽 시장을 책임지고 있는 인텔 / ATI / NVIDIA 가 이 새로운 규격을 차세 대 그래픽 인터페이스로 확실하게 인정  기존 PCI 의 제한 때문에 탄생한 그래픽 프로세싱 유닛 (GPUs) 에 독보적 존재였던 AGP 가 PCI Express 로 대체되고 있는 상황 PCI PCIe x1 PCIe x16 GeForce 7800 GTX (PCIe x16)

44. Generation I: 3dfx Voodoo (1996) One of the first true 3D game cards Worked by supplementing standard 2D video card. Did not do vertex transformations: these were done in the CPU Did do texture mapping, z-buffering. Primitive Assembly Primitive Assembly Vertex Transforms Vertex Transforms Frame Buffer Frame Buffer Raster Operations Rasterization and Interpolation CPUGPU PCI Image from “7 years of Graphics”

45. 1995-1998: Texture Mapping and Z-Buffer - PCI: Peripheral Component Interconnect - 3dfx’s Voodoo

46. Texture Mapping

47. Texture Mapping : Perspective-Correct Interpolation

49. Aside: Mario Kart 64 ■ High fragment load / low vertex load Image from: http://www.gamespot.com/users/my_shoe/

50. Aside: Mario Kart Wii ■ High fragment load / low vertex load? Image from: http://wii.ign.com/dor/objects/949580/mario-kart-wii/images/

51. Vertex Transforms Vertex Transforms Generation II: GeForce/Radeon 7500 (1998) Main innovation: shifting the transformation and lighting calculations to the GPU Allowed multi-texturing: giving bump maps, light maps, and others.. Faster AGP bus instead of PCI Primitive Assembly Primitive Assembly Frame Buffer Frame Buffer Raster Operations Rasterization and Interpolation GPU AGP Image from “7 years of Graphics”

52. 1998: Multitexturing - AGP: Accelerated Graphics Port - NVIDIA’s TNT, ATI’s Rage

53. Multitexturing Light Mapping

54. 1999-2000: Transform and Lighting - Register Combiner: Offer many more texture/color combinations - NVIDIA’s Geforce 256 and Geforce2, ATI’s Radeon 7500)

55. Bump Mapping

56. Environment Mapping

57. Projective Texture Mapping

58. Vertex Transforms Vertex Transforms Generation III: GeForce3/Radeon 8500(2001) For the first time, allowed limited amount of programmability in the vertex pipeline Also allowed volume texturing and multi-sampling (for antialiasing) Primitive Assembly Primitive Assembly Frame Buffer Frame Buffer Raster Operations Rasterization and Interpolation GPU AGP Small vertex shaders Small vertex shaders Image from “7 years of Graphics”

59. 2001: Programmable Vertex Shader - Z-Cull: Predicts which fragments will fail the Z test and discard them - Texture Shader: Offer more texture addressing and operations - NVIDIA’s Geforce3 and Geforce4 Ti, ATI’s Radeon 8500 A programmable processor for any per-vertex computation

60. Volume Texture Mapping

61. Vertex Transforms Vertex Transforms Generation IV: Radeon 9700/GeForce FX (2002) This generation is the first generation of fully-programmable graphics cards Different versions have different resource limits on fragment/vertex programs Primitive Assembly Primitive Assembly Frame Buffer Frame Buffer Raster Operations Rasterization and Interpolation AGP Programmable Vertex shader Programmable Vertex shader Programmable Fragment Processor Programmable Fragment Processor Texture Memory Image from “7 years of Graphics” Slide from Suresh Venkatasubramanian and Joe Kider

62. 2002-2003: Programmable Pixel Shader - MRT: Multiple Render Target - NVIDIA’s Geforce FX, ATI’s Radeon 9600 to 9800 A programmable processor for any per-pixel computation

63. Shader: Static vs. Dynamic flow control ■ Static flow control  Condition varies per batch of triangles ■ Dynamic flow control  Condition varies per vertex or pixel ■ Full flow control  Static and dynamic flow control

64. Generation IV.V: GeForce6/X800 (2004) ■ Simultaneous rendering to multiple buffers ■ True conditionals and loops ■ PCIe bus ■ Vertex texture fetch Vertex Transforms Vertex Transforms Primitive Assembly Primitive Assembly Frame Buffer Frame Buffer Raster Operations Rasterization and Interpolation PCIe Programmable Vertex shader Programmable Vertex shader Programmable Fragment Processor Programmable Fragment Processor Texture Memory

65. 2004: Shader Model 3.0 and 64 bit Color Support - PCIe: Peripheral Component Interconnect Express - NVIDIA’s Geforce 6800

66. Real-time Tone Mapping ■ The image is entirely computed in 64-bit color and tone-mapped for display  64-bit color  16 bit floating-point value per channel (R, G, B, A)  Tone Mapping HDRI(High Dynamic Range Image)  low dynamic range device From low to high exposure image of the same scene

67. Generation V: GeForce8800/HD2900 (2006) Input Assembler Input Assembler Programmable Pixel Shader Programmable Pixel Shader Raster Operations Programmable Geometry Shader PCIe Programmable Vertex shader Programmable Vertex shader Output Merger Ground-up GPU redesign Support for Direct3D 10 / OpenGL 3 Geometry Shaders Stream out / transform-feedback Unified shader processors Support for General GPU programming

68. Geometry Shaders: Point Sprites

70. Geometry Shaders Image from David Blythe : http://download.microsoft.com/download/f/2/d/f2d5ee2c-b7ba-4cd0-9686-b6508b5479a1/direct3d10_web.pdf

71. NVIDIA G80 Architecture Slide from David Luebke: http://s08.idav.ucdavis.edu/luebke-nvidia-gpu-architecture.pdf

72. Why Unify Shader Processors? Slide from David Luebke: http://s08.idav.ucdavis.edu/luebke-nvidia-gpu-architecture.pdf

73. Why Unify Shader Processors? Slide from David Luebke: http://s08.idav.ucdavis.edu/luebke-nvidia-gpu-architecture.pdf

74. Unified Shader Processors Slide from David Luebke: http://s08.idav.ucdavis.edu/luebke-nvidia-gpu-architecture.pdf

75. Terminology Shader Model Direct3DOpenGLVideo card Example 292.x NVIDIA GeForce 6800 ATI Radeon X800 310.x3.x NVIDIA GeForce 8800 ATI Radeon HD 2900 411.x4.x NVIDIA GeForce GTX 480 ATI Radeon HD 5870

76. Evolution of the Programmable Graphics Pipeline Slide from Mike Houston: http://s09.idav.ucdavis.edu/talks/01-BPS-SIGGRAPH09-mhouston.pdf

77. Evolution of the Programmable Graphics Pipeline Slide from Mike Houston: http://s09.idav.ucdavis.edu/talks/01-BPS-SIGGRAPH09-mhouston.pdf

78. Vertex Index Stream 3D API Commands Assembled Primitives Pixel Updates Pixel Location Stream Programmable Fragment Processor Programmable Fragment Processor Transformed Vertices Programmable Vertex Processor Programmable Vertex Processor GPU Front End GPU Front End Primitive Assembly Primitive Assembly Frame Buffer Frame Buffer Raster Operations Rasterization and Interpolation 3D API: OpenGL or Direct3D 3D API: OpenGL or Direct3D 3D Application Or Game 3D Application Or Game Pre-transformed Vertices Pre-transformed Fragments Transformed Fragments GPU Command & Data Stream CPU-GPU Boundary (AGP/PCIe) Fixed-function pipeline

79. Vertex Index Stream 3D API Commands Assembled Primitives Pixel Updates Pixel Location Stream Programmable Fragment Processor Programmable Fragment Processor Transformed Vertices Programmable Vertex Processor Programmable Vertex Processor GPU Front End GPU Front End Primitive Assembly Primitive Assembly Frame Buffer Frame Buffer Raster Operations Rasterization and Interpolation 3D API: OpenGL or Direct3D 3D API: OpenGL or Direct3D 3D Application Or Game 3D Application Or Game Pre-transformed Vertices Pre-transformed Fragments Transformed Fragments GPU Command & Data Stream CPU-GPU Boundary (AGP/PCIe) Programmable pipeline

80. The Future ■ Unified general programming model at primitive, vertex and pixel levels ■ Scary amount of:  Floating point horsepower  Video memory  Bandwidth b/w system and video memory ■ Lower chip costs and power requirements to make 3D graphics hardware ubiquitous  Automotive (gaming, navigation, head-up displays)  Home (remotes, media center, automation)  Mobile (PDAs, cell phones)

81. Programming the GPU

82. The Evolution of GPU Programming Language

83. Programmable Pipeline

85. GPU Programming ■ GPU Programming  Low-level Language Assembler-like best performance Platform-dependent Vertex programming, Fragment programming Ex) OpenGL extensions, Direct 9  High-level shading language Easier programming Easier code reuse Easier debugging Easy to read Ex) Cg, HLSL, GLSL

86. Assembly vs. High-Level Language

87. Data Flow through Pipeline

88. GPU Programming ■ GPU Programming  Low-level Language OpenGL extensions GL_ARB_vertex_program, GL_ARB_fragment_program Direct 9 Vertex Shader 2.0, Pixel Shader 2.0  High-level shading language Cg “C for Graphics” By Nvidia HLSL “High-Level Shading Language”, Part of DirectX 9 (Microsoft) GLSL “OpenGL 2.0 Shading Language”, Proposal by 3D Labs HLSL and Cg are much more similar to each other than they are to GLSL

89. Workflow in Cg

90. Reference ■ Reference  David Luebke, General-Purpose Computation on Graphics Hardware  Daniel Weiskopf, Basic of GPU-Based Programming  Cyril Zeller, Introduction to the Hardware Graphics Pipeline  Randy Fernando, Programming the GPU  Suresh Venkatasubramanian, GPU Programming and ArchitectureGPU Programming and Architecture  GPGPU (http://www.gpgpu.org/)http://www.gpgpu.org/  GPU Programming http://euclid.uits.iupui.edu/wiki/index.php/GPU_Programming http://euclid.uits.iupui.edu/wiki/index.php/GPU_Programming  Shader::Tech http://www.shadertech.com/http://www.shadertech.com/  Nvidia Developer http://developer.nvidia.com/object/gpu_programming_guide.html http://developer.nvidia.com/object/gpu_programming_guide.html  GPGPU DEVELOPER RESOURCES GPGPU DEVELOPER RESOURCES  CIS 665: GPU Programming and Architecture : University of Pennsylvania http://www.seas.upenn.edu/~cis665/Schedule.htm http://www.seas.upenn.edu/~cis665/Schedule.htm