1. [CSCI-4800/5800] - Shaders and GPU Programming Professor Min-Hyung Choi January 20, 2016

2. Welcome to Shaders and GPU Programming  Instructor: Dr. Min Choi ‒ Website: cse.ucdenver.edu/~mchoi ‒ Office: Lawernce St. Center LW-812 ‒ Office Hours: MW: 2:00[pm] – 3:30[pm] ‒ Graphics Lab: MF: 9:00[am] – 7:00[pm]  Prerequisites: CSCI-2421 – Data Structures and Program Design ‒ Co-requisites: N/A ‒ Required Knowledge-base: Trigonometry, Linear Algebra, C++, Graphics  Course Format: Hybrid (Online/Lecture) ‒ HW (Programming Assignments) 60% ‒ Final Project (Project Proposal/Implementation) 40%  Course Website: graphics.ucdenver.edu/SGPU/

3. Shader and GPU Programming  Required Textbook: OpenGL 4 Cookbook ‒ Author: David Wolff ‒ Publisher: Packt Publishing: 2 nd Revised Edition ‒ ISBN-10: 1782167021 ‒ ISBN-13: 978-1782167020 ‒ Recommended Textbooks: ‒ OpenGL SuperBible  Author: Graham Sellers, et al.  ISBN-10: 0321902947 ‒ Interactive Computer Graphics ‒ Author: Edward Angel and D. Shreiner  ISBN-10: 0132545233 ‒ OpenGL Shading Language  Author: Randi J. Rost et al.  ISBN-10: 0321637631

4. Shader and GPU Programming  Course Objective: Build experience in shader programming and GPU programming using existing examples to produce high level, interactive, graphical applications.  Use existing examples to build high level interactive applications  C++/OpenGL Shading (Using newest OpenGL versions)  Qt-based Interactive GUI Applications  Final Project: Application in imaging, graphics, simulation, or virtual reality  Explore the history and development of shader development and the introduction of General Purpose GPU programming (GPGPU)  Introduction to shaders with the Graphics Library Shading Language (GLSL)  Shaders for general purpose parallel computation  Transition from shaders to GPGPU languages (OpenCL/CUDA)

5. Shader and GPU Programming  Course Topics: ‒ Graphics Pipeline and Rendering ‒ Graphics Processing Unit (GPU) Architecture ‒ 3D Mathematics and Modeling ‒ Graphics and Special Effect Shaders ‒ Lighting, Illumination, and Shadowing ‒ General Purpose GPU Parallel Programming (GPGPU) ‒ Applications:  Animation: Movies, Game design  Game Design: Graphics, Real-time rendering  Scientific Visualization: Data visualization, model visualization  Physical Simulation: Rigid bodies, deformable objects, fluids, etc.  Virtual Reality: Augmented reality, interactive visualization  GPGPU: Numerical Algorithms, Simulation, etc. Cloth Simulation [1]

6. Course Structure  Programming Assignments (60%) ‒ Programming in C++ with OpenGL (v. > 4.0) ‒ Provided Libraries  Mathematics (Templated Linear Algebra)  Graphics Library (Meshes, Texturing, etc.) ‒ Visual Studio 2013 ‒ Graphical User Interface (GUI) Programming  OpenGL/Qt  Final Project (40%) ‒ Project Proposal ‒ Project Implementation  Can be based on provided example code/shaders  Broader Application Domain (Graphics, Simulation, Virtual Reality, etc.) Grading Scale: 90.0 - 100 % A/A+ 80.0 - 89.9 % B/B+ 70.0 – 79.9 % C/C+ 60.0 – 69.9 % D/D+ < 60.0 % F

7. What is Shading? Crytek Engine

8. What is Shading? Unreal Engine 4 (UT4)

9. What is Shading? Game Engines  Target both visual effects and visual quality for real-time rendering  Extremely common terminology for describing ‘high-quality’ graphics  Massive form of parallel computation Animation  Generate high quality images  Photorealistic environments  Advanced Lighting  Complex Surface Materials  Surface Reflections UE4 High Fidelity Interior Render

10. What is Shading?  What is shading?  Initially: Perception of 3D objects ‒ Originally based in Lighting Modeling ‒ Rasterization of 3D objects with lighting ‒ Rendering realistic effects  Currently: Advanced 3D Effects ‒ Motion Blur/Depth of Field etc. ‒ Lighting/Light Rays/Bloom/Emission ‒ Shadowing ‒ Reflection/Refraction ‒ Bump mapping/Displacement Mapping etc.  What is a shader? ‒ A shader is a set of techniques and graphics resources that incorporate several graphics algorithms to be executed on a Graphical Processing Unit (GPU) to efficiently render environmental materials and complex visual effects in games, animation, virtual reality, and scientific visualization. Simple Lighting-based Shading (Sketch [vs] Rendered)

11. What is Shading?  Primitive Components of Shading ‒ Vertex Data (Loaded from Model)  Vertex Position – Position of every 3D point that the model is made of  Surface Normals – Defines the orientation of the surface for each face  Texture Coordinates – Defines how a texture is mapped to a surface  Color – Red, Blue, Green (RGB) color of each vertex ‒ Vertex Connectivity (Loaded from Model)  Triangles  Quads  Polygons ‒ Fragment Generation  Rasterization  Interpolation

12. (Fixed) Graphics Pipeline Overview Detailed Graphics Pipeline: From 3D model to 2D Screen display

13. (Fixed) Graphics Pipeline Overview CPUs and GPUs share resources but have very different execution models. Shaders provide a means of implementing massively parallel programs on a GPU.

14. Graphics Pipeline Overview What is Shading: Basic Shader Pipeline Process of rasterizing a shaded triangle and quad based on a set of vertex colors. All shaders have been derived from this simple principle.

15. (Programmable) Graphics Pipeline Overview The modifiable components within the graphics pipeline are the Vertex shader and the Fragment shader. These are the programmable sections of the graphics pipeline.

16. Programmable Graphics Shaders  Graphics Shaders ‒ Heavily rely on mathematical derivations of physical phenomenon ‒ Introduce textures and techniques for producing advanced effects ‒ Massively parallel computation for real-time applications ‒ Extensive amount of research for over 20 years ‒ Creativity + Math = Shader Design

17. Normal Mapping 1.) Design low-poly mesh 2.) Improve detail with high resolution 3.) Bake high resolution to normal texture 4.) Normal texture defines surface detail 5.) Fast high-quality models Modeling Applications: 3ds Max 2016 Z Brush 4

18. Parallax Mapping Parallax mapping provides the illusion of large surface offsets using a displacement texture. This technique has been used extensively in real-time applications and game design.

19. Displacement Mapping Displacement mapping illustrates the potential of real-time tessellation: the low poly mesh is subdivided during runtime to reproduce the visual quality of the high poly model.

20. Displacement Mapping Displacement mapping illustrates the potential of real-time tessellation: the low poly mesh is subdivided during runtime to reproduce the visual quality of the high poly model.

21. Advanced Lighting: Water Shaders Even 3D scenes composed of simplistic geometry and low resolution textures can be drastically improved using shaders (Minecraft)

22. Advanced Lighting: Water Shaders A primary research direction in shader programming has focused on the accurate rendering of water and fluids within physical simulations

23. Shader Library  Objective: Provide an implementation of all the basic graphics components required to load and view OpenGL shaders (Graphics + Mathematics). ‒ Provides high level abstractions to accelerate shader implementations  Shifts the focus from loading resources to writing GLSL code  Simplifies the process of writing high-level applications ‒ Shaders rely on complex data structures and initial calculations of various geometric properties  Surface Normals, Tangents, etc. ‒ Shaders require an extensive set of resources including models and textures  Model loading  Texture loading  Source code provided on course website

24. Graphics Library  Shader Library: A graphics library that facilitates the basic operations required for any interactive graphics application. This library includes the following components: ‒ 3D Camera: Provides the ability to change the view of the loaded 3D model. This is critically important for inspecting and viewing complex shader effects. ‒ Mesh Loading: Countless model file formats provide textual representations of 3D models. The provided implementation ‒ Shader Loading: The process of loading, compiling, and linking shaders can be simplified to build higher-level applications ‒ Texture Loading: All higher level shaders require textures of some kind. From lighting to normal mapping, to displacement mapping, parallax mapping and others

25. Mathematics Library  Templated Linear Algebra Library: A mathematics library that can be used by only copying and pasting it into a working project. The templated implementation allows for vectors and matrices of type float and double This library includes the following primary components: ‒ Vectors: Mathematical representations for points, displacements, surface normals, and directions. ‒ Matrices: Used to store rotations and transformations of 3D objects ‒ Quaternions: Used to represent rotations in 3D space as rotations in the complex plane. Used within most game engines and animation packages. ‒ Transformations: Provides an easy to use interface for moving, scaling, and rotating objects in 3D space

26. Programmable Shader Example (OpenGL 4.X) Programmable Shading: Standard Light Model Shader (Metal Teapot)  Example: Phong Lighting Model + Texture + Specular Map + Normal Map  Methodology used in many games, modeling, and animation software  Example: How to model, light, and texture a shaded 3D object  Implemented using the C++ source code for this course  Uses GLSL and the programmable graphics pipeline 1.Loaded Model Geometry (Teapot.obj) 2.No lighting 3.Diffuse Lighting (Single light source) 4.Specular Mapping (Surface highlights) 5.Normal + Specular Mapping (Surface roughness, Surface highlights)

27. Graphical Shader Overview Standard Light Model Shader (Teapot Geometry – No lighting)

28. Graphical Shader Overview Standard Light Model Shader (Textured)

29. Graphical Shader Overview Standard Light Model Shader (Diffuse Lighting)

30. Graphical Shader Overview Standard Light Model Shader (Specular Mapping)

31. Graphical Shader Overview Standard Light Model Shader (Bump Mapping + Specular Mapping)

32. Graphical Shader Overview Standard Light Model Shader Result (Bump Mapping + Specular Mapping)

33. Extending Shading to other Applications  Shading Not Limited to Graphics ‒ Lighting and Modeling in Simulation ‒ Procedural Textures ‒ Graphical Effects ‒ Scientific Visualization ‒ Augmented Reality

34. References [1] Duksu Kim et al. Hybrid Parallel Continuous Collision Detection using CPUs and GPUs, Computer Graphics Forum, 2009. [2] Unreal Engine 4, Free Materials Mini Pack, https://forums.unrealengine.com/showthread.php?6980-Free- materials-mini-PACK, 2014.