Presentation is loading. Please wait.

Presentation is loading. Please wait.

GLSL Applications: 1 of 2 Joseph Kider Source: Patrick Cozzi – Spring 2011 University of Pennsylvania CIS 565 - Fall 2011.

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "GLSL Applications: 1 of 2 Joseph Kider Source: Patrick Cozzi – Spring 2011 University of Pennsylvania CIS 565 - Fall 2011."— Presentation transcript:

1 GLSL Applications: 1 of 2 Joseph Kider Source: Patrick Cozzi – Spring 2011 University of Pennsylvania CIS 565 - Fall 2011

2 Agenda GLSL Applications  Per-Fragment Lighting  Image Processing Finish last week’s slides  OpenGL Drawing  OpenGL Multithreading

3 Per-Fragment Lighting Slide from http://www.opengl-redbook.com/SIGGRAPH/08/OpenGL.pdf

4 Per-Fragment Lighting Images from http://www.opengl-redbook.com/SIGGRAPH/08/OpenGL.pdf Per-Vertex LightingPer-Fragment Lighting

5 Per-Fragment Lighting: Diffuse Slide from http://www.opengl-redbook.com/SIGGRAPH/08/OpenGL.pdf

6 Per-Fragment Lighting: Diffuse Slide from http://www.opengl-redbook.com/SIGGRAPH/08/OpenGL.pdf

7 Per-Fragment Lighting: Diffuse uniform vec3 u_Color; in vec3 fs_Incident; in vec3 fs_Normal; in vec3 fs_Texcoord; out vec3 out_Color; void main(void) { vec3 incident = normalize(fs_Incident); vec3 normal = normalize(fs_Normal); float diffuse = max(0.0f, dot(-incident, normal)); out_Color = vec4(diffuse * u_Color, 1.0f); }

8 Per-Fragment Lighting: Diffuse uniform vec3 u_Color; in vec3 fs_Incident; in vec3 fs_Normal; in vec3 fs_Texcoord; out vec3 out_Color; void main(void) { vec3 incident = normalize(fs_Incident); vec3 normal = normalize(fs_Normal); float diffuse = max(0.0f, dot(-incident, normal)); out_Color = vec4(diffuse * u_Color, 1.0f); } in vectors are not normalized. Why? Good practice: don’t write to in variables

9 Per-Fragment Lighting: Diffuse uniform vec3 u_Color; in vec3 fs_Incident; in vec3 fs_Normal; in vec3 fs_Texcoord; out vec3 out_Color; void main(void) { vec3 incident = normalize(fs_Incident); vec3 normal = normalize(fs_Normal); float diffuse = max(0.0f, dot(-incident, normal)); out_Color = vec4(diffuse * u_Color, 1.0f); } Know the graph: Why max ? Graph from http://www.wsd1.org/waec/math/pre-calculus%20advanced/Trigonometry/Graphing/graphingintro.htm

10 Per-Fragment Lighting: Specular Slide from http://www.opengl-redbook.com/SIGGRAPH/08/OpenGL.pdf

11 Per-Fragment Lighting: Specular Slide from http://www.opengl-redbook.com/SIGGRAPH/08/OpenGL.pdf

12 Per-Fragment Lighting: Specular uniform vec3 u_Color; uniform vec3 u_SpecColor; uniform float u_SpecHardness; in vec3 fs_Incident; in vec3 fs_Viewer; in vec3 fs_Normal; in vec3 fs_Texcoord; out vec3 out_Color; void main(void) { vec3 incident = normalize(fs_Incident); vec3 normal = normalize(fs_Normal); vec3 H = normalize(-incident + fs_Viewer); float specular = pow(max(0.0f, dot(H, normal)), u_SpecHardness); float diffuse = max(0.0f, dot(-incident, normal)); out_Color = vec4(diffuse*u_Color + specular*u_SpecColor,1.0f); }

13 Per-Fragment Lighting: Specular uniform vec3 u_Color; uniform vec3 u_SpecColor; uniform float u_SpecHardness; in vec3 fs_Incident; in vec3 fs_Viewer; in vec3 fs_Normal; in vec3 fs_Texcoord; out vec3 out_Color; void main(void) { vec3 incident = normalize(fs_Incident); vec3 normal = normalize(fs_Normal); vec3 H = normalize(-incident + fs_Viewer); float specular = pow(max(0.0f, dot(H, normal)), u_SpecHardness); float diffuse = max(0.0f, dot(-incident, normal)); out_Color = vec4(diffuse*u_Color + specular*u_SpecColor,1.0f); } “half” vector

14 Per-Fragment Lighting: Specular uniform vec3 u_Color; uniform vec3 u_SpecColor; uniform float u_SpecHardness; in vec3 fs_Incident; in vec3 fs_Viewer; in vec3 fs_Normal; in vec3 fs_Texcoord; out vec3 out_Color; void main(void) { vec3 incident = normalize(fs_Incident); vec3 normal = normalize(fs_Normal); vec3 H = normalize(-incident + fs_Viewer); float specular = pow(max(0.0f, dot(H, normal)), u_SpecHardness); float diffuse = max(0.0f, dot(-incident, normal)); out_Color = vec4(diffuse*u_Color + specular*u_SpecColor,1.0f); } Blinn-Phong shading

15 Per-Fragment Lighting: Specular Slide from http://www.opengl-redbook.com/SIGGRAPH/08/OpenGL.pdf

16 Image Processing Our first look at GPGPU  General-Purpose computation on Graphics Processing Units Input: Image Output: Processed image A kernel runs on each pixel

17 Image Processing Examples Images from http://web.engr.oregonstate.edu/~mjb/cs519/Handouts/image.1pp.pdf Image Negative

18 Image Processing Examples Images from http://web.engr.oregonstate.edu/~mjb/cs519/Handouts/image.1pp.pdf Edge Detection

19 Image Processing Examples Images from http://web.engr.oregonstate.edu/~mjb/cs519/Handouts/image.1pp.pdf Toon Rendering

20 Image Processing Questions Is the GPU a good fit for image processing? Is image processing data-parallel? What about bus traffic? What type of shader should implement an image processing kernel?

21 Image Processing: GPU Setup Input:  Texture  Viewport-aligned quad, a.k.a full-screen quad Output: framebuffer  …for now Kernel: fragment shader

22 Image Processing: GPU Setup 1. Render viewport-aligned quad Fragment shader 2. A fragment is invoked for each screen pixel 3. Each fragment shader can access any part of the image stored as a texel: gather 4. Each fragment shader executes the kernel and writes the color to the framebuffer Images from http://web.engr.oregonstate.edu/~mjb/cs519/Handouts/image.1pp.pdf

23 Image Processing: GPU Setup 1. Render viewport-aligned quad Fragment shader 2. A fragment is invoked for each screen pixel 3. Each fragment shader can access any part of the image stored as a texel: gather 4. Each fragment shader executes the kernel and writes the color to the framebuffer Images from http://web.engr.oregonstate.edu/~mjb/cs519/Handouts/image.1pp.pdf

24 Image Processing: GPU Setup 1. Render viewport-aligned quad Fragment shader 2. A fragment is invoked for each screen pixel 3. Each fragment shader can access any part of the image stored as a texel: gather 4. Each fragment shader executes the kernel and writes the color to the framebuffer Images from http://web.engr.oregonstate.edu/~mjb/cs519/Handouts/image.1pp.pdf

25 Image Processing: GPU Setup 1. Render viewport-aligned quad Fragment shader 2. A fragment is invoked for each screen pixel 3. Each fragment shader can access any part of the image stored as a texel: gather 4. Each fragment shader executes the kernel and writes the color to the framebuffer Images from http://web.engr.oregonstate.edu/~mjb/cs519/Handouts/image.1pp.pdf

26 Image Processing: GPU Setup 1. Render viewport-aligned quad Fragment shader 2. A fragment is invoked for each screen pixel 3. Each fragment shader can access any part of the image stored as a texel: gather 4. Each fragment shader executes the kernel and writes the color to the framebuffer Images from http://web.engr.oregonstate.edu/~mjb/cs519/Handouts/image.1pp.pdf

27 Image Processing: GPU Setup How do we model the viewport-aligned quad? Two triangles? One big triangle? Screen

28 Image Processing: GPU Setup Which has more vertex shader overhead?  Does it matter? Which is simpler to implement? Which has less fragment shader overhead?

29 Image Processing: GPU Setup Triangle edges are redundantly shaded Fragments are processed in 2x2 blocks  Why?

30 Image Processing: GPU Setup Triangle edges are redundantly shaded Image and Chart from http://www.humus.name/index.php?page=News&ID=228 Number of vertices FPS FanStripMax area

31 Image Processing: GPU Setup The viewport has the same width and height as the image to be processed  The texture also has the same dimensions  How does the fragment shader access texels?

32 Image Processing: GPU Setup Store texture coordinates per vertex in vec3 Position; in vec2 Texcoords; out vec2 fs_Texcoords; void main(void) { fs_Texcoords = Texcoords; gl_Position = vec4(Position, 1.0); } Vertex Shader uniform sampler2D u_Image; in vec2 fs_Texcoords; out vec4 out_Color; void main(void) { out_Color = texture(u_Image, fs_Texcoords); } Fragment Shader

33 Image Processing: GPU Setup Store texture coordinates per vertex  What memory costs does this incur? Does it matter?  What bandwidth costs does this incur?  What non-obvious optimization does it allow?

34 Image Processing: GPU Setup Compute texture coordinate in fragment shader in vec3 Position; void main(void) { gl_Position = vec4(Position, 1.0); } Vertex Shader uniform sampler2D u_Image; uniform vec2 u_inverseViewportDimensions; out vec4 out_Color; void main(void) { vec2 txCoord = u_inverseViewportDimensions * gl_FragCoord.xy; out_Color = texture(u_Image, txCoord); } Fragment Shader

35 Image Processing: GPU Setup Compute texture coordinate in fragment shader in vec3 Position; void main(void) { gl_Position = vec4(Position, 1.0); } Vertex Shader uniform sampler2D u_Image; uniform vec2 u_inverseViewportDimensions; out vec4 out_Color; void main(void) { vec2 txCoord = u_inverseViewportDimensions * gl_FragCoord.xy; out_Color = texture(u_Image, txCoord); } Fragment Shader What is u_inverseViewportDimensions ?

36 Image Processing: GPU Setup How do you access adjacent texels? uniform sampler2D u_Image; in vec2 fs_Texcoords; out vec4 out_Color; void main(void) { vec4 c0 = texture(u_Image, fs_Texcoords); vec4 c1 = textureOffset(u_Image, fs_Texcoords, ivec2(-1, 0)); vec4 c2 = textureOffset(u_Image, fs_Texcoords, ivec2( 1, 0)); vec4 c3 = textureOffset(u_Image, fs_Texcoords, ivec2( 0, -1)); vec4 c4 = textureOffset(u_Image, fs_Texcoords, ivec2( 0, 1)); out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2; }

37 Image Processing: GPU Setup How do you access adjacent texels? uniform sampler2D u_Image; in vec2 fs_Texcoords; out vec4 out_Color; void main(void) { vec4 c0 = texture(u_Image, fs_Texcoords); vec4 c1 = textureOffset(u_Image, fs_Texcoords, ivec2(-1, 0)); vec4 c2 = textureOffset(u_Image, fs_Texcoords, ivec2( 1, 0)); vec4 c3 = textureOffset(u_Image, fs_Texcoords, ivec2( 0, -1)); vec4 c4 = textureOffset(u_Image, fs_Texcoords, ivec2( 0, 1)); out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2; } (-1, 0)(1, 0) (0, -1) (0, 1)

38 Image Processing: GPU Setup textureOffset requires constants  E.g. ivec2(x, y) is not allowed How else do you access adjacent texels?

39 Image Processing: GPU Setup How else do you access adjacent texels? uniform sampler2D u_Image; uniform vec2 u_inverseViewportDimensions; out vec4 out_Color; void main(void) { vec2 txCoord = u_inverseViewportDimensions * gl_FragCoord.xy; vec2 delta = 1.0 / textureSize(u_Image); vec4 c0 = texture(u_Image, txCoord); vec4 c1 = texture(u_Image, txCoord + (delta * vec2(-1.0, 0.0))); vec4 c2 = texture(u_Image, txCoord + (delta * vec2( 1.0, 0.0))); vec4 c3 = texture(u_Image, txCoord + (delta * vec2( 0.0, -1.0))); vec4 c4 = texture(u_Image, txCoord + (delta * vec2( 0.0, 1.0))); out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2; }

40 Image Processing: GPU Setup How else do you access adjacent texels? uniform sampler2D u_Image; uniform vec2 u_inverseViewportDimensions; out vec4 out_Color; void main(void) { vec2 txCoord = u_inverseViewportDimensions * gl_FragCoord.xy; vec2 delta = 1.0 / textureSize(u_Image); vec4 c0 = texture(u_Image, txCoord); vec4 c1 = texture(u_Image, txCoord + (delta * vec2(-1.0, 0.0))); vec4 c2 = texture(u_Image, txCoord + (delta * vec2( 1.0, 0.0))); vec4 c3 = texture(u_Image, txCoord + (delta * vec2( 0.0, -1.0))); vec4 c4 = texture(u_Image, txCoord + (delta * vec2( 0.0, 1.0))); out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2; }

41 Image Processing: GPU Setup How else do you access adjacent texels? uniform sampler2D u_Image; uniform vec2 u_inverseViewportDimensions; out vec4 out_Color; void main(void) { vec2 txCoord = u_inverseViewportDimensions * gl_FragCoord.xy; vec2 delta = 1.0 / textureSize(u_Image); vec4 c0 = texture(u_Image, txCoord); vec4 c1 = texture(u_Image, txCoord + (delta * vec2(-1.0, 0.0))); vec4 c2 = texture(u_Image, txCoord + (delta * vec2( 1.0, 0.0))); vec4 c3 = texture(u_Image, txCoord + (delta * vec2( 0.0, -1.0))); vec4 c4 = texture(u_Image, txCoord + (delta * vec2( 0.0, 1.0))); out_Color = (c0 + c1 + c2 + c3 + c4) * 0.2; } (-1, 0)(1, 0) (0, -1) (0, 1)

42 Image Processing: Kernel Examples Image Negative uniform sampler2D u_Image; in vec2 fs_Texcoords; out vec4 out_Color; void main(void) { out_Color = vec4(1.0) - texture(u_Image, fs_Texcoords); } Images from http://web.engr.oregonstate.edu/~mjb/cs519/Handouts/image.1pp.pdf

43 Image Processing: Kernel Examples Image from http://www.librow.com/articles/article-9 2D Gaussian Gaussian Blur

44 Filter for 3x3 Gaussian Blur: [1 2 1] 1/16 * [2 4 2] [1 2 1] The elements add to one Other filters are also used for Edge detection Sharpen Emboss … Image Processing: Kernel Examples

45 Gaussian Blur  How would you implement the fragment shader?  How is the memory coherence? 3x3, 5x5, etc.

46 Image Processing: Kernel Examples Image from http://www.ozone3d.net/tutorials/image_filtering_p2.php

47 Image Processing: Kernel Examples What does this filter do? [1 1 1] 1/9 * [1 1 1] [1 1 1]

48 Image Processing: Read backs How do we get the contents of the framebuffer into system memory?  Print Screen? It doesn’t matter if we are using: Efficient read backs are important

49 Image Processing: Read backs glReadPixels glUseProgram(/*... */); glDraw*(/*... */); unsigned char rgb = new unsigned char[width * height * 3]; glReadPixels(0, 0, width, height, GL_RGB, GL_UNSIGNED_BYTE, rgb); //... delete [] rgb;

50 Image Processing: Read backs glReadPixels glUseProgram(/*... */); glDraw*(/*... */); unsigned char rgb = new unsigned char[width * height * 3]; glReadPixels(0, 0, width, height, GL_RGB, GL_UNSIGNED_BYTE, rgb); //... delete [] rgb; Use GPU for image processing

51 Image Processing: Read backs glReadPixels glUseProgram(/*... */); glDraw*(/*... */); unsigned char rgb = new unsigned char[width * height * 3]; glReadPixels(0, 0, width, height, GL_RGB, GL_UNSIGNED_BYTE, rgb); //... delete [] rgb; Allocate buffer for processed image

52 Image Processing: Read backs glReadPixels glUseProgram(/*... */); glDraw*(/*... */); unsigned char rgb = new unsigned char[width * height * 3]; glReadPixels(0, 0, width, height, GL_RGB, GL_UNSIGNED_BYTE, rgb); //... delete [] rgb; Ask for framebuffer’s color buffer

53 Image Processing: Read backs glReadPixels glUseProgram(/*... */); glDraw*(/*... */); unsigned char rgb = new unsigned char[width * height * 3]; glReadPixels(0, 0, width, height, GL_RGB, GL_UNSIGNED_BYTE, rgb); //... delete [] rgb; You guys are sharp

54 Image Processing: Read backs What is the major problem with glReadPixels ?

55 Image Processing: Use Cases Photoshop-type applications Post-processing in games On the fly video manipulation Augmented reality These last three don’t even need read backs

Download ppt "GLSL Applications: 1 of 2 Joseph Kider Source: Patrick Cozzi – Spring 2011 University of Pennsylvania CIS 565 - Fall 2011."

Similar presentations

POST-PROCESSING SET09115 Intro Graphics Programming.

POST-PROCESSING SET09115 Intro Graphics Programming.
CS123 | INTRODUCTION TO COMPUTER GRAPHICS Andries van Dam © 1/16 Deferred Lighting Deferred Lighting – 11/18/2014.

CS123 | INTRODUCTION TO COMPUTER GRAPHICS Andries van Dam © 1/16 Deferred Lighting Deferred Lighting – 11/18/2014.
Understanding the graphics pipeline Lecture 2 Original Slides by: Suresh Venkatasubramanian Updates by Joseph Kider.

Understanding the graphics pipeline Lecture 2 Original Slides by: Suresh Venkatasubramanian Updates by Joseph Kider.
The Graphics Pipeline Patrick Cozzi University of Pennsylvania CIS 565 - Fall 2012.

The Graphics Pipeline Patrick Cozzi University of Pennsylvania CIS Fall 2012.
TEXTURE MAPPING JEFF CHASTINE 1. TEXTURE MAPPING Applying an image (or a texture ) to geometry 2D images (rectangular) 3D images (volumetric – such as.

TEXTURE MAPPING JEFF CHASTINE 1. TEXTURE MAPPING Applying an image (or a texture ) to geometry 2D images (rectangular) 3D images (volumetric – such as.
Introduction to Geometry Shaders Patrick Cozzi Analytical Graphics, Inc.

Introduction to Geometry Shaders Patrick Cozzi Analytical Graphics, Inc.
Control Flow Virtualization for General-Purpose Computation on Graphics Hardware Ghulam Lashari Ondrej Lhotak University of Waterloo.

Control Flow Virtualization for General-Purpose Computation on Graphics Hardware Ghulam Lashari Ondrej Lhotak University of Waterloo.
Announcments Sign up for crits!. Reading for Next Week FvD 16.1-16.3 – local lighting models GL 5 – lighting GL 9 (skim) – texture mapping.

Announcments Sign up for crits!. Reading for Next Week FvD – local lighting models GL 5 – lighting GL 9 (skim) – texture mapping.
GLSL I May 28, 2007 (Adapted from Ed Angel’s lecture slides)

GLSL I May 28, 2007 (Adapted from Ed Angel’s lecture slides)
GLSL I Ed Angel Professor of Computer Science, Electrical and Computer Engineering, and Media Arts Director, Arts Technology Center University of New Mexico.

GLSL I Ed Angel Professor of Computer Science, Electrical and Computer Engineering, and Media Arts Director, Arts Technology Center University of New Mexico.
Introduction to Geometry Shaders Patrick Cozzi Analytical Graphics, Inc.

Introduction to Geometry Shaders Patrick Cozzi Analytical Graphics, Inc.
GLSL Applications: 2 of 2 Patrick Cozzi University of Pennsylvania CIS 565 - Spring 2011.

GLSL Applications: 2 of 2 Patrick Cozzi University of Pennsylvania CIS Spring 2011.
Status – Week 277 Victor Moya.

Status – Week 277 Victor Moya.
Evolution of the Programmable Graphics Pipeline Patrick Cozzi University of Pennsylvania CIS 565 - Spring 2011.

Evolution of the Programmable Graphics Pipeline Patrick Cozzi University of Pennsylvania CIS Spring 2011.
Mohan Sridharan Based on slides created by Edward Angel GLSL I 1 CS4395: Computer Graphics.

Mohan Sridharan Based on slides created by Edward Angel GLSL I 1 CS4395: Computer Graphics.
GEOMETRY SHADER. Breakdown  Basics  Review – graphics pipeline and shaders  What can the geometry shader do?  Working with the geometry shader  GLSL.

GEOMETRY SHADER. Breakdown  Basics  Review – graphics pipeline and shaders  What can the geometry shader do?  Working with the geometry shader  GLSL.
Image Processing.  a typical image is simply a 2D array of color or gray values  i.e., a texture  image processing takes as input an image and outputs.

Image Processing.  a typical image is simply a 2D array of color or gray values  i.e., a texture  image processing takes as input an image and outputs.
Michael Robertson Yuta Takayama. WebGL is OpenGL on a web browser. OpenGL is a low-level 3D graphics API Basically, WebGL is a 3D graphics API that generates.

Michael Robertson Yuta Takayama. WebGL is OpenGL on a web browser. OpenGL is a low-level 3D graphics API Basically, WebGL is a 3D graphics API that generates.
Queensland University of Technology CRICOS No. 000213J INB382/INN382 Real-Time Rendering Techniques Lecture 13: Revision Ross Brown.

Queensland University of Technology CRICOS No J INB382/INN382 Real-Time Rendering Techniques Lecture 13: Revision Ross Brown.
REAL-TIME VOLUME GRAPHICS Christof Rezk Salama Computer Graphics and Multimedia Group, University of Siegen, Germany Eurographics 2006 Real-Time Volume.

REAL-TIME VOLUME GRAPHICS Christof Rezk Salama Computer Graphics and Multimedia Group, University of Siegen, Germany Eurographics 2006 Real-Time Volume.

Similar presentations

Ads by Google