1. 1 Introduction to Computer Graphics with WebGL ADOPTED from Ed Angel Chapter 2 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

2. 2 References Interactive Computer Graphics (7 th Edition) The OpenGL Programmer’s Guide (the Redbook) 8 th Edition OpenGL ES 2.0 Programming Guide WebGL Programming Guide WebGL Beginner’s Guide WebGL: Up and Running Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

3. 3 Web Resources www.cs.unm.edu/~angel/ www.cs.unm.edu/~angel/WebGL/7E www.opengl.org get.webgl.org www.kronos.org/webgl www.chromeexperiments.com/webgl learningwebgl.com Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

4. JavaScript Notes 4 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

5. JavaScript Notes JavaScript (JS) is the language of the Web ­All browsers will execute JS code ­JavaScript is an interpreted object-oriented language References ­Flanagan, JavaScript: The Definitive Guide, O’Reilly ­Crockford, JavaScript, The Good Parts, O’Reilly ­Many Web tutorials 5 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

6. JS Notes Very few native types: ­numbers ­strings ­booleans Only one numerical type: 32 bit float ­var x = 1; ­var x = 1.0; // same ­potential issue in loops ­two operators for equality == and === Dynamic typing 6 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

7. Scoping Different from other languages Function scope variables are hoisted within a function ­can use a variable before it is declared Note functions are first class objects in JS 7 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

8. JS Arrays JS arrays are objects ­inherit methods ­var a = [1, 2, 3]; is not the same as in C++ or Java ­a.length // 3 ­a.push(4); // length now 4 ­a.pop(); // 4 ­avoids use of many loops and indexing ­Problem for WebGL which expects C-style arrays 8 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

9. Typed Arrays JS has typed arrays that are like C arrays var a = new Float32Array(3) var b = new Uint8Array(3) Generally, we prefer to work with standard JS arrays and convert to typed arrays only when we need to send data to the GPU with the flatten function in MV.js 9 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

10. From WebGL to GPU 10 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

11. WebGL Data Object Source Code Control VariablesTriangles/Polygons Arrays (xyz, color, normal for each vertex) Geometry (x, y, z) Color (K λ ) Normals (n) GLSL (GL Shading Language) + HTML5 + JavaScript (C on GPU) Copy toBuffers Code Text Memory Compile/Load 1 array element for 1 Stream Processor (SP) Each array index is mapped to a SP ID. Each element is pointed by an attribute pointer (vPosition, vColor, vNormal). Parallel Processing - SIMD (1:m) Each SP processes 1 array element, then 1 pixel. CPU GPU (Vertex Shader & Fragment/Pixel Shader)

12. 12 Identified table GPU Vertex shader Pixel shader xyz...... rgb IdNameLocation 1Vsource 2Vobj 3Vposition 4Vcolour 5Psource 6Pobj CP U Colours: "Host" Memoy(data): main() Id: 12...... Points: xyz......rgb H.W CPU Code main()...... 1010....... David Kirk/NVIDIA and Wen-mei W. Hwu, University of Illinois, Urbana-Champaign

13. 13 Practical Approach Process objects one at a time in the order they are generated by the application ­Can consider only local lighting Pipeline architecture All steps can be implemented in hardware on the graphics card application program display Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

14. Execution in Browser 14 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

15. GLSL https://www.opengl.org/documentation/glsl/ https://www.opengl.org/documentation/glsl/ 15 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

16. GLSL OpenGL Shading Language C-like with ­Matrix and vector types (2, 3, 4 dimensional) ­Overloaded operators ­C++ like constructors Similar to Nvidia’s Cg and Microsoft HLSL Code sent to shaders as source code WebGL functions compile, link and get information to shaders 16 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

17. Shaders We assign names to the shaders that we can use in the JS file These are trivial pass-through (do nothing) shaders that which set the two required built-in variables ­gl_Position ­gl_FragColor Note both shaders are full programs Note vector type vec2 Must set precision in fragment shader Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

18. A Simple WebGL Example http://pausch.cs.uakron.edu/~xiao/WebGL/Chap3/triangle.html http://pausch.cs.uakron.edu/~xiao/WebGL/Chap3/triangle.html 18 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

19. Example: triangle.html 19 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

20. Example Code 20 attribute vec4 vPosition; void main(){ gl_Position = vPosition; } precision mediump float; void main(){ gl_FragColor = vec4( 1.0, 0.0, 0.0, 1.0 ); } Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

21. HTML File (cont) 21 Oops... your browser doesn't support the HTML5 canvas element Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

22. JS File 22 var gl; var points; window.onload = function init(){ var canvas = document.getElementById( "gl-canvas" ); gl = WebGLUtils.setupWebGL( canvas ); if ( !gl ) { alert( "WebGL isn't available" ); } // Three Vertices var vertices = [ vec2( -1, -1 ), vec2( 0, 1 ), vec2( 1, -1 ) ]; Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

23. JS File (cont) 23 // Configure WebGL // gl.viewport( 0, 0, canvas.width, canvas.height ); gl.clearColor( 1.0, 1.0, 1.0, 1.0 ); // Load shaders and initialize attribute buffers var program = initShaders( gl, "vertex-shader", "fragment-shader" ); gl.useProgram( program ); // Load the data into the GPU var bufferId = gl.createBuffer(); gl.bindBuffer( gl.ARRAY_BUFFER, bufferId ); gl.bufferData( gl.ARRAY_BUFFER, flatten(vertices), gl.STATIC_DRAW ); Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

24. JS File (cont) 24 // Associate out shader variables with our data buffer var vPosition = gl.getAttribLocation( program, "vPosition" ); gl.vertexAttribPointer( vPosition, 2, gl.FLOAT, false, 0, 0 ); gl.enableVertexAttribArray( vPosition ); render(); }; function render() { gl.clear( gl.COLOR_BUFFER_BIT ); gl.drawArrays( gl.TRIANGLES, 0, 3 ); } Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

25. Coding WebGL and Shaders 25 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

26. Files../Common/webgl-utils.js : Standard utilities for setting up WebGL context in Common directory on website../Common/initShaders.js : contains JS and WebGL code for reading, compiling and linking the shaders../Common/MV.js : our matrix-vector package triangle.js : the application file Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

27. Notes initShaders used to load, compile and link shaders to form a program object Load data onto GPU by creating a vertex buffer object on the GPU ­Note use of flatten() to convert JS array to an array of float32’s Finally we must connect variable in program with variable in shader ­need name, type, location in buffer Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

28. 28 Simple Vertex Shader attribute vec4 vPosition; void main(void) { gl_Position = vPosition; } Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

29. 29 Simple Fragment Program precision mediump float; void main(void) { gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0); } Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

30. Coordinate Systems and Shaders Vertex shader must output in clip coordinates Input to fragment shader from rasterizer is in window coordinates Application can provide vertex data in any coordinate system but shader must eventually produce gl_Position in clip coordinates Simple example uses clip coordinates Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

31. WebGL Camera WebGL places a camera at the origin in object space pointing in the negative z direction The default viewing volume is a box centered at the origin with sides of length 2 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

32. Orthographic Viewing z=0 In the default orthographic view, points are projected forward along the z axis onto the plane z=0 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

33. Viewports Do not have use the entire window for the image: gl.viewport(x,y,w,h) Values in pixels (window coordinates) Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

34. 34 Fragment Shader Applications Per fragment lighting calculations per vertex lighting per fragment lighting Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

35. 35 Fragment Shader Applications Texture mapping smooth shadingenvironment mapping bump mapping Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

36. A More Advanced Example https://www.khronos.org/webgl/wiki/Tutorial https://www.khronos.org/webgl/wiki/Tutorial 36 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

37. Programming Shaders 37 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

38. 38 Data Types C types: int, float, bool Vectors: ­float vec2, vec3, vec4 ­Also int (ivec) and boolean (bvec) Matrices: mat2, mat3, mat4 ­Stored by columns ­Standard referencing m[row][column] C++ style constructors ­vec3 a =vec3(1.0, 2.0, 3.0) ­vec2 b = vec2(a) Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

39. 39 No Pointers There are no pointers in GLSL We can use C structs which can be copied back from functions Because matrices and vectors are basic types they can be passed into and output from GLSL functions, e.g. mat3 func(mat3 a) variables passed by copying Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

40. 40 Qualifiers GLSL has many of the same qualifiers such as const as C/C++ Need others due to the nature of the execution model Variables can change ­Once per primitive ­Once per vertex ­Once per fragment ­At any time in the application Vertex attributes are interpolated by the rasterizer into fragment attributes Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

41. 41 Attribute Qualifier Attribute-qualified variables can change at most once per vertex There are a few built in variables such as gl_Position but most have been deprecated User defined (in application program) ­attribute float temperature ­attribute vec3 velocity ­recent versions of GLSL use in and out qualifiers to get to and from shaders Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

42. 42 Uniform Qualified Variables that are constant for an entire primitive Can be changed in application and sent to shaders Cannot be changed in shader Used to pass information to shader such as the time or a bounding box of a primitive or transformation matrices Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

43. 43 Varying Qualified Variables that are passed from vertex shader to fragment shader Automatically interpolated by the rasterizer With WebGL, GLSL uses the varying qualifier in both shaders varying vec4 color; More recent versions of WebGL use out in vertex shader and in in the fragment shader out vec4 color; //vertex shader in vec4 color; // fragment shader Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

44. Our Naming Convention attributes passed to vertex shader have names beginning with v (v Position, vColor) in both the application and the shader ­Note that these are different entities with the same name Variable variables begin with f (fColor) in both shaders ­must have same name Uniform variables are unadorned and can have the same name in application and shaders 44 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

45. 45 Example: Vertex Shader attribute vec4 vColor; varying vec4 fColor; void main() { gl_Position = vPosition; fColor = vColor; } Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

46. 46 Corresponding Fragment Shader precision mediump float; varying vec3 fColor; void main() { gl_FragColor = fColor; } Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

47. Sending Colors from Application 47 var cBuffer = gl.createBuffer(); gl.bindBuffer( gl.ARRAY_BUFFER, cBuffer ); gl.bufferData( gl.ARRAY_BUFFER, flatten(colors), gl.STATIC_DRAW ); var vColor = gl.getAttribLocation( program, "vColor" ); gl.vertexAttribPointer( vColor, 3, gl.FLOAT, false, 0, 0 ); gl.enableVertexAttribArray( vColor ); Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

48. Sending a Uniform Variable 48 // in application vec4 color = vec4(1.0, 0.0, 0.0, 1.0); colorLoc = gl.getUniformLocation( program, ”color" ); gl.uniform4f( colorLoc, color); // in fragment shader (similar in vertex shader) uniform vec4 color; void main() { gl_FragColor = color; } Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

49. 49 Operators and Functions Standard C functions ­Trigonometric ­Arithmetic ­Normalize, reflect, length Overloading of vector and matrix types mat4 a; vec4 b, c, d; c = b*a; // a column vector stored as a 1d array d = a*b; // a row vector stored as a 1d array Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

50. 50 Swizzling and Selection Can refer to array elements by element using [] or selection (.) operator with ­x, y, z, w ­r, g, b, a ­s, t, p, q ­a[2], a.b, a.z, a.p are the same Swizzling operator lets us manipulate components vec4 a, b; a.yz = vec2(1.0, 2.0, 3.0, 4.0); b = a.yxzw; Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

51. Linking Shaders with Application 51 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

52. Linking Shaders with Application Read shaders Compile shaders Create a program object Link everything together Link variables in application with variables in shaders ­Vertex attributes ­Uniform variables 52 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

53. Program Object Container for shaders ­Can contain multiple shaders ­Other GLSL functions var program = gl.createProgram(); gl.attachShader( program, vertShdr ); gl.attachShader( program, fragShdr ); gl.linkProgram( program ); 53 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

54. Reading a Shader Shaders are added to the program object and compiled Usual method of passing a shader is as a null-terminated string using the function gl.shaderSource( fragShdr, fragElem.text ); If shader is in HTML file, we can get it into application by getElementById method If the shader is in a file, we can write a reader to convert the file to a string 54 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

55. Adding a Vertex Shader var vertShdr; var vertElem = document.getElementById( vertexShaderId ); vertShdr = gl.createShader( gl.VERTEX_SHADER ); gl.shaderSource( vertShdr, vertElem.text ); gl.compileShader( vertShdr ); // after program object created gl.attachShader( program, vertShdr ); 55 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

56. Shader Reader Following code may be a security issue with some browsers if you try to run it locally ­Cross Origin Request 56 function getShader(gl, shaderName, type) { var shader = gl.createShader(type); shaderScript = loadFileAJAX(shaderName); if (!shaderScript) { alert("Could not find shader source: "+shaderName); } Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

57. Precision Declaration In GLSL for WebGL we must specify desired precision in fragment shaders ­artifact inherited from OpenGL ES ­ES must run on very simple embedded devices that may not support 32-bit floating point ­All implementations must support mediump ­No default for float in fragment shader Can use preprocessor directives (#ifdef) to check if highp supported and, if not, default to mediump 57 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

58. Pass Through Fragment Shader #ifdef GL_FRAGMENT_SHADER_PRECISION_HIGH precision highp float; #else precision mediump float; #endif varying vec4 fcolor; void main(void) { gl_FragColor = fcolor; } 58 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

59. 59 WebGL Primitives and Attributes Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

60. 60 WebGLPrimitives GL_TRIANGLE_STRIP GL_TRIANGLE_FAN GL_POINTS GL_LINES GL_LINE_LOOP GL_LINE_STRIP GL_TRIANGLES Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

61. 61 Polygon Issues WebGL will only display triangles ­Simple: edges cannot cross ­Convex: All points on line segment between two points in a polygon are also in the polygon ­Flat: all vertices are in the same plane Application program must tessellate a polygon into triangles (triangulation) OpenGL 4.1 contains a tessellator but not WebGL nonsimple polygon nonconvex polygon Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

62. Triangularization Convex polygon Start with abc, remove b, then acd, …. 62 a c b d Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

63. 63 Attributes Attributes determine the appearance of objects ­Color (points, lines, polygons) ­Size and width (points, lines) ­Stipple pattern (lines, polygons) ­Polygon mode Display as filled: solid color or stipple pattern Display edges Display vertices Only a few (gl_PointSize) are supported by WebGL functions Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

64. 64 RGB color Each color component is stored separately in the frame buffer Usually 8 bits per component in buffer Color values can range from 0.0 (none) to 1.0 (all) using floats or over the range from 0 to 255 using unsigned bytes Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

65. 65 Smooth Color Default is smooth shading ­Rasterizer interpolates vertex colors across visible polygons Alternative is flat shading ­Color of first vertex determines fill color ­Handle in shader Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

66. Setting Colors Colors are ultimately set in the fragment shader but can be determined in either shader or in the application Application color: pass to vertex shader as a uniform variable or as a vertex attribute Vertex shader color: pass to fragment shader as varying variable Fragment color: can alter via shader code 66 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

67. 67 Interact with WebGL in HTML5 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

68. Interaction Interactive GUI: menus, buttons, keyboard Event – Event Mapping – Event Haddler http://pausch.cs.uakron.edu/~xiao/WebGL /Chap3/cad2.htmlhttp://pausch.cs.uakron.edu/~xiao/WebGL /Chap3/cad2.html http://learningwebgl.com/blog/?p=571 68 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

69. Menu (HTML) Black Red Yellow Green Blue Magenta Cyan 69 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

70. Menu (JS) var m = document.getElementById("mymenu"); m.addEventListener( "click", function() { cindex = m.selectedIndex;} ); 70 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

71. Button (HTML) End Polygon 71 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

72. Button (JS) var a = document.getElementById("Button1"); a.addEventListener( "click", function(){ ….} ); 72 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

73. Keyboard (HTML) No code is needed in HTML for getting the keyboard events since there is no GUI for the keyboard. http://learningwebgl.com/blog/?p=571 73 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

74. Keyboard (JS) function Key(event) { if (String.fromCharCode(event.keyCode) == “b") {// code for blinking } else if (String.fromCharCode(event.keyCode) == “c") {// code for changing color or texture } … In the initialization function add document.onkeydown = Key; 74 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015