Fundamentals of Computer Graphics Sai-Keung Wong National Chiao Tung University Department of Computer Science Date: 5 March References: wikipedia, UDK, Ogre3D

Contents World coordinates Matrix, scaling, translation, rotation Meshes, vertices, edges, triangles Lighting, spot lights, point lights, directional lights Slerp Textures, materials Key frames Rendering pipeline

The World 3 3D Graphics Layers A World Frame Buffers (front + back) A world is a set of layers where the 3D objects acts on

Example of Graphics Layers: UDK

The 3D Graphics Layer 5 Camera 3D Models Lights Projection Plan Board A 3D Scene A 3D graphics layer is a projection of the rendering of a 3D world “Scene” : the 3D world “Viewport” : the place for projection

6 X-, Y- Z-axis No assumptions about the orientation of models. Use Vector class to represent both position and direction Vector Types: Vector2, Vector3 Coordinates and Vectors

Matrix basics –Definition –Transpose –Addition 7Matrices A = (a ij ) = a 11.. a 1n.. a m1.. a mn C = A T c ij = a ji C = A + B c ij = a ij + b ij

8 –To make translation be a linear transformation, we introduce the homogeneous coordinate system V (x, y, z, w) where w is always 1 –Translation Transformation x’ = x + T x y’ = y + T y z’ = z + T z V’ = V T [x ’ y ’ z ’ 1] = [x y z 1] = [x y z 1] T T x T y T z 1

9 –Scaling Transformation x’ = xS x y’ = yS y z’ = zS z V’ = V S [x ’ y ’ z ’ 1] = [x y z 1] = [x y z 1] S S x S y S z Here S x, S y and S z are scaling factors.

10 –Rotation Transformations cos  sin  0 0 -sin  cos  R x = R y = R z = cos  0 -sin  sin  0 cos  cos  sin  0 0 -sin  cos 

A vector is an entity that possesses magnitude and direction. A 3D vector is a triple : – V = (v 1, v 2, v 3 ), where each component v i is a scalar. A ray (directed line segment), that possesses position, magnitude and direction. 11 Vectors & Rays (x 1,y 1,z 1 ) (x 2,y 2,z 2 ) V = (x 2 -x 1, y 2 -y 1, z 2 -z 1 )

Addition of vectors Length of vectors 12 X = V + W = (x 1, y 1, z 1 ) = (v 1 + w 1, v 2 + w 2, v 3 + w 3 ) V W V + W V W |V| = (v v v 3 2 ) 1/2 U = V / |V| Vector Arithmetic

Cross product of vectors –Definition –Application A normal vector to a polygon is calculated from 3 (non-collinear) vertices of the polygon. 13 X = V X W = (v 2 w 3 -v 3 w 2 )i + (v 3 w 1 -v 1 w 3 )j + (v 1 w 2 -v 2 w 1 )k where i, j and k are standard unit vectors : i = (1, 0, 0), j = (0, 1, 0), k = (0, 0, 1) NpNp V2V2 V1V1 polygon defined by 4 points N p = V 1 X V 2 Vector Arithmetic

14 Ray Cast – The Ray x = x 0 + (x 1 – x 0 ) t y = y 0 + (y 1 – y 0 ) t, t = 0, z = z 0 + (z 1 – z 0 ) t { Shoot a ray to calculate the intersection between the ray and models Use a parametric equation to represent a ray 8 The ray emits from (x 0,y 0,z 0 ) Only the t  0 is the answer candidate The smallest positive t is the answer

Rotation vs Orientation 15 Orientation: relative to a reference alignment Rotation: change object from one orientation to another Represent an orientation as a rotation from the reference alignment

Orientation Source: wiki Changing orientation is the same as moving the coordinate axes.

18 A rotation is described as a sequence of rotations about three mutually orthogonal coordinates axes fixed in space (e.g. world coordinate system) –X-roll, Y-roll, Z-roll There are 6 possible ways to define a rotation –3! R(  1,  2,   ) represents an x-roll, followed by y-roll, followed by z-roll R(  1,  2,    c 2 c 3 c 2 s 3 -s 2 0 s 1 s 2 c 3 -c 1 s 3 s 1 s 2 s 3 +c 1 c 3 s 1 c 2 0 c 1 s 2 c 3 +s 1 s 3 c 1 s 2 s 3 -s 1 c 3 c 1 c where s i = sin  i and c i = cos  i Euler Angles Left hand system

19 R( , n), n is the rotation axis n rRr  n r rvrv rhrh V  rvrv V Rr v r h = (n. r)n r v = r - (n. r)n, rotate into position Rr v V = nxr v = nxr Rr v = (cos  )r v + (sin  )V -> Rr = Rr h + Rr v = r h + (cos  )r v + (sin  )V = (n. r)n + (cos  )  r - (n. r)n) + (sin  ) nxr = (cos  )r + (1-cos  ) n (n. r) + (sin  ) nxr Angular Displacement

20 Sir William Hamilton (1843) From Complex numbers (a + ib), i 2 = -1 16,October, 1843, Broome Bridge in Dublin 1 real + 3 imaginary = 1 quaternion q = a + bi + cj + dk i 2 = j 2 = k 2 = -1 ij = k & ji = -k, cyclic permutation i-j-k-i q = (s, v), where (s, v) = s + v x i + v y j + v z k Used for representing the orientation or rotation of an objectQuaternion

21 Spherical linear interpolation, slerp A B P  t slerp(q 1, q 2, t) = q 1 + q 2 sin((1 - t)  ) sin  sin(t  ) Quaternion Interpolation

22 Initial value problems ODE –Ordinary differential equation Numerical solutions –Euler’s method –The midpoint method –Kunge –Kutta methods Differential Equation Basics

23 An ODE An ODE Vector field Vector field Solutions Solutions –Symbolic solution –Numerical solution x = f (x, t) wheref is a known function x is the state of the system, x is the x’s time derivative x & x are vectors x(t 0 ) = x 0, initial condition. Start here Follow the vectors … Initial Value Problems..

24 A numerical solution –A simplification from Tayler series Discrete time steps starting with initial value Simple but not accurate –Bigger steps, bigger errors –Step error: O( h 2 ) errors –Total error: O(h) Can be unstable Not efficient x(t + h) = x(t) + h f(x, t) Euler’s Method

Dynamics – Particle system Game FX System 25 Initial Value Problems - Application

26 Geometry for Game

Geometry – Position / vertex normals / vertex colors / texture coordinates Topology – Primitive Lines / triangles / surfaces / … Property – Materials – Textures Motion Hierarchy Level-of-detail 27 Game Models

28 Vertex position Vertex position –(x, y, z, w) –In model space or screen spane Vertex normal Vertex normal –(n x, n y, n z ) Vertex color Vertex color –(r, g, b) or (diffuse, specular) Texture coordinates on vertex Texture coordinates on vertex –(u 1, v 1 ), (u 2, v 2 ), … Skin weights Skin weights –(bone 1, w 1, bone 2, w 2, …) Geometry Data

29 Lines –Line segments –Polyline »Open / closed Indexed triangles Triangle Strips / Fans Surfaces –Non-uniform Rational B Spline (NURBS) Subdivision Topology Data

30 Geometric data Vertex data v 0, v 1, v 2, v 3, … (x, y, z, n x, n y, n z, t u, t v ) or (x, y, z, c r, c g, c b, t u, t v, … ) Topology Face v 0 v 3 v 6 v 7 Edge table v0v0 v3v3 v6v6 v7v7 Right-hand rule for index polygon normal vertex normal Indexed Triangles

31 v0v0 v1v1 v2v2 v3v3 v4v4 v5v5 v6v6 v7v7 T0T0 T1T1 T2T2 T3T3 T4T4 T5T5 v 0, v 1, v 2, v 3, v 4, v 5, v 6, v 7 Triangle Strips “Get great performance to use triangle strips for rendering on current hardware

v0v0 v1v1 v2v2 v3v3 v4v4 v5v5 v6v6 v7v7 T0T0 T1T1 T2T2 T3T3 T4T4 T5T5 v 0, v 1, v 2, v 3, v 4, v 5, v 6, v 7 Trianglulation v0v0 v1v1 v2v2 v3v3 v4v4 v5v5 v6v6 v7v7

UDK: Geometry tool, Edit one vertex

35 Material Textures Shaders Property on Surface

36 Material –Ambient »Environment »Non-lighted area –Diffuse »Dynamic lighting –Emissive »Self-lighting –Specular with shineness »Hi-light »View-dependent »Not good for hardware rendering Local illumination For fixed function rendering pipelineMaterials

37 Textures –Single texture –Texture coordinate animation –Texture animation –Multiple textures –AlphamapTextures Material or vertex colors Base color texture Lightmap

Example Source: U Direction V Direction 1.0 0, 0

Material Example: UDK

42 Programmable Shading Language –Vertex shader –Pixel shader Procedural way to implement some process of rendering –Transformation –Lighting –Texturing –BRDF –Rasterization –Pixel fill-inShaders

Wireframe, no shading

Shading, no materials

Shading, with materials

Light types *Ambient Light -Lights all objects within a scene equally, brightening them without adding shading. *Directional Light - Illuminates all objects equally from a given direction. It’s infinite distance from the scene. There is shading, but cannot be any distance falloff. *Point Light - Originates from a single point, and spreads outward in all directions. *Spotlight - Originates from a single point, and spreads outward in a coned direction. Area Light Volume Light Referene:

47 Vertex DataTopology Data Classic Transform & Lighting Vertex Shader Clipping & Viewport Mapping TexturingPixel Shader Fog Alpha, Stencil, Depth Testing Geometry Stage Rasterizer Stage Shader Pipeline Rendering : the process of generating an image from a model, by means of computer programs.

Per-pixel lighting Motion blur Volume / Height fog Volume lines Depth of field Fur fighting Reflection / Refraction NPR ( non-photorealistic rendering ) Shadow Linear algebra operators Perlin noise Quaternion Sparse matrix solvers Skin bone deformation Normal map Displacement map Particle shader 48 Procedural Morphing Water Simulation Powered by Shader

Water simulation

50 Discrete LOD –Switch multiple resolution models run-timely Continuous LOD –Use progressive mesh to dynamically reduce the rendered polygons View-dependent LOD –Basically for terrainLevel-of-detail

51 Real-time optimal adapting meshes (ROAM) Use height map Run-timely to re-construct the active (for rendering) geometric topology (re-mesh) to get an optimal mesh (polygon count) to improve the rendering performance Someone calls this technique as the view-dependent level-of-detail Very good for fly-simulation-like application View-dependent LOD for Terrain - ROAM

52 Bounding sphere Bounding cylinder Axis-aligned bounding box (AABB) Oriented bounding box (OBB) Discrete oriented polytope (k-DOP); k/2 pairs Bounding Sphere AABB OBB k-DOP Bounding Volume Bounding Cylinder

53 Collision Detection Visibility Culling Hit Test Bounding Volume - Application

54 Bounding sphere B 1 (c 1, r 1 ), B 2 (c 2, r 2 ) If the distance between two bounding spheres is larger than the sum of radius of the spheres, then these two objects have no chance to collide. d > r 1 + r 2 D B1B1 B2B2 c1c1 c2c2 Application Example - Bounding Sphere

55 Axis-aligned bounding box (AABB) –Simplified computation based on the axis- aligned feature –Have to compute the AABBs at runtime AABB Application Example - AABB

56 Oriented bounding box (OBB) –Intersection computation based on the transformed OBB geometric data »3D containment test »Line intersection with plane OBB Application Example - OBB

Some objects

Bounding volumes in UDK for collision detection

59 This is a game-type-oriented issue Bounding Volume Hierarchies (BVHs) Binary space partition trees (BSP Trees) “Quake” Octree PVS (Potentially visible set) Culling Skills Advanced Scene Graphs

60 Hierarchical structure of bounding spheres R B Bounding Volume Hierarchies (BVHs)

61 plane1 plane0 plane2 plane Axis-aligned BSP Tree

62 A B C D E F G A B C D E FG Polygon-aligned BSP Tree

63 Quickly to identify where you are BSP = Sorting Need a pre-processor to generate the PVS Visibility culling + occlusion culling PVS : Potentially Visible Set Optimized for in-door game environment Why BSP Tree ?

64 Octree Similar to axis-aligned BSP tree A box is split simultaneously along all three axes The split point must be the center of the box This creates eight new smaller boxes Quadtree is the 2D version of octree Octree & Quadtree

65 Quadtree - Example

66 Data structure coherence Apply visibility culling from parents Split or not split ? Outdoor game scene ? Octree – Some Discussion

67 Culling means “remove from a flock” Visibility culling Remove the object not in view frustum Backface culling Remove the polygons facing away from camera Hardware standard Occlusion culling Remove the objects hidden by the others Culling (1/2)

68 eye View frustum Visibility culling Backface culling Occlusion culling Culling (2/2)

69 Characters

The Characters are the Actors of the Games. Three Types of Characters : – Segmented – Mesh – Bone-skin Root-base Concept Production : – 3D animation tools 3dsMax Maya Softimage … – Motion capture (Mocap) For motion data 70Introduction Base

71 A character is composed by a set of models with motion data to simulate a live creature in real world Benefits –Hierarchical structure –Easy to implement in a scene tree Drawbacks –Segment-like A Segmented Character

72 Vertex animation on skins –Animated positional data on skins –3D warping Benefits –Easy to implement –Flexible mesh in animation Drawbacks –No hierarchy –No keyframes –Massive dataset A Mesh Character

73 Bone-Skin Skeleton –Hierarchical bones –Skin deformation run-timely Benefits –Hierarchical structure –Not segmented look Drawbacks –More complicated than the other solutions –Skin deformation need more CPU cost than transformation only Bone A Skin Bone B A Bone-skin Character

74 Time-dependent data Transformation data –Position –Orientation Formats –Pivot –Position vector –Quaternion –Eurler angles –Angular displacement Motion Data

Euler Angles Angular Displacement Quaternion – Slerp Converted into (4x4) matrix for computation 75 Motion Data

Data Acquired – From skin to joint (Mocap) – From joint to skeleton (Post-processing) – From skeleton to skin (In-game) Device The Shooting Plan 76 Optical Motion Capture

77 Raw Data (Positional Data) Joint End Point Bio-Data Data Acquirement During the Mocap

78 Skeletons Skin Skeletons Bone-Skin Bone-skin Implementation In Game

79 Mocap Devices

80

81

82 To Create More Animation from Limited Ones To Create More Animation from Limited Ones Run-time or Pre-processing Run-time or Pre-processing Issues : Issues : –Motion re-targeting »Run-time –Re-key-framing (key-frame: ) »Pre-processing –Interpolation »Run-time –Motion blending »Run-time Motion Editing

83 A set of frame data to describe the character’s motion Walk, Run, Attack, … Keyframed or Non-keyframed walkrunattackfallPoses

84 Pose 1 Pose 2 cut_frame start_frame length Pose Connection

85 Pose 1 Pose Cross Fade

86 Pose 1 Pose Continuous Blending

87 左右搏擊 左右搏擊 Pose 1Pose 2 Pose 3 Feather Blending

Game Type Oriented Terrain – For visual ( 廣義的場景 ) Ground / Building / Static models / Dynamic models – For terrain following Polygon mesh Grids – For path finding Polygon mesh Grids Terrain Following – Make a 3D entity walking on terrain Path Finding – Find a path before walking 88Terrain

Terrain

Summary Fundamentals of computer graphics