Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Lecture 16 of 42 William H. Hsu Department of Computing and Information Sciences, KSU KSOL course pages: Course web site: Instructor home page: Readings: Disregard This Play aka Bingo: SIGGRAPH 2007 highlights: Chapter 5, Eberly 2 e – see Animation: Interpolation

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Lecture Outline Animation Problems  Previsualization  Modeling  Keyframing  Dynamics and Interpolation  Lighting Fast Previsualization Modeling Keyframe Interpolation  Position  Orientation  Scale Camera Animation

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Review: Easing Adjust the timing of the inbetween frames. Can be automated by adjusting the stepsize of parameter, t. © 2001 Brogan, D., University of Virginia CS 445 (Undergrad) / CS 645 (Graduate) – Introduction to Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Computer Animation Displaying animation sequences raster animation Creating animation sequences object definition path specification key frames in-betweening Parametric equations © 2006 Escardó, University of Birmingham Graphics 2

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Steps of a simple computer animation 1. Creating animation sequences  object definition  path specification (for an object or a camera)  key frames  in-betweening 2. Displaying the sequences  raster animation  colour-table animation © 2006 Escardó, University of Birmingham Graphics 2

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Displaying animation sequences Movies work by fooling our eyes A sequence of static images presented in a quick succession appears as continuous flow © 2006 Escardó, University of Birmingham Graphics 2

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Why animation works The eye cannot register images faster than approximately 50 frames per second (30 is just about adequate) If a gap in the projection occurs, the eye seems to perform spatial interpolation over the gap

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Displaying animation sequences To achieve smooth animation, a sequence of images (frames) have to be presented on a screen with the speed of at least 30 per second Animations frames can be  pre-computed in advance and pre-loaded in memory  computed in real time (e.g. movement of the cursor) © 2006 Escardó, University of Birmingham Graphics 2

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Raster animation This is the most common animation technique Frames are copied very fast from off-screen memory to the frame buffer Copying usually done with bitBLT-type operations Copying can be applied to  complete frames  only parts of the frame which contain some movement

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Examples © 2006 Escardó, University of Birmingham Graphics 2

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics BitBLT with logic

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Raster animation - procedures A part of the frame in the frame buffer needs to be erased The static part of the frame is re-projected as a whole, and the animated part is over-projected.

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Double buffering Used to achieve smooth animation The next frame of animation is computed to an off-screen buffer at the same time when the current frame is transferred to the frame buffer.

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Colour-table animations Simple 2D animations can be easily implemented using colour lookup table. This technique will be described later

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics CREATING ANIMATION SEQUENCES

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Object definition In simple manual systems, the objects can be simply the artist drawings In computer-generated animations, models are used Examples of models:  a "flying logo" in a TV advert  a walking stick-man  a dinosaur attacking its prey in Jurassic Park

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Models can be Rigid (i.e. they have no moving parts) Articulated (subparts are rigid, but movement is allowed between the sub-parts) Dynamic (using physical laws to simulate the motion) Particle based (animating individual particles using the statistics of behaviour) Behaviour based (e.g. based on behaviour of real animals)

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Simple rigid objects can be defined in terms of  polygon tables (3D)  basic shapes such as line segments, circles, splines etc. (2D) Rigid body animation is an extension of the three-dimensional viewing

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Path specification Impression of movement can be created for two basic situations, or for their combination:  static object, moving camera  static camera, moving object The path defines the sequence of locations (for either the camera or the object) for the consecutive time frames

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Static object, moving camera F3 F1 F2 F4 Time F5

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Static camera, moving object F1F2F3F4

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Static object, moving camera The path specifies the spatial coordinates along which the camera moves The path is usually specified for a single point, e.g. the VRP

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Static object, moving camera During movement, the target point in the World coordinate system can  remain the same (e.g. when walking or flying around the object to see it from all directions);  change (e.g. standing in one location and looking round, or moving along a given path and showing the view seen by the observer while moving).

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Static camera, moving object Path specifying the object movement has to be defined The path is defined as the spatial coordinates along which the object moves

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Static camera, moving object Objects and their parts are defined in a local coordinate system Animation path is defined in the World coordinate system The path is specified for a single point, e.g. the centre of the object's local coordinate system Coordinates of the actual points describing the object are calculated afterwards

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics It is important to remember that when the object moves along the path, not only its position changes, but also its orientation

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics It is important to remember that when the object moves along the path, not only its position changes, but also its orientation X Y Z

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics KEY FRAMES AND IN-BETWEENING

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Rigid body animation Rigid body animation uses standard 3D transformations At least 30 frames per second to achieve smooth animation Computing each frame would take too long

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Key frames Compute first a small number of key frames Interpolate the remaining frames in-between these key frames (in- betweening) Key frames can be computed  at equal time intervals  according to some other rules  for example when the direction of the path changes rapidly

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics In-betweening The simplest method of in-betweening is linear interpolation Interpolation is normally applied to the projected object points

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics 1 2 Key frame k 1’ 2’ Key frame k+1 3 3’ added point Halfway frame

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics In-betweening - example Given coordinates of a 2D point  key frame n:(x n,y n )  key frame n+1: (x n +1,y n +1)  time interval between the two key frames: 1/10 second To get smooth animation, needs at least 30 frames per second Solution: insert at least further 2 frames between the given two key frames

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Calculating in-between frames using linear interpolation  x = (x n+1 - x n ) / 3  y = (y n+1 - y n ) / 3 for ( i=1; i<3; i++ ) { x i = x n + i *  x y i = y n + i *  y }

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics In-betweening Linear interpolation will not always produce realistic results. Example: an animation of a bouncing ball where the best in- betweening can be achieved by dynamic animation

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics In-betweening In-betweening should use interpolation based on the nature of the path, for example:  straight pathlinear interpolation  circular pathangular interpolation  irregular pathlinear interpolation spline

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Parametric equations Formulae using parametric representation of lines and curves, e.g.  line segment  circle  Bezier curve A flexible tool for interpolation

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Parametric equations - line segment Example for line segment between two points, (x n,y n ) and (x n+1,y n+1 )  calculate points in between the two given points  x i = x n + t (x n+1 - x n )  y i = y n + t (y n+1 - y n )  t is the parameter which always changes between 0 and 1  when t = 0, we get x n  when t = 1 we get x n+1  for 0 < t < 1 we get the points in between Parametric equations - line segment

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Parametric equations - line segment The only thing to decide is the number steps between point n and point n+1 This allows us to set the value of  t, which is 1 divided by the number of steps For example, for 10 steps,  t = 1/10 = 0.1 This formula works also for points in 3D

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Parametric equations - Circle GivenCircle radius:r Centre at:(0,0) Parametric quations x( t ) = r * cos( 2  t ) y( t ) = r * sin( 2  t )

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Parametric equations - Circle Parametric quations x( t ) = r * cos( 2  t ) y( t ) = r * sin( 2  t ) Algorithm Select  t for(t=0; t<=1; t = t+  t ) { plot point atx = x( t ) = r * cos( 2  t ) y = y( t ) = r * sin( 2  t ) }

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Human Face Modeling and Animation

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Introduction Face is very important INTERFACE for people It is an interface which is very natural, very reach in signalling We have special structures in the brain for processing face information We would like to have computer interfaces using face talking to us. HOW TO MAKE THIS?

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Application Areas Advanced user interfaces: Social agents and avatars Education: Pedagogical agents Medicine: Facial tissue surgical simulation Criminalistics: Forensic face recognition Teleconferencing: Transmitting & receiving facial images Game industry: Realistic games Media and art: Movies: Shrek, The Incredibles….not realistic faces

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics FACE IS COMPLICATED Face anatomy: Face muscles More than 200 Shape varying Connected to bones & tissues Skin

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Facial Modeling Involves determination of geometric descriptions & animation capabilities. Face has a very complex, flexible 3-D surface. It has color and texture variations and usually contains creases and wrinkles. Methods for effective animation and efficient rendering: volume representation, surface representation and face features.

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Methods for Effective Animation and Efficient Rendering Volume representation: includes constructive solid geometry (CSG), volume element (voxel) arrays and aggregated volume elements such as octrees. Surface representation: includes implicit surfaces, parametric surfaces and polygonal surfaces.

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Surface Representation Implicit surfaces: defined by a function F(x,y,z)=0. Parametric surfaces: generated by three functions of two parametric variables, one function for each of the spatial dimensions. Include B-splines, Beta-splines, Bezier patches, NURBS.

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Surface Representation Polygonal surfaces: include regular polygonal meshes and arbitrary polygon networks. Face Features: is usually the sum of many parts and details. A complete approach is to integrate the facial mask, facial features details and hair into models of the complete head. Polygonal surface Face mask Detailed features

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Techniques Used for Surface Measurements Specifying the 3-D surface of a face is a significant challenge. The general approaches to specify the 3-D face are: 3-D digitizer Photogrammetric measurement Modeling based on laser scans

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics 3-D Digitizer Special hardware devices that rely on mechanical, electro- magnetic or acoustic measurements to locate positions in space. Works best with sculptures or physical models that do not change during the measuring process unlike real faces. Used to sequentially measure the vertex position for each sculpture. Plastic sculpture is digitized as meshes

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Photogrammetric Measurement This method captures facial surface shapes and expressions photographically. Basic idea is to take multiple simultaneous photographs of the face, each from different views. If certain constraints are observed when taking the photographs, the desired 3-D surface data points can be computed based on measured data from the multiple 2-D views. Then a 3-D coordinate system with a coordinate origin near the center of the head is established.

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Procedure of Photogrammetric Measurement Setup of cameras and projectors Photographs from different views Mesh

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Modeling Based on Laser Scans Laser based surface scanning devices can be used to measure faces. These devices typically produce a very large regular mesh of data values in a cylindrical coordinate system. In addition to the range data, a color camera captures the surface color of the head, so for each range point the corresponding color is also available. This is useful for texture mapping in the final rendering of the talking head.

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Procedure of Modeling Based on Laser Scans Scanned surface range data Scanned surface color data An adapted mesh overlaid on its surface color data

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Face Modeling 1. Geometrical graphic model as generic face model 2. Cyber scanner to obtain texture & range data of a face and range data viewed in 3-D 3. Feature points are selected and face model is automatically warped to produce a customized face model

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics The end result

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Facial Animation To make face ”alive. Several approaches to facial animation: Interpolation Performance driven animation Direct parametrization Pseudo muscle based animation Muscle based animation

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Interpolation Most widely used technique. Uses key-framing approach. The desired facial expression is specified for a certain point in time and then again for another point in time some number of frames later. A computer algorithm then generates the frames in between these key frames. Facial animation is achieved by digitizing the face in each of the several different expressions and then interpolating between these expressions. Key-frame animation requires complete specification of the model geometry for each key facial expression.

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Interpolation Between Expressions Surprised Sad Worried ActualActual Interpolated

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Summary Animation Problems  Previsualization  Modeling  Keyframing  Dynamics and Interpolation  Lighting Modeling Keyframe Interpolation  Position  Orientation  Scale

Computing & Information Sciences Kansas State University Lecture 16 of 42CIS 636/736: (Introduction to) Computer Graphics Terminology Tasks  Previsualization  Modeling  Keyframing  Lighting Keyframe Interpolation Dynamics  Forward – ballistic simulation  Inverse – ballistics (fire control) Kinematics  Forward – simulation of articulated model  Inverse – control of articulated model