CS 99k: Digital Actors URL: Instructor: Jean-Claude Latombe Computer Science Department Stanford University

Computer Animated Characters A Bug’s Life (Pixar/Disney) Toy Story (Pixar/Disney) Tomb Raider 3 (Eidos Interactive)Final Fantasy VIII (SquareOne)The Legend of Zelda (Nintendo) Antz (Dreamworks)

Autonomous Agents Definition: “An autonomous agent is a system situated within and a part of an environment that senses that environment and acts on it, over time, in pursuit of its own agenda and so as to effect what it senses in the future.” Franklin & Graesser, 1996 “Is it an Agent, or just a Program?” Proc. of 3rd Int. Workshop on Agent Theories

Animated Graphic Agent Basic definition: Animated character whose motion does not need to be explicitly defined by a user. ? ? Other commonly-used terms: Digital Actor, Virtual Human Ultimate Goal: Fully-autonomous animated agents able to sense, act, and interact in changing virtual environments

Goals vs. Motions Autonomous Goal Generation: Algorithm to decide what the character should do next (specifies high-level goal) Autonomous Motion Generation: Algorithm to generate underlying motion given a high-level goal

Example: Moving an Object Reach Grab Transfer Release Return

Two Classes of Motion Two Classes of Motion u u Primary (Active) Motion: gross body movements, facial animation u u Secondary (Passive) Motion: clothes, skin, hair, env. effects Virtual Clothes (MIRALAB / Univ. of Geneva) Facial animation (Geri’s Game / Pixar) Endgame (Stanford)

Autonomy vs. Interactivity Autonomy Interactivity User-controlled Completely Autonomous Off-lineReal-time Key-frame Animation Systems Video Game Adversaries Web Avatars Digital Actors Semi-Autonomous Characters Urban / Factory Simulation Virtual Tour Guides Historical Narratives

1999 Interactive Chess (Stanford) Playing Chess 1994 ENDGAME (Stanford) 1997 Geri’s Game (Pixar)

Animation Variables q 2 q 1 q 3 q 0 q n q 4 Q(t) Parts DOF L H

Motion Generation Techniques Synthetic Characters (MIT) Blumberg, et. Al. Reactive Behaviors Brooks ‘85, Reynolds ‘87, Tu & Terzopoulos ‘94, Reich, et al. ‘94, Blumberg & Galyean ‘95 Marilyn (EPFL/Miralab) Thalmann, et. Al. Artificial Fish (U. Toronto) Tu / Terzopoulos Scripting Kahn ‘79, Ridsdale ‘87, Haumann & Parent ‘88, Wilhelms ‘90, Strassmann ‘91, Bates, et al. ‘92, Perlin & Goldberg ‘96 OZ project (CMU) Bates, et. Al. Kinematic Warping Bruderlin & Calvert ‘89, Witkin & Popovich ‘95, Unuma, et al. ‘95, Gliecher ‘98

Motion Generation Techniques Physically-Based Simulation Armstrong & Green ‘85, Hahn ‘88, Baraff ‘89, McKenna & Zeltzer ‘90, Raibert & Hodgins ‘91, Mirtich ‘95, Ngo & Marks ‘93, Sims ‘94, Metaxas, et al. ‘97 Motion Planning Lengyel, et at. ‘90, Ching & Badler ‘92, Koga, et al. ‘94, Bandi & Thalmann ‘97, Hsu & Cohen ‘98 Human Dynamics (GaTech) Hodgkins, et. Al. HFAP (Microsoft) Cohen, et. Al. ENDGAME (Stanford) Koga, et. Al. Spacetime Constraints Isaacs & Cohen ‘87, Witkin & Kass ‘88, Rose, et al. ‘96, Gleicher ‘98, Popovic & Witkin ‘99 Jack (U Penn) Badler, et. Al.

Motion Planning for Digital Actors Manipulation Sensory-based locomotion

Applications u Videogames u Movies u Ergonomics u Training u Web page animation u Slide shows u E-commerce

Course Outline u Class 1: Introduction + Motion Representation u Class 2: Digital Actor Authoring and Runtime Software (Motivate) Guest speaker: Yotto Koga u Class 3: Geometric/Kinematic/Dynamic modeling of a Digital Actor u Class 4: Motion Generation Techniques Guest speaker: Chris Bregler u Class 5: Motion Planning Guest Speaker: Oussama Khatib

Course Outline (cont’d) u Class 6: Animation of Soft Tissues Guest speakers: Joel Brown and Stephen Sorkin u Class 7: Digital Actors with Sensing and Memory Guest speaker: Josh Tenenbaum u Class 8: High-Level Behaviors and Man-Machine Interaction

What is a Motion?

Definition of an Object’s Configuration u The configuration of an object is a specification of the positions of all the points in this object relative to a fixed coordinate system u Usually it is expressed as a “vector” of position and orientation parameters q1q1q1q1 q2q2q2q2 q3q3q3q3 qnqnqnqn q=(q 1,…,q n )

Rigid Object Example x y  reference point robot reference direction workspace Object’s configuration is: q = (x,y,  ) Object’s configuration is: q = (x,y,  ) In a 3-D workspace q would be of the form (x,y,z,  )

Articulated Object Example q1q1q1q1 q2q2q2q2 q = (q 1,q 2,…,q 10 )

Configuration Space of a Moving Object u Space of all its possible configurations u The topology of this space is usually not that of a Cartesian space C = S 1 x S 1

Case of a Rigid Robot in 3-D Workspace  q = (x,y,z,  ) u Other representation: q = (x,y,z,r 11,r 12,…,r 33 ) where r 11, r 12, …, r 33 are the nine components of a 3x3 rotation matrix r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 with: –r i1 2 +r i2 2 +r i3 2 = 1 –r i1 r j1 + r i2 r 2j + r i3 r j3 = 0 –det = +1 the configuration space is a 6-D space (manifold) embedded in a 12-D Cartesian space. It is denoted by R 3 xSO(3)

Parameterization of SO(3)  Euler angles: (   Unit quaternion: (cos  n 1 sin  /2, n 2 sin  /2, n 3 sin  /2) x y z x yzx y z x y z    

Metric in Configuration Space u A metric or distance function d in C is a map d: (q 1,q 2 ) d(q 1,q 2 ) > 0 such that: –d(q 1,q 2 ) = 0 if and only if q 1 = q 2 –d(q 1,q 2 ) = d (q 2,q 1 ) –d(q 1,q 2 ) < d(q 1,q 3 ) + d(q 3,q 2 ) u Example: Given an object A and a point x of A, let x(q) be the point of the workspace occupied by x when the robot is at configuration q. A distance d is defined by: d(q,q’) = max ||x(q)-x(q’)|| over all points x of A where ||a - b|| denotes the distance between points a and b

A Specific Example: Metric in R 2 x S 1  Let q = (x,y,  ) and q ’ = (x ’,y ’,  ’ ) with  and  ’ in [0,2  )  d(q,q ’ ) = sqrt[(x-x ’ ) 2 + (y-y ’ ) 2 +  2 ] where  = min{|  -  ’ |, 2  -|  ’ |}  d(q,q ’ ) = sqrt[(x-x ’ ) 2 + (y-y ’ ) 2 +  2 ] where  = min{|  -  ’ |, 2  -|  ’ |}  d(q,q ’ ) = sqrt[(x-x ’ ) 2 + (y-y ’ ) 2 + (   ] where  is the maximal distance between the reference point and a robot point ’’’’ 

Notion of a Path  A path in C is a piece of continuous curve connecting two configurations q and q’:  : s in [0,1]  (s) in C u Other possible constraints: finite length, smoothness, … u A trajectory is a path parameterized by time q 1 q 3 q 0 q n q 4 q 2  (t)

Homotopic Path u Two paths with the same endpoints are homotopic if one can be continuously deformed into the other  R x S 1 example: Paths  1 and  2 are homotopic Paths  1 and  3 are not homotopic Infinity of homotopic classes q q’   

Configuration Space Connectedness u C is connected, meaning that every two configurations can be connected by a path u C is simply-connected if any two paths connecting the same endpoints are homotopic Example: R 2 or R 3 u Otherwise C is multiply-connected Examples: S 1 and SO(3) are multiply connected In S 1, infinity of homotopic classes In SO(3), only two homotopic classes

Obstacles in Configuration Space u A configuration is collision-free, or free, if the robot placed at this configuration has no intersection with the obstacles in the workspace u The free space is the set of all free configurations u A C-obstacle is the set of all configurations where the robot collides with a given workspace obstacle u A configuration is semi-free if the robot at this configuration touches obstacles without overlap

Disc Robot in 2-D Workspace

Rigid Robot Translating in 2-D CB = B A = {b - a | a in A, b in B}

Linear-Time Computation of C-Obstacle in 2-D

Rigid Robot Translating and Rotating in 2-D

C-Obstacle for Articulated Robot

A Remark on the Topology of the Free Space u The robot and the obstacles in the workspace are modeled as closed subsets. This means that they contain their boundaries u One can show that the C-obstacles are closed subsets of the configuration space C u Consequently, the free space F is an open subset of C. Hence, each free configuration is the center of a ball of non-zero radius entirely contained in F u The semi-free space is a closed subset of C. Its boundary is a superset of the boundary of F

Free and Semi-Free Paths u A free path is one that lies entirely in the free space u A semi-free path is one that lies entirely in the semi-free space

Classes of Homotopic Free Paths