CPSC 641 Computer Graphics: Animation with Motion Capture

Slides:



Advertisements
Similar presentations
CS 6353 Computer Graphics and Multimedia
Advertisements

Virtual Me. Motion Capture The process of recording movement and translating that movement onto a digital model Originally used for military tracking.
Motion Capture The process of recording movement and translating that movement onto a digital model Games Fast Animation Movies Bio Medical Analysis VR.
Virtual Me. Motion Capture (mocap) Motion capture is the process of simulating actual movement in a computer generated environment The capture subject.
Hierarchical Transformation. Summary Alternatively, OpenGL thinks: A transformation updates the coordinate system. For each change, the transformation.
MOTION CAPTURE IN LIFE SCIENCES Mario Lamontagne.
Speech and Gesture Corpus From Designing to Piloting Gheida Shahrour Supervised by Prof. Martin Russell Dr Neil Cooke Electronic, Electrical and Computer.
Computer Graphics Computer Animation& lighting Faculty of Physical and Basic Education Computer Science Dep Lecturer: 16 Azhee W. MD.
1 Characters. 2Introduction The Characters are the Actors of the Games. The Characters are the Actors of the Games. Three Types of Characters : Three.
Animation. 12 Principles Of Animation (1)Squash and Stretch (2)Anticipation (3)Staging (4)Straight Ahead Action and Pose to Pose (5)Follow Through and.
Lecture 5 Jinxiang Chai. Outline Motion capture data format Class homepage & paper assignment Presentation tips.
Character Setup Character Setup is the process of creating handles and controls for anything that a character animator will need to adjust in order to.
CSCE 641: Forward kinematics and inverse kinematics Jinxiang Chai.
Motion Editing and Retargetting Jinxiang Chai. Outline Motion editing [video, click here]here Motion retargeting [video, click here]here.
Motion Capture CS294-7 Jacqueline Takeshita Mindy Lue.
© , The Ohio State University Motion Capture History, Pipeline, and Areas of Interest Advanced Computing Center for the Arts and Design Ohio State.
CSCE 441 Computer Graphics: Animation with Motion Capture
CSCE 641: Forward kinematics and inverse kinematics Jinxiang Chai.
Lecture 4: Motion Capture Jinxiang Chai. Outline Mocap history Mocap technologies Mocap pipeline Mocap Data Mocap Challenges.
CSCE 641 Computer Graphics: Animation with Motion Capture Jinxiang Chai.
Drawing a Skeleton Fast From Motion Capture Data Jonathan Kipling Knight Nov 7, 2006.
Tracking Gökhan Tekkaya Gürkan Vural Can Eroğul. Outline Tracking –Overview –Head Tracking –Eye Tracking –Finger/Hand Tracking Demos.
Motion Representation & forward Kinematics Jinxiang Chai.
CPSC 641 Computer Graphics: Animation with Motion Capture Jinxiang Chai.
Motion Capture in 3D Animation Animation : Movies Animation : Movies Video Games Video Games Robot Control Robot Control.
Animation Theory.
Motion Capture Animation
Motion Capture in 3D Animation Edward Tse. Motion Capture as a Tool Motion capture (MOCAP) is an effective 3D animation tool for realistically capturing.
Animation is mainly used to produce cartoons, however, it is also used to produce special effects in commercials, live-action movies, video games, and.
Algirdas Beinaravičius Gediminas Mazrimas.  Introduction  Motion capture and motion data  Used techniques  Animating human body  Problems.
History of Motion Capture Dr. Midori Kitagawa Arts and Technology Program University of Texas at Dallas.
Algirdas Beinaravičius Gediminas Mazrimas.  Introduction  Motion capture and motion data  Used techniques  Animating human body  Problems  Conclusion.
Motion Capture Laboratory School of Arts and Humanities Erik Jonsson School of Engineering and Computer Science University of Texas at Dallas.
Motion Capture CSE 3541 Matt Boggus.
Characters.
Computer Animation Rick Parent Computer Animation Algorithms and Techniques Motion Capture.
Animation.
1 Lecture 19: Motion Capture. 2 Techniques Morphing Motion Capture.
Motion Capture. Animation – A broad Brush Traditional Methods  Cartoons, stop motion Keyframing  Digital inbetweens Motion Capture  What you record.
Prepared By: Menna Hamza Mohamed Mohamed Hesham Fadl Mona Abdel Mageed El-Koussy Yasmine Shaker Abdel Hameed Supervised By: Dr. Magda Fayek.
Character Animation. Contents Keyframe animation Mocap Smooth skin by vertex blending Rigging & retargeting Gait analysis (Ragdoll physics) Fall
Jinxiang Chai Composite Transformations and Forward Kinematics 0.
Algirdas Beinaravičius Gediminas Mazrimas.  Introduction  Motion capture and motion data  Used techniques  Animating human body  Problems  Conclusion.
CS-378: Game Technology Lecture #13: Animation Prof. Okan Arikan University of Texas, Austin Thanks to James O’Brien, Steve Chenney, Zoran Popovic, Jessica.
Algirdas Beinaravičius Gediminas Mazrimas.  Introduction  Motion capture and motion data  Used techniques  Animating human body  Problems  Conclusion.
Character Setup In addition to rigging for character models, rigging artists are also responsible for setting up animation controls for anything that is.
Rick Parent - CIS681 Motion Capture Use digitized motion to animate a character.
1cs426-winter-2008 Notes. 2 Kinematics  The study of how things move  Usually boils down to describing the motion of articulated rigid figures Things.
-BY SAMPATH SAGAR( ) ABHISHEK ANAND( )
Chapter 6.7 Animation. 2 Overview When to use animation Feedback to player about interaction with UI and in-game action Communicating environmental* conditions.
Fundamentals of Computer Animation Motion Synthesis.
CPSC 641 Computer Graphics: Animation with Motion Capture Jinxiang Chai.
CGDD 4003 Character Animation. The Skeletal Hierarchy (aka the “rig”) Based on the concept of bones Each bone has exactly one parent Each bone has a transform.
Computer Animation Rick Parent Computer Animation Algorithms and Techniques Motion Capture.
Forward Projection Pipeline and Transformations CENG 477 Introduction to Computer Graphics.
Motion Capture CSE 3541 Matt Boggus.
CSCE 441 Computer Graphics: Animation with Motion Capture
CSCE 441: Computer Graphics Forward/Inverse kinematics
Computer Graphics.
Computer Animation cgvr.korea.ac.kr.
Chapter 6.7 Animation.
CAPTURING OF MOVEMENT DURING MUSIC PERFORMANCE
Chapter 6.7 Animation.
Three Dimensional Viewing
Skeletons and Skinning
CSCE 441: Computer Graphics Forward/Inverse kinematics
Chapter XIII Character Animation
Prepared by: Engr . Syed Atir Iftikhar
Assist. Lecturer Safeen H. Rasool Faculty of SCIENCE IT Dept.
Computer Graphics Lecture 15.
Presentation transcript:

CPSC 641 Computer Graphics: Animation with Motion Capture Jinxiang Chai

Outline of Mocap (Motion Capture) Mocap history Mocap technologies Mocap pipeline Mocap data fromat

Motion Capture “ …recording of motion for immediate or delayed analysis or playback…” David J. Sturman “The creation of a 3d representation of a live performance” - Alberto Menache “…is a technique of digitally recording movements for entertainment, sports, and medical applications.” - Wikipedia

History of Motion Capture Eadweard Muybridge (1830-1904) first person to photograph movement sequences He was the first person to photograph movement sequences. At the time, it is not known if a horse ever had all feet off the ground while trotting. By recording multiple pictures over a short

History of Motion Capture Eadweard Muybridge (1830-1904) first person to photograph movement sequences whether during a horse's trot, all four hooves were ever off the ground at the same time. the flying horse Sequence of a horse jumping (courtesy of E. Muybridge)

History of Motion Capture Eadweard Muybridge (1830-1904) first person to photograph movement sequences the flying horse animal locomotion (20k pictures about men, women, children, animals, and birds). Woman walking downstairs (courtesy of E. Muybridge)

Rotoscope Allow animators to trace cartoon character over photographed frames of live performances invented by Max Fleischer in 1915

A horse animated by rotoscoping from Muybridge’s photos Rotoscope Allow animators to trace cartoon character over photographed frames of live performances invented by Max Fleischer in 1915 2D manual motion capture A horse animated by rotoscoping from Muybridge’s photos

Rotoscoping Mocap Overview “rotoscoping can be thought of as a primitive form or precursoro to motion capture, where the motion is ‘captured’ painstakingly by hand.” - Sturman

Another example

Rotoscope Allow animators to trace cartoon character over photographed frames of live performances invented by Max Fleischer in 1915 2D manual capture the first cartoon character to be rotoscoped -- “Koko the clown” the human character animation -- snow white and her prince (Walt Disney, 1937) In 1915, using Muybridge’s idea, cartoonist Max Fleischer created the rotoscope, a device that allowed animators to trace cartoon characters over photographed frames of live performers. A time consuming process, mainly used for human motion, it was the first time a real-life performance was used to help create an animated character. The first cartoon character to be rotoscoped was Koko the Clown. The rotoscope was subsequently used by Walt Disney in 1937 to get realistic human motion for Snow White and her prince. Rotoscoping is a two-dimensional approach to capturing motion. [Source: Menache, Alberto, “Understanding Motion Capture for Computer Animation and Video Games,” Morgan Kaufmann, 2000]

Current Motion Capture Technologies “3D Rotoscoping”: measuring 3D positions, orientations, velocities or accelerations Current motion capture systems Electromagnetic Electromechanical Fiber optic Optical

Electromagnetic Mocap Each sensor record 3D position and orientation Each sensor placed on joints of moving object Full-body motion capture needs at least 15 sensors Popular system: http://www.ascension-tech.com/

Electromagnetic Mocap See video demo!

Electromagnetic Mocap Pros measure 3D position and orientation no occlusion problems can capture multiple subjects simultaneously Cons magnetic perturbations (metal) small capture volume cannot capture deformation (facial expression) hard to capture small bone movement (finger motion) not as accurate as optical mocap system

Electromechanical Mocap Each sensor measures 3D orientation

Electromechanical Mocap Each sensor measures 3D orientation Each sensor placed on joints of moving object Full-body motion capture needs at least 15 sensors Popular systems: http://www.xsens.com/

Electromechanical Mocap See video demo!

Electromechanical Mocap Pros measure 3D orientation no occlusion problems can capture multiple subjects simultaneously large capture volume Outdoors capture (e.g. skiing) Cons getting 3D position info is not easy cannot capture deformation (facial expression) hard to capture small bone movement (finger motion) not as accurate as optical mocap system

Fiber Optic Mocap Measures 3D position and orientation of entire tape Binding the tape to the body Popular systems: http://www.measurand.com/ Optical fiber systems such as data gloves use a fiber-optic sensor along the fingers. As the fingers bend, they bend the fiber and the transmitted light is attenuated. The finger joint rotation measurements are controlled by the strength of the attenuated light.

Fiber Optic Mocap See video demo!

Fiber Optic Mocap Pros measure 3D orientation and position no occlusion problems can capture multiple subjects simultaneously go anywhere mocap system can capture hand/finger motion Cons intrusive capture cannot capture deformation (facial expression) not as accurate as optical mocap system

Optical Mocap Multiple calibrated cameras (>=8) digitize different views of performance Wears retro-reflective markers Accurately measures 3D positions of markers

Optical Mocap Vicon mocap system: http://www.vicon.com See video demo!

Optical Mocap Pros measure 3D position data also orientation the most accurate capture method very high frame rate can capture very detailed motion (body, finger, facial deformation, etc.) Cons has occlusion problems hard to capture interactions among multiple ppl limited capture volume

Mocap Pipeline Optical Mocap pipeline Planning Calibration Data processing

Planning Motion capture requires serious planning Anticipate how the data will be used Garbage in garbage out Shot list Games motions need to be able to blend into one an another capture base motions and transitions which motions transition into which other transitions cycles/loops

Movement Flowchart for Games Planning and Directing Motion Capture For Games By Melianthe Kines Gamasutra January 19, 2000 URL: http://www.gamasutra.com/features/20000119/kines_01.htm

Planning Character/prop set up - character skeleton topology (bones/joints number, Dofs for each bone) - location and size of props Marker Setup - the number of markers - where to place markers Given a storyboard or game design, the first step in planning a motion capture shoot is to determine the character and marker setup of any characters or props to be captured. Character setup deals with determining the skeletal topology of the character including number of joints and/or bones and degrees of freedom of each joint. Marker setup refers to the locations of the markers on the performer/prop that will be used to capture the motion. Character setup depends on the marker setup since bone rotations are determined from marker positions. The marker positions used must be adequate to determine bone rotations in the character.

Calibration Camera Calibration: determine the location and orientation of each camera determine camera parameters (e.g. focal length) Subject calibration - determine the skeleton size of actors/actresses (.asf file) - relative marker positions in terms of bones - determine the size and location of props

Processing Markers Each camera records capture session Extraction: markers need to be identified in the image determines 2d position problem: occlusion, marker is not seen use more cameras Markers need to be labeled which marker is which? problem: crossover, markers exchange labels may require user intervention Compute 3d position: if a marker is seen by at least 2 cameras then its position in 3d space can be determined http://www.xbox.com/NR/rdonlyres/3164D1BE-C1C4-46A1-90F0-26507CF2C9BD/0/ilmnflfever2003lightscam001.jpg

Data Process Complete 3D marker trajectories (.c3d file) 3D marker positions (.c3d file) Fill in missing data Filter mocap data

Data Process Complete 3D marker trajectories (.c3d file) 3D marker positions (.c3d file) Fill in missing data Filter mocap data Inverse Kinematics Joint angle data (.amc file)

Data Process How to represent motion data in joint angle space? Complete 3D marker trajectories (.c3d file) 3D marker positions (.c3d file) Fill in missing data Filter mocap data Inverse Kinematics How to represent motion data in joint angle space? Joint angle data (.amc file)

Motion Capture Data Files Each sequence of human motion data contains two files: Skeleton file (.asf): Specify the skeleton model of character Motion data file (.amc): Specify the joint angle values over the frame/time Both files are generated by Vicon software 35

Human skeletal file Described in a default pose 36

Human skeletal model This is still a tree! 37

Human skeletal model How to describe the skeletal model? What should you know about each bone? This is still a tree! 38

Human skeletal file (.asf) individual bone information - length of the bone - direction of the bone - local coordinate frame - number of Dofs - joint limits bone hierarchy/connections 39

Individual bone information begin id bone_id                  /* Unique id for each bone */ name bone_name        /* Unique name for each bone */ direction dX dY dZ    /* Vector describing direction of the bone in world */ coor. system length 7.01722           /* Length of the bone*/ axis 0 0 20 XYZ         /* Rotation of local coordinate system for                                    this bone relative to the world coordinate                                    system. In .AMC file the rotation angles                                     for this bone for each time frame will be                                    defined relative to this local coordinate                                     system**/ dof rx ry rz                /* Degrees of freedom for this bone. limits (-160.0 20.0) /* joint limits*/             (-70.0 70.0)             (-60.0 70.0) end 40

Individual bone information begin id 2                 name lfemur        direction 0.34 -0.93 0    length 7.01722           axis 0 0 20 XYZ         dof rx ry rz                limits (-160.0 20.0)             (-70.0 70.0)             (-60.0 70.0) end xw yw zw 41

Individual bone information begin id 2                 name lfemur        direction 0.34 -0.93 0    length 7.01722           axis 0 0 20 XYZ         dof rx ry rz                limits (-160.0 20.0)             (-70.0 70.0)             (-60.0 70.0) end xw yw zw 42

Individual bone information begin id 2                 name lfemur        direction 0.34 -0.93 0    length 7.01722           axis 0 0 20 XYZ         dof rx ry rz                limits (-160.0 20.0)             (-70.0 70.0)             (-60.0 70.0) end xw yw zw 43

Individual bone information begin id 2                 name lfemur        direction 0.34 -0.93 0    length 7.01722           axis 0 0 20 XYZ         dof rx ry rz                limits (-160.0 20.0)             (-70.0 70.0)             (-60.0 70.0) end yk xk zk xw yw zw Euler angle representation: Rk=Rz(γ)Ry(β)Rx(α) 44

Individual bone information begin id 2                 name lfemur        direction 0.34 -0.93 0    length 7.01722           axis 0 0 20 XYZ         dof rx ry rz                limits (-160.0 20.0)             (-70.0 70.0)             (-60.0 70.0) end yk xk zk xw yw zw - The number of dof for this joint - The minimal and maximum joint angle for each dof 45

Individual bone information begin id 2                 name lfemur        direction 0.34 -0.93 0    length 7.01722           axis 0 0 20 XYZ         dof rx ry rz                limits (-160.0 20.0)             (-70.0 70.0)             (-60.0 70.0) end yk xk zk xw yw zw 1-dof joint 2-dof joint 3-dof joint 46

Individual bone information begin id 2                 name lfemur        direction 0.34 -0.93 0    length 7.01722           axis 0 0 20 XYZ         dof rx ry rz                limits (-160.0 20.0)             (-70.0 70.0)             (-60.0 70.0) end yk xk zk yk+1 Xk+1 begin id 3                 name ltibia        direction 0.34 -0.93 0    length 7.2138           axis 0 0 20 XYZ         dof rx            limits (-10.0 170.0) end zk+1 47

Root representation :root order TX TY TZ RX RY RZ axis XYZ position 0 0 0 orientation 0 0 0 xw yw zw 48

Root representation :root order TX TY TZ RX RY RZ axis XYZ position 0 0 0 orientation 0 0 0 xw yw zw How to compute the coordinate of a joint in the world coordinate frame? 49

Root representation :root order TX TY TZ RX RY RZ axis XYZ position 0 0 0 orientation 0 0 0 xw yw zw How to compute the coordinate of a joint in the world coordinate frame? 50

Hierarchy/Bone Connections begin root lhipjoint rhipjoint lowerback lhipjoint lfemur lfemur ltibia ltibia lfoot lfoot ltoes rhipjoint rfemur rfemur rtibia rtibia rfoot rfoot rtoes lowerback upperback upperback thorax thorax lowerneck lclavicle rclavicle … end 51

Hierarchy/Bone Connections begin root lhipjoint rhipjoint lowerback lhipjoint lfemur lfemur ltibia ltibia lfoot lfoot ltoes rhipjoint rfemur rfemur rtibia rtibia rfoot rfoot rtoes lowerback upperback upperback thorax thorax lowerneck lclavicle rclavicle … end lowerback root rhipjoint 52

Hierarchy/Bone Connections begin root lhipjoint rhipjoint lowerback lhipjoint lfemur lfemur ltibia ltibia lfoot lfoot ltoes rhipjoint rfemur rfemur rtibia rtibia rfoot rfoot rtoes lowerback upperback upperback thorax thorax lowerneck lclavicle rclavicle … end lowerback root rhipjoint lhipjoint lfemur 53

Hierarchy/Bone Connections begin root lhipjoint rhipjoint lowerback lhipjoint lfemur lfemur ltibia ltibia lfoot lfoot ltoes rhipjoint rfemur rfemur rtibia rtibia rfoot rfoot rtoes lowerback upperback upperback thorax thorax lowerneck lclavicle rclavicle … end lowerback root rhipjoint lhipjoint lfemur ltibia 54

Hierarchy/Bone Connections begin root lhipjoint rhipjoint lowerback lhipjoint lfemur lfemur ltibia ltibia lfoot lfoot ltoes rhipjoint rfemur rfemur rtibia rtibia rfoot rfoot rtoes lowerback upperback upperback thorax thorax lowerneck lclavicle rclavicle … end lowerback root rhipjoint lhipjoint lfemur ltibia lfoot 55

Hierarchy/Bone Connections begin root lhipjoint rhipjoint lowerback lhipjoint lfemur lfemur ltibia ltibia lfoot lfoot ltoes rhipjoint rfemur rfemur rtibia rtibia rfoot rfoot rtoes lowerback upperback upperback thorax thorax lowerneck lclavicle rclavicle … end lowerback root rhipjoint lhipjoint lfemur ltibia lfoot ltoe 56

What can we do with .asf file? We can visualize the default pose We can compute various transforms in the default pose - between world coordinate frame and local coordinate - between parent coordinate frame and child coordinate frame 57

From local coordinate to world coordinate xw yw zw yk xk zk 58

From local coordinate to world coordinate xw yw zw ? ? yk xk zk 59

From local coordinate to world coordinate xw yw zw yk 60

From local coordinate to world coordinate xw yw zw yk xk zk 61

From child to parent node How to Compute the transformation Tkk-1 from a child local coordinate frame to its parent local coordinate frame Tkk-1 x 62

Bone transform parent Tkk-1? world child 63

Bone transform parent Tkk-1? world child 64

Bone transform parent Tkk-1? world child 65

Motion data file (.amc) i // frame number root 2.36756 16.4521 12.3335 -165.118 31.188 -179.889 // root position and orientation lowerback -17.2981 -0.243065 -1.41128 // joint angles for lowerback joint upperback 0.421503 -0.161394 2.20925 // joint angles for thorax joint thorax 10.2185 -0.176777 3.1832 lowerneck -15.0172 -5.84786 -7.55529 upperneck 30.0554 -3.19622 -4.68899 head 12.6247 -2.35554 -0.876544 rclavicle 4.77083e-014 -3.02153e-014 rhumerus -23.3927 30.8588 -91.7324 rradius 108.098 rwrist -35.4375 rhand -5.30059 11.2226 rfingers 7.12502 rthumb 20.5046 -17.7147 lclavicle 4.77083e-014 -3.02153e-014 lhumerus -35.2156 -19.5059 100.612 66

Motion data file (.amc) - Rotation described in local coordinate frame i // frame number root 2.36756 16.4521 12.3335 -165.118 31.188 -179.889 // root position and orientation lowerback -17.2981 -0.243065 -1.41128 // joint angles for lowerback joint upperback 0.421503 -0.161394 2.20925 // joint angles for thorax joint thorax 10.2185 -0.176777 3.1832 lowerneck -15.0172 -5.84786 -7.55529 upperneck 30.0554 -3.19622 -4.68899 head 12.6247 -2.35554 -0.876544 rclavicle 4.77083e-014 -3.02153e-014 rhumerus -23.3927 30.8588 -91.7324 rradius 108.098 rwrist -35.4375 rhand -5.30059 11.2226 rfingers 7.12502 rthumb 20.5046 -17.7147 lclavicle 4.77083e-014 -3.02153e-014 lhumerus -35.2156 -19.5059 100.612 - Rotation described in local coordinate frame - Euler angle representation x-y-z 67

Composite 3D Transformation From .asf file 68

Composite 3D Transformation From .amc file 69

Composite 3D Transformation 70

Composite 3D Transformation 71

Composite 3D Transformation 72

Composite 3D Transformation 73

Motion capture data http://mocap.cs.cmu.edu/