Motion Capture

Animation – A broad Brush Traditional Methods  Cartoons, stop motion Keyframing  Digital inbetweens Motion Capture  What you record is what you get Simulation  Animate what you can model (with equations)

What is Motion Capture ? Capture of motion of (human) actor  Motion Capture of an object involves sensing, digitizing, and recording that object in motion Whole body Hands Face One way of acting out an animation

Why Motion Capture ? All the fine details of human motion will be reproduced Naturalness of human motion is in its subtle details  Style  Mood  Weight shift, and so on Applications  Animation / Interactive characters  Feature films / video games  Medicine  Sports  Robotics

What is captured ? What we need is  Position and orientation of the root segment  Joint angles  Length of each link (is it possible ?)  Skeleton connectivity  Skin deformation The mocap system actually provides  Marker positions on the skin, or  The positions and orientations of markers

Motion Capture Pipeline Record motion from physical object Use motion to animate virtual object Set up Equipment Set up Equipment Record Motion Record Motion Process Data Process Data Generate Animation Generate Animation

What Types of Objects? Human whole body Portions of body Facial animation Animals Puppets Other objects

How to use the data? Off-line  Processing by filtering, inverse kinematics  Produce libraries of motion trajectories Choose among them Blend between them Modify on the fly On-line (performance animation)  Driving character directly based on what actor does in real time

Motion Capture Technologies Optical passive Optical active Electromagnetic Electromechanical Acoustic Optical fiber

Optical Passive High resolution, high speed cameras  Hz, 1000x1000 pixels  Infrared or visible light strobe  Retro-reflective markers Pros  High quality  Flexible marker placement  No cables  Not seriously constrained by markers Cons  Extensive post-processing  Controlled environments (Indoor only, no sunlight)  Correspondence problem  Occlusion

Optical Active Phoenix Tech, ReActor, Optitrak, Visualeyez Markers emit electric signal (LEDs) No correspondence problem

Electromagnetic Electromagnetic field transmitter Sensors on the body  Both position and orientation information Pros  Realtime  No occlusion/correspondence Cons  Limited accuracy  Smaller workspace  Heavier sensors and wires on body  Sensitive to electromagnetic interference Company: Ascension, Polhemus

Electromechanical Exoskeleton Mechanical skeleton attached on body Pros  Truly realtime (500 Hz)  No range limit  No occlusion/correspondence problem Cons  Restriction of movement  Fixed sensor positions Company: Sacros, Gypsy

Optical Fiber Bending the fiber attenuates the transmitted light Recently used for full-body capture Fiber Optic Sensors  Flexible FO sensors strapped to various parts of the performers body.  Sensors can directly measure joint rotations  Used in conjunction with electromagentic sensor for head and torso. Shapewrap II  Measurand

Acoustic The multiple transmitters trigger “click” The receivers on the body measures the time taken for the sound travel Pros  No occlusion Cons  Limited range  Limited number of sensors  Cables on body  Acoustic interference

Motion Capture Systems Challenges:  Signal is not perfect Noisy missing data not perfectly aligned with joints  Retargeting Data is only valid for virtual character who possesses same scale as real character.

Motion Capture Systems Challenges:  Even if motion capture data was perfect, we still have the following challenges: Re-use – use the motion for a slightly different purpose Creating impossible motion – Motion capture won’t do it, but may be desired in animation Change of intent – we can’t always predict what motion we will need  Take Home Message: Motion Capture captures a particular, single motion.

Motion Capture Data So what CAN we do with motion capture data?  We can speed up slow down time warp Motion warp  However, one must remember that Captured data is Sampled Data.

Retargeting Motion Capture Data In general, moCap data is useful for a single articulated figure.  Adapt motion to another character

Motion capture data formats No “standard” moCap data format  Defacto standards from motion capture system manufacturers  Must specify both structure of skeleton as well as sampled data for each joint.

Demo motion capture technologies  Vicon (passive optical) Vicon  PTI (active optical) PTI  Ascension technolgy (magnetic) Ascension technolgy  MetaMotion (mechanical) MetaMotion  OrganicMotion (markerless) OrganicMotion  Image Metrics (markerless -facial) Image Metrics  Moven (inertial) Moven  LumiNetra (light aware SIGGRAPH 2007 paper) LumiNetra  Mova (make-up) Mova

Motion capture data formats Popular formats  Acclaim File Format.asf (Acclaim skeleton format).amc (Acclaim motion capture)  Biovision.bva (BioVision animation).bvh (BioVision Hierarchical)  C3D Independent Binary format with programmer support.

Acclaim The Acclaim format is made up of two files  a skeleton file the ASF file (Acclaim Skeleton File).  A motion file the AMC file (Acclaim Motion Capture data).

Acclaim Skeleton File Parsing the ASF File  “:version” the version of the skeleton definition  ":name" the skeleton to be named  ":units" defines the units to be used for various types of data Ex) mass, length, angle  ":documentation"  ":root" the root segment of the skeleton hierarchy "axis" : defines the rotation order of the root object "order" : order they will appear in the AMC file –TX TY TZ RZ RX RY  Translation & Rotation Order "position" and "orientation" –the starting position and orientation of the root –These are typically, but not always, zero

Acclaim Skeleton File Parsing the ASF File  “:bonedata” a description of each of the segments "begin" and "end" pair “id” : a unique id for the segment "name" : the name of the segment "direction" –defines the direction from the parent to the child segment "length" : The length of the segment "axis" : an axis of rotation for the segment "dof" : specifies the number of motion channels "limits" : limits on each of the channels in the dof specification

Acclaim Skeleton File Parsing the ASF File  “:hierarchy” Describes the hierarchy of the bones declared in the “:bonedata” section :hierarchy begin root hips hips hips1 hips2 hips3 hips1 chest chest chest1 chest2 chest3 chest1 neck neck head chest2 leftcollar leftcollar leftuparm leftuparm leftlowarm leftlowarm lefthand chest3 rightcollar rightcollar rightuparm rightuparm rightlowarm rightlowarm righthand hips2 leftupleg leftupleg leftlowleg leftlowleg leftfoot hips3 rightupleg rightupleg rightlowleg rightlowleg rightfoot end

Acclaim Motion Capture Data Parsing the AMC file  defines the actual channel animation  a line declaring the frame number  the bone animation data bone name and data for each channel defined for that bone. Tool  ASF/AMC Viewer ASF/AMC Viewer

Biovision BVH BVH(BioVision Hierarchical data )  developed by Biovision, a motion capture services company  A BVH file has two parts a header section –describes the hierarchy and initial pose of the skeleton a data section –contains the motion data

Biovision BVH Header section  "HIERARCHY“ "ROOT" –followed by the name of the root – " { " and " } " pair –"OFFSET" »X,Y and Z offset of the segment from its parent –"CHANNELS" »the number of channels »the type of each channel "JOINT" –identical to the root definition except for the number of channels –"OFFSET ", "CHANNELS" "End Site" –indicates that the current segment is an end effector (no children) –"OFFSET " - 6 channels for the root (Tx Ty Tz Rz Rx Ry) - 3 channels for every other object (Rz Rx Ry)

Biovision BVH Motion Section  "MOTION" followed by a line indicating the number of frames "Frames:" –the number of frames "Frame Time:" –the sampling rate of the data –Ex)  30 frames a second The rest of the file contains the actual motion data –The numbers appear in the order of the channel specifications as the skeleton hierarchy was parsed

Biovision BVH Interpreting the data  To calculate the position of a segment Translation information –For any joint segment »the translation information will simply be the offset as defined in the hierarchy section –For the root object »The translation data will be the sum of the offset data and the translation data from the motion section Rotation information –comes from the motion section

Motion Tools Tools  Bvhacker:  Avimator:  Qavimator:  Motion Capture Display and Editing: Motion Capture Display and Editing:  ASF/AMC viewer ASF/AMC viewer

Examples Avimator 1. The File Menu File management, Program options, and Settings. 2. Figure: The Posing / Animation dummy (or Actor), standing on the stage: This is our main click and drag edit element. 3. Frame Slider Bar: the control to move between frames. 4. Edit Part Menu: An alternative to click select. A Menu to pick a particular body part to be edited. 5. Rotational Sliders: Fine controls to edit the X,Y,Z coordinates of selected body part. 6. Play Button / Keyframe Button: Preview plays animation / Sets the “Keyframe” of an animation. 7. Frame Quantity edit box: This is the control to edit the number of frames in an animation.

Examples : Avimator Step 1  Open “Relaxed.bvh”  Change the number of frames to “90” Step2  choosing male or female and turning off the joint limits. Step3  Keyframe Animation in between frames (=Keyframe) Step4  Move to frame “2” and click the “key frame” button.  Setup our Actor to the starting position of the salute. 1. Shift Double-Click rShldr X=-4 Y=-4 Z=82 2. Shift Double-Click rForeArm X=16 Y=9 Z=0 3. Shift Double-Click rThigh X=0 Y=1 Z=0 4. Shift Double-Click rShin X=1 Y=1 Z=0 5. Shift Double-Click rFoot X=0 Y=-3 Z=-4

Examples : Avimator Step 5  Move to frame “30”. 1. Select rCollar X=0 Y=-52 Z= Select rShldr X=23 Y=106 Z=82 3. Select rForeArm X=-12 Y=99 Z=-9 Step 6  Frame “30”  COPY FRAME.  move to frame “80”  PASTE FRAME  click the “Key Frame” button Step 7  Frame “2”  COPY FRAME.  move to frame “90”  PASTE FRAME  click the “Key Frame” button Step 8  play our animation  our actor (or actress) should be repeatedly saluting us. Step 9  save our file as before with FILE and SAVE. “salute_anim.bvh”