Optical Motion Capture Bobby Bruckart Ben Heipp James Martin Molly Shelestak

Outline History –Types of Motion Capture –Development of Motion Capture State-of-the-Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future –Possible solutions to current problems –Possible future paths

Types of Motion Capture History –Types of Motion Capture –Development of Motion Capture State-of-the Art Future 1.Mechanical Motion Capture 2.Optical Motion Capture 3.Electromagnetic (magnetic) Motion Capture

Mechanical Motion Capture performer wears a human-shaped set of straight metal pieces other types: gloves, mechanical arms, articulated models pro: –no interference from light or magnetic fields con: –the technology has no awareness of ground –equipment must be calibrated often. –does not know which way the performer's body is pointing –absolute positions are not known but are calculated from the rotations

Optical Motion Capture performer wears reflective dots that are followed by several cameras and the information is triangulated markers are either reflective, or infra-red emitting developed primarily for biomedical applications pro: –performer feels free to move due to no cables connecting body to the equipment –larger volumes possible –more performers are possible –very clean, detailed data con: –it is prone to light interference –occlusion – rotations of body parts must be solved for and are not absolute –performer must wear a suit with dots and balls –information has to be post-processed or 'tracked' before viewing –comparatively high cost

Electromagnetic Motion Capture performer wears an array of magnetic receivers which track location with respect to a static magnetic transmitter First use: military often layered pro: –positions are absolute, rotations are measure absolutely –can be real-time –relatively cheaper than optical con: –magnetic distortion –data prone to noise and interference –performers wear cables connecting them to a computer –relatively low sampling speed

Development History –Types of Motion Capture –Development of Motion Capture State-of-the Art Future

Early 1980s – Beginning of optical systems MIT Architecture Machine Group & New York Institute of Technology Computer Graphics Lab experimented with optical tracking of the human body few cameras and markers with limited success. created virtual stick-figure marionette

Jim Henson Productions ability to control the position and mouth movements of a low resolution character in real-time Waldo C. Graphic

1988: deGraf/Wahrman “Mike the Talking Head” shows off the real-time capabilities driven by a specially built controller The Silicon Graphics hardware provided real-time interpolation between facial expressions and head geometry as controlled by the performer

1989 Kleiser-Walczak Dozo used optically-based solution from Motion Analysis (multiple cameras to triangulate the images of small pieces of reflective tape placed on the body) resulting output is the 3-D trajectory of each reflector in the space

SimGraphics facial tracking system called a "face waldo.“ could track the most important motions of the face and map them in real-time onto computer puppets Important: one actor could manipulate all the facial expressions of a character by just miming the facial expression himself

SIGGRAPH 1993: Acclaim realistic and complex two-character animation done entirely with motion capture developed a high-performance optical motion tracking system able to track up to a 100 points simultaneously in real-time used the system to generate character motion sequences for video games

Optical Motion Capture History State-of-the Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future Data Acquisition

Optical Motion Capture History State-of-the Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future Data Translation

Optical Motion Capture History State-of-the Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future Data Implementation

Data Collection Algorithms History State-of-the Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future Cameras placed around center of room Recording at 15fps Cameras calibrated into common coordinate system Voxel-based volume rendered through a space-carving technique Also uses color-consistency to enhance quality

Data Collection Algorithms History State-of-the Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future Run a quick test to determine if the point lies on, in, or outside the shape Moment analysis is then used to determine other information about the point The fitting error is then found A split and merge technique is used to recreate the volume

Data Collection Algorithms History State-of-the Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future The whole voxel volume is approximated as one ellipsoid It is subdivided into two ellipsoids so D is less than some threshold value A new voxel set is created from a pairing of voxel sets and neighboring sets A novel ellipsoid is paired with this new set D is computed, the smallest D value is used to replace the two ellipsoids its replacing This is performed only at the first time step to reduce error in merging

Data Collection Algorithms History State-of-the Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future Problems arise when there are differing numbers of ellipsoids from one time step to the next ellipsoid sets are merged

Data Collection Algorithms History State-of-the Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future To identify rigid body structures, it must be known that the mutual Euclidean distance between two points will not change if they are in the same rigid body The paths through the time steps are compared in an iterative fashion until all ellipsoids are handled

Data Collection Algorithms History State-of-the Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future Two adjacent rigid bodies found in the previous step are used to sample points along any of the three major axes which lie on this ellipse The number of point samples increases along all three axes simultaneously until the average 3D location of these points lies within the adjacent ellipse (or vice versa)

Pros of Optical Motion Capture History State-of-the Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future Pros

Cons of Optical Motion Capture History State-of-the Art –Hardware / Software –Data Collection Algorithms –Pros & Cons Future Cons

Current Issues History State-of-the Art Future –Possible solutions to current problems –Possible future paths 1.Geometric Dissimilarity Make human-shaped data work on one of these characters, without introducing effects of mismatched proportions

Current Issues History State-of-the Art Future –Possible solutions to current problems –Possible future paths 2. Different movement qualities in performers and cartoon characters Ability to adjust the speeds of capture to simulate more cartoon-like movements when capturing natural human movement

Current Issues History State-of-the Art Future –Possible solutions to current problems –Possible future paths 3.Increasing data collection ability Make motion capture easier to process with increased technology in the current software

Current Issues History State-of-the Art Future –Possible solutions to current problems –Possible future paths 4. Increased number of performers which can be captured simultaneously Ability to capture groups of performers with out reducing the image quality

Future Paths History State-of-the Art Future –Possible solutions to current problems –Possible future paths  enhancement of performance conditions through lack of tethering and simplification of performance apparel  increased speed of the technology  increased 'volume' or area in which performances can be captured  lower cost, so that consumers and independent artists can have access & experiment / expand the technology

Future Paths History State-of-the Art Future –Possible solutions to current problems –Possible future paths  increased accuracy of the results, including improved physical abilities, so that characters can touch each other and feet meet solidly on the ground  greater ability to capture data from multiple characters  combination of virtual reality and existing motion capture technologies could aid in technological development

Discussion Questions 1.Should using motion capture for animation be considered art? 2.Could motion capture be used to create realistic human motion in robotics? 3.What would be some other ways to capture motion? Audio? 4.Four