1. CS 527 – Computer AnimationOctober 17, 2006 Estimating Cloth Simulation Parameters from Video Kiran S. Bhat, Christopher D. Twigg, Jessica K. Hodgins, Pradeep K. Khosla, Zoran Popovic and Steven M. Seitz, School of Computer Science, Carnegie Mellon University Department of Computer Science and Engineering, University of Washington Presented by: Ratko Jagodic

2. CS 527 – Computer AnimationOctober 17, 2006 Cloth Need realism for mixed CG/real scenes Realism now possible

3. CS 527 – Computer AnimationOctober 17, 2006 Simulating Cloth Cloth characteristics determined by resistance to: –Bending, stretching, shearing, external forces, aerodynamic effects, friction and collisions Great results are achievable with the right set of parameters BUT, difficult and time consuming to choose parameters Different approach: estimate parameters from real-life video of cloth movement

4. CS 527 – Computer AnimationOctober 17, 2006 Related Work Various cloth models: –Using stiff springs (Baraff and Witkin) –Bending energy model (Choi and Ko) Modeling cloth collisions and friction: –Untangling cloth –Preemptively avoiding collisions Estimating parameters: –Mechanical tests measuring force needed for deforming (Breen) –Searching for parameters that match a piece of real cloth draped over a sphere (Jojic and Huang) –Neither considered dynamic parameters

5. CS 527 – Computer AnimationOctober 17, 2006 Proposed Approach Estimate parameters from the video of moving cloth pieces Choose simulated cloth parameters based on a cloth model Apply an optimization function until the simulated cloth behaves like the real one from the video

6. CS 527 – Computer AnimationOctober 17, 2006 Cloth Model Cloth model requirements: –Realism and practicality –Scalable to varying resolutions of cloth –Input parameters independent of meshing Baraff and Witkin cloth model was chosen with minor adjustments Collisions based on Bridson’s model (again, slightly modified)

7. CS 527 – Computer AnimationOctober 17, 2006 A Metric for Matching Simulation to Video Two sequences compared frame-by-frame Average error is computed across the whole sequence Perceptually motivated metric –Human perceptual system is sensitive to moving edges in video –Moving edges = folds in cloth The metric returns the difference in folds between the real and simulated cloth

8. CS 527 – Computer AnimationOctober 17, 2006 Fold Detection and Representation Project striped pattern on pieces of cloth to eliminate lighting and material reflectance Compute dominant orientations of edge pixels  angle map Threshold the gradient of the angle map  gradient mask

9. CS 527 – Computer AnimationOctober 17, 2006 Fold and Silhouette Comparison Angle map from videoAngle map from simulation Angle map difference Angle map diff x gradient mask Silhouette Comparison – the difference between the shapes

10. CS 527 – Computer AnimationOctober 17, 2006 Optimization Before optimization: –4 values are chosen for each parameter across its entire range –Then simulate the fabric for each of those 4 points and compare with the real fabric. –Use the best of the 4 points to start the optimization with Optimization: –Uses simulated annealing for n-variable optimization

11. CS 527 – Computer AnimationOctober 17, 2006 Experiments Perform simpler experiments that would yield parameters for the cloth model = calibration Fabric: –Linen, fleece, satin and knit Static Test: –How the fabric naturally hangs under gravity Waving Test: –One corner is fixed and the other moved back and forth Swatches of cloth were used –give reasonable estimates without the need to optimize on complex fabric geometries

12. CS 527 – Computer AnimationOctober 17, 2006 Experimental Setup Setup: –Vicon motion capture system for locating two top corners of fabric –Motion capture markers for calibrating the camera and the projector –2.8 GHz Xeon processor Trials: –2 trials per fabric –1 trial = 50 frames of video –each takes 50 hours to optimize Parameters: –Bend, stretch, shear, bend damping, stretch damping, shear damping, linear drag, quadratic drag, drag degradation

13. CS 527 – Computer AnimationOctober 17, 2006 Results – Static Test Good estimates for the parameters that have low variability (stretch and bend) Bad estimates for high variability parameters since they do not contribute much to the total error (dynamic parameters like damping)

14. CS 527 – Computer AnimationOctober 17, 2006 Results – Waving Test Added air drag parameters Results good but probably needed longer or more complex sequences since a few parameters still had high variability

15. CS 527 – Computer AnimationOctober 17, 2006 More Results Evaluated obtained parameters on longer sequences (150 frames) = good match Evaluated on a more complex piece of cloth (a skirt) = approximately correct

16. CS 527 – Computer AnimationOctober 17, 2006 Conclusion The estimation produces parameters that yield appealing results. Realism in the end is still dependent on the cloth model so good ones are needed Future work: –Different metrics could be explored (e.g. time varying as opposed to frame-by-frame) –More complex garments may need additional test that mimic behaviors better suited to a particular application of fabric (such as form fitting pants) –Different parameters are needed for more complex effects (e.g. stretching in one direction causing shrinking in the other)

17. CS 527 – Computer AnimationOctober 17, 2006 Video Movie

18. CS 527 – Computer AnimationOctober 17, 2006 Questions?

19. CS 527 – Computer AnimationOctober 17, 2006 Simulated Annealing