1. 1 Toward real-time multi-model physical simulation Laurent Grisoni December, 9th, 2005 LIFL (USTL), INRIA Futurs/IRCICA, CNRS

2. 2 Short CV Dec. 1999: PhD thesis from University of Bordeaux I, Subject: multiresolution for geometric modeling (on splines, and implicits) From Sept. 2000: assistant professor at Polytech’Lille engineering school Sept 2003  Sept 2005: INRIA temporary full-time research position Research advising: -2 completed PhD thesis (J. Lenoir and J. Dequidt) -2 on the run (A. Theetten and R. Vatavu) -6 Master thesis advised since 1997. Toward real-time multi-model physical simulation

3. 3 introduction Main research context: « real-time » physical simulation  Allow users to interact on virtual models that behave like real-life ones  Real-time physical simulation for end-point applications involves several points: -Collision detection and response -Mechanical modeling -Visual modeling (geometry, texturing, etc…) -Interaction with human (possibly haptic): not covered by the whole of this expose (only interaction between mechanical systems here)  For designing a sophisticated model in a simulator, we usually see an object as a four-layered box: « multi-layered objects » [Meseure 02]  Each layer is a wide research subject in itself  The question to know how each interacts with others raises many interesting research problems (e.g. dynamic cutting?) Toward real-time multi-model physical simulation [Müller 04]

4. 4 introduction Medical simulation is an intrinsically complex application field:  big issue to provide complex protocols  simulation speed, visual and mechanical accuracy are crucial  Strong application potential (teaching, pre-op) Toward real-time multi-model physical simulation

5. 5 introduction Some physical phenomena are known to be complex … Toward real-time multi-model physical simulation

6. 6 introduction … but most are more complex than we usually perceive. Toward real-time multi-model physical simulation

7. 7 introduction In physical simulation: Each model has its own cost, and representation limits Toward real-time multi-model physical simulation

8. 8 introduction For a given object, choice of model usually (implicitely) depends heavily on the context of use As computer scientists, we would like: Flexibility, Modularity, Re-usability, Etc…,  not the case at all so far, and there is much more than coding here Toward real-time multi-model physical simulation

9. 9 introduction How to overcome such limits for real-time physical simulation?  important to understand each model potential, and limits  provide as much flexibility to models as possible, through adaptivity  propose mechanical systems that would be seen as black boxes, dynamically adapting their representation to context of use: multi- model objects Toward real-time multi-model physical simulation

10. 10 introduction According to our perception of research activity in each part of the global problem: Toward real-time multi-model physical simulation Collision & response MechanicalVisual Model representation potential Poor activityActive field adaptivity Active field Still emergingActive field Multi-modelActive field emerging Quite classical (this array does not take into account non-separable aspects…e.g. cutting) : presented contributions

11. 11 overview Models in physical simulation Adaptivity Multi-model Conclusion and Future work Toward real-time multi-model physical simulation

12. 12 overview  Models in physical simulation Adaptivity Multi-model Conclusion and Future work Toward real-time multi-model physical simulation

13. 13 Models in physical simulation Collision detection: 2 sub-classes of problems Geometrical detection –Huge litterature –Only simple primitives are accessible in real-time –Discrete, or continuous… time matters here. From detection to response –Not necessarily a force (e.g. exact contact handling [Cani 93]) –Most often relies on interpenetration distance between objects –Difficult, so far, to overcome discrete nature of simulation Toward real-time multi-model physical simulation

14. 14 Mechanical modeling : 3 sub-class of problems From Forces to PDE: –Many different models: mass-spring, particules, FEM, meshless… Movement integration –Several techniques available –Implies strong mathematical background for contributing –Main trend: choose between stability, accuracy, and computation cost Constraints between objects –Anywhere forces are no longer enough for simulation –Main trend: choose between constraint respect and system stability Toward real-time multi-model physical simulation Models in physical simulation

15. 15 Models in physical simulation Visual modeling Refers to geometry, texturation, real-time rendering: once again, active research fields From geometry point of view, usefullness of models usually depend on object dimension: –1D: splines, subdivision, implicits, generalized cylinders –2D: meshes, tensor-based splines, subdivision surfaces, point-based reconstruction –3D: implicits, volumic spatial partitionning, particule-based density fields Each has its pros and cons More or less expensive More or less topologically flexible (dynamic cut)  We think real-time physical simulator is a healthy context for research on geometry/texture manipulation, and real-time rendering in a more general way. Toward real-time multi-model physical simulation

16. 16 Contribution: fast visualisation of implicit using controlled blending and contact (Graphite 03, with F. Triquet) Blending graph represented as binary matrix, handled using dword bit-clusters For each primitive, operations on blending list is expressed as bitwise operations on dwords Adaptation of marching cube to contact control  Improvement of speed and visual result Toward real-time multi-model physical simulation Models in physical simulation

17. 17 Models in physical simulation Contribution: deformable 1D model (J. Lenoir PhD thesis) Deformable spline model (MS4CMS’02) –Based on Lagrange equations –Provides real-time, continuous mechanics Set of constraints using Lagrange multipliers (Graphite 03) –Sliding point constraints Toward real-time multi-model physical simulation

18. 18 overview Models in physical simulation  Adaptivity Multi-model Conclusion and Future work Toward real-time multi-model physical simulation

19. 19 About terminology: Blurry definition, throughout litterature, of words such as « multiresolution », « adaptive », « hierarchical » « level-of –detail » seems the most general one « adaptive » can relate to basic methods that refine representation where needed. Two questions may help classify things: –Is the studied problem taking advantage of the hierarchy to speed up the calculus? (yes: multiresolution; no: adaptive) –does the representation have the same memory cost as the non-hierarchical version? (yes: multiresolution/hierarchical; no: level-of-details)  Another problem: no specific keyword distinction between time and space Toward real-time multi-model physical simulation adaptivity

20. 20 Mechanical modeling Many results on generating PDE on adaptive models [debunne00, grinspun02] Few on numerical integration for real-time: –a lot on PDE solving using wavelets, based on Galerkin’s work on linear operators –Not usable within real-time context… To our knowledge, nothing on adaptive handling of constraints ([Redon 05]?) Toward real-time multi-model physical simulation adaptivity

21. 21 Collision detection Very classical to use bounding-box hierarchy (ex: k-DOP [Klosowski 98]) Seen as the most practical way to overcome the cost of that stage Available on low-deformable volumes [James 04] Some results exist about anytime-interruptible collision detection schemes ([O’Sullivan 01]) Toward real-time multi-model physical simulation adaptivity

22. 22 Geometric modeling Splines have important wavelet- based framework for hierarchical manipulation Sweldens’ Lifting Scheme opens road to biorthogonal constructions for any interpolation scheme Subdivision curves and surfaces are intrinsically adaptive, and (theoretically) parameterized  Strong theoretical support, yet difficult so far to handle for real- time physical simulation skinning Toward real-time multi-model physical simulation adaptivity

23. 23 adaptivity Contribution: High performance Generalized Cylinders Visualization (Shape Modeling 03) (with D. Marchal) Work achieved during the ARC SCI (collaboration with EVASION project, and IRCAD) Tesselation achieved using vertex blending, combined with non-deformed surface primitive Provides significant improvement of surface drawing quality, and speed First example of external control for guaranteed frame-rate Toward real-time multi-model physical simulation

24. 24 adaptivity Contribution: adaptive 1D mechanical B- spline (Graphite 2005) (J. Lenoir PhD thesis) B-spline provide strong theoretical support for adaptivity (refinable basis functions, knot removal techniques) Mechanical model is adapted to B- spline refinement, and knot removal  Thread cutting provided at no-cost  Interesting first example of a model that semantically becomes fully independant from its resolution Toward real-time multi-model physical simulation

25. 25 adaptivity Contribution: Time-critical deformation solids (CASA’05) (J. Dequidt PhD thesis, collaboration with D. Marchal) Adaptive octree physical simulation, used as FFD for animating complex meshes Control loop supervises the model for automatic tuning of model to computation cost (under some tolerance) From multi-layer objects point of view: –adaptive collision, and mechanics –generic deformable model for complex (topologically constant) shapes Toward real-time multi-model physical simulation

26. 26 overview Models in physical simulation Adaptivity  Multi-model Conclusion and Future work Toward real-time multi-model physical simulation

27. 27 Multi-model Mechanical Modeling Few yet about such use for mechanical modeling Some results about level-of-details for animation [Carlson 97] Use of corotationnal [Felippa 00] for large deformations can be seen as a use of rigid rules into deformable Some work use rigid approximations when needed (e.g. explosions) [Müller 05] takes the best of each world between rigid and deformable Toward real-time multi-model physical simulation

28. 28 Multi-model Collision detection Collision pipe-line [Zachman 01] is already multi-model No real study yet, to our knowledge, on dynamic switching from a collision algorithm to another one (e.g. discrete-continuous) Toward real-time multi-model physical simulation

29. 29 Multi-model For visual modeling Level-of-detail is well-known for geometry handling: notion of cost function [Funkhouser 93] Some work on smooth transition between color texture, bump and displacement textures [Becker 93] Toward real-time multi-model physical simulation

30. 30 Multi-model Contribution: Asynchronous interactive Physical Simulation (INRIA Tech. Report) (J. Dequidt PhD thesis) First software framework for such objects Each mechanical system lives in its own world Practical implementation of each system has no influence on the global software structure Toward real-time multi-model physical simulation

31. 31 overview Models in physical simulation Adaptivity Multi-model  Conclusion and Future work Toward real-time multi-model physical simulation

32. 32 Conclusion and Future Work Short-term directions of work: -Surgical thread simulation (A. Theetten thesis, co- advising with french CEA) -Dynamic cutting -First multi-model study -What about GPU? Toward real-time multi-model physical simulation

33. 33 Conclusion and Future Work The SOFA Project: Simulation Open-Framework Architecture Co-initiated by the simulation group at CIMIT (MIT/Harvard Hospital) and ALCOVE project Currently joined effort of 4 research teams (CIMIT Sim. Group, EVASION, EPIDAURE and ALCOVE) Supported by the Odysseus european project 3 INRIA engineers currently working on the project: P.-J. Bensoussan, S. Fonteneau, F. Poyer  goal:  provide modularity to real-time physical simulation  Application designers should be able to exchange functionnal moduluses (generic API for each bloc)  Take advantage of future multi-processors architectures Toward real-time multi-model physical simulation

34. 34 Conclusion and Future Work Toward real-time multi-model physical simulation The core is almost done. We can now start to split work into small activity groups, and have other research teams participate…

35. 35 Conclusion and Future Work Toward real-time multi-model physical simulation Mid and long-term research: we think real-time physical simulation would get much of:  extend, as much as possible, model limits for representing real world Mechanical accuracy for real-time (what about computation time as a physical perception parameter??) Computation cost (the smaller, the better) Topological control (e.g. cutting) Get free from classical link between visual skinning and mechanical model  provide flexibility to models: Adaptivity anywhere possible Dynamic control of such adaptivity (AI? Robotics?)  propose multi-model objects: Evaluate each model limits Propose ways to switch from a representation to another: how, and when (AI?) Provide software solutions for manipulation of such objects (collision, constraints, etc…) (potentially much to get from software research community) [F. Blondel, S. Karpf]