1. Physics and Sound Zhimin & Dave

2. Motivation Physical simulation Games Movies Special effects

3. What is done now? Foley artist Lucasfilm Foley Artist

4. VibrationPressure WavePerception How sound works Transverse waves Doppler effect

5. Why sound rendering is hard Very small time step Ideally 44.1kHz Small time step at lest 10 -6 Stability

6. Sound Rendering Method for synchronized soundtracks and animations Sound scripts Sound rendering pipeline Key framing audio, Approximate refraction, Different models for sound generation, etc.

7. Sound Rendering Sound rendering definition –The process of creating a composite sound track from component sound objects Sound scripts –Control sound instantiation and modification User specified From system, ex. simulation, periodic, etc.

8. Sound Rendering Pipeline Vibrational Analysis Instantiate Sound Propagation Resample Modeling Rendering

9. Vibrational Analysis Generate prototype sound, characteristic of each object and interaction with other objects Physically Based –FEM, Natural vibration modes, Approximation Friction and Turbulence Procedural Sounds (what they used)

10. Instantiate Sound Prototype sound instantiate Attach to some 3D object Start energy of sound wave Use of sound scripts

11. Propagate Physical Simulation Beam tracing Frustum tracing Refraction Effects - reverberation, refraction

12. Resampling Sound prototype to sound track –Sound at time steps (key-frame audio) Amplitude - linear interpolation Frequency - high order approximation or very dense sampling Like texture mapping –Object space algorithm –Image space algorithm –Warning: aliasing problems similar to texture mapping, solved similar methods

13. Sound Rendering Demo

14. Sound Rendering Pipeline Vibrational Analysis Instantiate Sound Propagation Resample Modeling Rendering Deformable body Simulation

15. Deformable objects Piggy back on top of deformable body simulator Use audible vibrational component of simulation Generate sound pressure and model sound propagation

16. Requirements Temporal resolution higher then 10 -5 Dynamic deformation –FEM –mass spring Explicit Surface Representation Physical Realism, if not –sounds will reveal errors in motion –strange sound responses

17. Modeling surface vibration I will put the details of this here (commming soon)

18. Deformable objects Demo

19. Sound Rendering Pipeline Vibrational Analysis Instantiate Sound Propagation Resample Modeling Rendering Physical Shapes

20. NOTE to Ming Some of the slides on Sound of Physical Shapes I am going to replace with an example using a Circular membrane, since I have pictures that it may be more clear.

21. The Sound of Physical Shapes Sounds of physical shapes are important for enhance realism Sounds produced by colliding objects depend on object material, shape and the location of contact. Sounds provide listeners with clues about the object.

22. The framework Precompute characteristic frequencies Divide object boundary into small regions Determine the amplitudes of excitation for an impulse applied to a region Normalize energies energies, so only scaling is needed when rendering

23. Vibrating Shapes from Impact Calculating the Modes: Vibration of the object obeys the wave equation: Initial conditions: Norm of the eigenfunctions:

24. Calculating the Modes: Solution to the wave equation: are related to eigenvalues of That is Vibrating Shapes from Impact

25. Calculating the Modes: Calculating the amplitudes of the vibration modes Time averaged energy of the vibration (if mass distributed uniformly) Vibrating Shapes from Impact

26. Initial conditions:,the k-dimensional Dirac delta function The amplitudes are: The energy is: Mode Amplitudes from Impact Location

27. Compute sound emitted from the vibration: Intensity of the sound emitted: It integrates the intensity of the vibration. Sound Sources from Vibrating Shapes

28. Each mode decays with decay time: is the internal friction parameter, which depends on material only. Having emulated the external damping of object, sound-wave: Sounds and Material Properties

29. Store the pre-computed frequency spectrum, and the excitation spectrum for a number of locations on the surface. Treat two sounds and as the same, if the the difference between them, where is a threshold. Sound Map

30. http://www.cs.ubc.ca/spider/kvd oel/app6JavaSound/demo.html Live Demo

31. Sound Rendering Pipeline Vibrational Analysis Instantiate Sound Propagation Resample Modeling Rendering Rigid body Simulation

32. Sound from Rigid body simulations Precompute shape and frequency of deformation modes –Result, quick computation of vibrational response at runtime Numerically via eigen-decomposition of system of matrices from FEM model Model sound from geometric description and material paarameters

33. Modal Analysis Zhimin and I were planning on ommiting the calculation of the eigenvectors from this so she can speak about it in Nikunj’s paper, as it uses the same process.

34. Rigid body simulation Normal rigid body simulation –One change, export collision forces to sound synthesizing process. One thing… –Sound synthesis can reveal flaws of bad rigid body simulator

35. Deforming the model Need to be able to generate K, C and M Mass spring, FEM etc. FEM used, because of past success, see [OSG02] for details

36. A Sound Generation Alg Details of the alg here

37. How to speed up Eval 18 use previous values e^{\omega(t+s)} = e^{\omega*t} + e^{\omega*s}

38. Other effects Cost of additional impulses is low (analytical solution) Convolve with Gaussian kernel –Force for soft collisions –Noise for small scale friction

39. Rigid body Demo