Physics and Sound Zhimin & Dave

Motivation Physical simulation Games Movies Special effects

What is done now? Foley artist Lucasfilm Foley Artist

VibrationPressure WavePerception How sound works Transverse waves Doppler effect

Why sound rendering is hard Very small time step Ideally 44.1kHz Small time step at lest Stability

Sound Rendering Method for synchronized soundtracks and animations Sound scripts Sound rendering pipeline Key framing audio, Approximate refraction, Different models for sound generation, etc.

Sound Rendering Sound rendering definition –The process of creating a composite sound track from component sound objects Sound scripts –Control sound instantiation and modification User specified From system, ex. simulation, periodic, etc.

Sound Rendering Pipeline Vibrational Analysis Instantiate Sound Propagation Resample Modeling Rendering

Vibrational Analysis Generate prototype sound, characteristic of each object and interaction with other objects Physically Based –FEM, Natural vibration modes, Approximation Friction and Turbulence Procedural Sounds (what they used)

Instantiate Sound Prototype sound instantiate Attach to some 3D object Start energy of sound wave Use of sound scripts

Propagate Physical Simulation Beam tracing Frustum tracing Refraction Effects - reverberation, refraction

Resampling Sound prototype to sound track –Sound at time steps (key-frame audio) Amplitude - linear interpolation Frequency - high order approximation or very dense sampling Like texture mapping –Object space algorithm –Image space algorithm –Warning: aliasing problems similar to texture mapping, solved similar methods

Sound Rendering Demo

Sound Rendering Pipeline Vibrational Analysis Instantiate Sound Propagation Resample Modeling Rendering Deformable body Simulation

Deformable objects Piggy back on top of deformable body simulator Use audible vibrational component of simulation Generate sound pressure and model sound propagation

Requirements Temporal resolution higher then Dynamic deformation –FEM –mass spring Explicit Surface Representation Physical Realism, if not –sounds will reveal errors in motion –strange sound responses

Modeling surface vibration I will put the details of this here (commming soon)

Deformable objects Demo

Sound Rendering Pipeline Vibrational Analysis Instantiate Sound Propagation Resample Modeling Rendering Physical Shapes

NOTE to Ming Some of the slides on Sound of Physical Shapes I am going to replace with an example using a Circular membrane, since I have pictures that it may be more clear.

The Sound of Physical Shapes Sounds of physical shapes are important for enhance realism Sounds produced by colliding objects depend on object material, shape and the location of contact. Sounds provide listeners with clues about the object.

The framework Precompute characteristic frequencies Divide object boundary into small regions Determine the amplitudes of excitation for an impulse applied to a region Normalize energies energies, so only scaling is needed when rendering

Vibrating Shapes from Impact Calculating the Modes: Vibration of the object obeys the wave equation: Initial conditions: Norm of the eigenfunctions:

Calculating the Modes: Solution to the wave equation: are related to eigenvalues of That is Vibrating Shapes from Impact

Calculating the Modes: Calculating the amplitudes of the vibration modes Time averaged energy of the vibration (if mass distributed uniformly) Vibrating Shapes from Impact

Initial conditions:,the k-dimensional Dirac delta function The amplitudes are: The energy is: Mode Amplitudes from Impact Location

Compute sound emitted from the vibration: Intensity of the sound emitted: It integrates the intensity of the vibration. Sound Sources from Vibrating Shapes

Each mode decays with decay time: is the internal friction parameter, which depends on material only. Having emulated the external damping of object, sound-wave: Sounds and Material Properties

Store the pre-computed frequency spectrum, and the excitation spectrum for a number of locations on the surface. Treat two sounds and as the same, if the the difference between them, where is a threshold. Sound Map

oel/app6JavaSound/demo.html Live Demo

Sound Rendering Pipeline Vibrational Analysis Instantiate Sound Propagation Resample Modeling Rendering Rigid body Simulation

Sound from Rigid body simulations Precompute shape and frequency of deformation modes –Result, quick computation of vibrational response at runtime Numerically via eigen-decomposition of system of matrices from FEM model Model sound from geometric description and material paarameters

Modal Analysis Zhimin and I were planning on ommiting the calculation of the eigenvectors from this so she can speak about it in Nikunj’s paper, as it uses the same process.

Rigid body simulation Normal rigid body simulation –One change, export collision forces to sound synthesizing process. One thing… –Sound synthesis can reveal flaws of bad rigid body simulator

Deforming the model Need to be able to generate K, C and M Mass spring, FEM etc. FEM used, because of past success, see [OSG02] for details

A Sound Generation Alg Details of the alg here

How to speed up Eval 18 use previous values e^{\omega(t+s)} = e^{\omega*t} + e^{\omega*s}

Other effects Cost of additional impulses is low (analytical solution) Convolve with Gaussian kernel –Force for soft collisions –Noise for small scale friction

Rigid body Demo