Physically Based Sound Modeling Class #13 (Feb 24) Doug James, CMU

Overview Class #13 (Feb 24) Physically based sound modeling Important aspects of PB sound: –generation (impact, vibration,...) –emission –propagation –listening

Properties of Sound Waves Speed: Speed in air is (approx) 344 m/s. Speed in aluminum (longitudinal (sound) waves) is (approx) 5000 m/s (~3 miles/s). Longitudinal waves usually travel faster than transverse In general, speed is a function of frequency; important for solid materials.

Visualization of acoustic behavior of performance halls Adam Stettner, Donald P. Greenberg, Computer Graphics Visualization For Acoustic Simulation, Computer Graphics (Proceedings of SIGGRAPH 89). 23(3), pp , 1989.Computer Graphics Visualization For Acoustic Simulation

Tapio Takala, James Hahn, Sound Rendering, Computer Graphics (Proceedings of SIGGRAPH 92). 26(2), pp , 1992.Sound Rendering Introduced sound rendering to computer graphics Modulate sound due to material properties Do not account for shape of objects or location of collisions between objects

Modal Vibration Sound Models Simple but effective sound model for many hard objects Modal vibration model accounts for –Shape of the object (related to frequency spectrum) –Location of the impact (related to timbre of sound) –Material of the struck object (via internal friction) –Force of the impact (related to amplitude of emitted sound) Some related work... –Alex Pentland, John Williams, Good Vibrations: Modal Dynamics for Graphics and Animation, Computer Graphics (Proceedings of SIGGRAPH 89). 23(3), pp , 1989.Good Vibrations: Modal Dynamics for Graphics and Animation –K. van den Doel and D. K. Pai, The Sounds of Physical Shapes, Presence: Teleoperators and Virtual Environments, 7:4, The MIT Press, pp The Sounds of Physical Shapes –Kees van den Doel, Paul G. Kry, Dinesh K. Pai, FoleyAutomatic: Physically Based Sound Effects for Interactive Simulation and Animation, SIGGRAPH 2001, p , August FoleyAutomatic: Physically Based Sound Effects for Interactive Simulation and Animation –O'Brien, J. F., Shen, C., Gatchalian, C. M., Synthesizing Sounds from Rigid-Body Simulations, ACM SIGGRAPH 2002 Symposium on Computer Animation, San Antonio, Texas, July 21-22, pp Synthesizing Sounds from Rigid-Body Simulations

K. van den Doel and D. K. Pai, The Sounds of Physical Shapes, Presence: Teleoperators and Virtual Environments, 7:4, The MIT Press, pp The Sounds of Physical Shapes Introduced modal sound maps Details on whiteboard (1996 preprint online) Sonic Explorer First 9 eigenfunc’s of square plate

An interesting aside... “Can one hear the shape of a drum?” M. Kac. Can one hear the shape of a drum? Amer. Math. Monthly, 73, C. Gordon, D. Webb, and S. Wolpert. One cannot hear the shape of a drum. Bull. Amer. Math. Soc., 27: , 1992 Example of a planar isospectral domain:

Dinesh K. Pai, Kees van den Doel, Doug L. James, Jochen Lang, John E. Lloyd, Joshua L. Richmond, Som H. Yau, Scanning Physical Interaction Behavior of 3D Objects, Proceedings of ACM SIGGRAPH pp , Modal sound maps can be scanned from real objects Estimate {f,d,A}

Dinesh K. Pai, Kees van den Doel, Doug L. James, Jochen Lang, John E. Lloyd, Joshua L. Richmond, Som H. Yau, Scanning Physical Interaction Behavior of 3D Objects, Proceedings of ACM SIGGRAPH pp , 2001.

Kees van den Doel, Paul G. Kry, Dinesh K. Pai, FoleyAutomatic: Physically Based Sound Effects for Interactive Simulation and Animation, SIGGRAPH 2001, p , August FoleyAutomatic: Physically Based Sound Effects for Interactive Simulation and Animation FoleyAutomatic slideshow...

James F. O'Brien, Perry R. Cook, Georg Essl, Synthesizing Sounds from Physically Based Motion, SIGGRAPH 2001, p , August Synthesizing Sounds from Physically Based Motion Brute force computation of sound emission for deformable objects FEM with explicit time-stepping dt = – Captures effects of larger deformations Acoustic pressure: Sound emission: Delay based on distance:

James F. O'Brien, Perry R. Cook, Georg Essl, Synthesizing Sounds from Physically Based Motion, SIGGRAPH 2001, p , August Synthesizing Sounds from Physically Based Motion

James F. O'Brien, Perry R. Cook, Georg Essl, Synthesizing Sounds from Physically Based Motion, SIGGRAPH 2001, p , August Synthesizing Sounds from Physically Based Motion

O'Brien, J. F., Shen, C., Gatchalian, C. M., Synthesizing Sounds from Rigid-Body Simulations, ACM SIGGRAPH 2002 Symposium on Computer Animation, San Antonio, Texas, July 21-22, pp Synthesizing Sounds from Rigid-Body Simulations Use modal models in a rigid body simulator Calculated per-mode emission coefficients

Sound Propagation Various approximations of sound wave propagation Specular reflection and diffraction phenomena important For a good summary, see Funkhouser et al., “Sounds Good to Me!” Computational Sound for Graphics, Virtual Reality, and Interactive Systems SIGGRAPH 2002 Course notes.

Wave Propagation Methods Solve wave equation explicitly FEM, BEM, or... Expensive Ineffective for real-time acoustics

Geometric Sound Propagation Ray-tracing (& beam-tracing) High-frequency approximation

Thomas A. Funkhouser, Patrick Min, Ingrid Carlbom. Real-Time Acoustic Modeling for Distributed Virtual Environments Proceedings of SIGGRAPH 99. pp , Beam-tracing Video

Nicolas Tsingos, Thomas Funkhouser, Addy Ngan, Ingrid Carlbom. Modeling Acoustics in Virtual Environments Using the Uniform Theory of Diffraction Proceedings of ACM SIGGRAPH pp , Extends beam tracing Includes diffraction effects Video

The Head Related Transfer Function (HRTF) Model overall effects of head, ear, and torso on sound propagation Simulated or measured MIT’s KEMAR dummy HRTF available