1. ANISH CHANDAK ACHANDAK@CS.UNC.EDU COMP 770 (SPRING’09) An Introduction to Sound Rendering © Copyright 2009 Anish Chandak

2. Sound Rendering: An Overview © Copyright 2009 Anish Chandak Modeling Acoustic Geometry -- surface simplification Source Modeling -- area source -- emitting characteristics -- sound signal Acoustic Material -- absorption coefficient -- scattering coefficient Rendering (Sweet Audio!) Personalized HRTFs for 3D sound Late Reverberation Digital Signal Processing Interpolation for Dynamic Scenes Scientific Visualization Propagation Diffraction Refraction Doppler Effect Attenuation Specular Reflection Scattering

3. Sound Rendering: An Overview © Copyright 2009 Anish Chandak Modeling Acoustic Geometry -- surface simplification Source Modeling -- area source -- emitting characteristics -- sound signal Acoustic Material -- absorption coefficient -- scattering coefficient Rendering (Sweet Audio!) Personalized HRTFs for 3D sound Late Reverberation Digital Signal Processing Interpolation for Dynamic Scenes Scientific Visualization Propagation Diffraction Refraction Doppler Effect Attenuation Specular Reflection Scattering

4. Applications Advanced Interfaces Multi-sensory Visualization Minority Report (2002) Multi-variate Data Visualization © Copyright 2009 Anish Chandak

5. Games VR Training Applications Game (Half-Life 2) Medical Personnel Training © Copyright 2009 Anish Chandak

6. Applications Acoustic Prototyping Symphony Hall, Boston Level Editor, Half Life © Copyright 2009 Anish Chandak

7. Acoustics vs. Graphics © Copyright 2009 Anish Chandak Low geometric details vs. High geometric details Modeling Acoustic Geometry -- surface simplification Source Modeling -- area source -- emitting characteristics -- sound signal Acoustic Material -- absorption coefficient -- scattering coefficient

8. Acoustics vs. Graphics © Copyright 2009 Anish Chandak 343 m/s vs. 300,000,000 m/s 20 to 20K Hz vs. RGB 17 m to 17 cm vs. 700 to 400 nm Propagation Specular Reflection Scattering Diffraction Refraction Doppler Effect Attenuation

9. Acoustics vs. Graphics © Copyright 2009 Anish Chandak Compute intensive DSP vs. Simple addition of colors 44.1 KHz vs. 30 Hz Psychoacoustics vs. Visual psychophysics Rendering (Sweet Audio!) Personalized HRTFs for 3D sound Late Reverberation Digital Signal Processing Interpolation for Dynamic Scenes

10. Sound Propagation in Games Strict time budget for audio simulations Games are dynamic  Moving sound sources  Moving listeners  Moving scene geometry Trade-off speed with the accuracy of the simulation Static environment effects (assigned to regions in the scene) © Copyright 2009 Anish Chandak

11. Sound Propagation Approaches © Copyright 2009 Anish Chandak Numerical Methods  Solve Helmholtz Wave Equation  Accurate  Compute intensive (fourth power of frequency)  Independent of model complexity  Methods: FEM, BEM, FDTD, DWM Geometric Methods  Ray-Approximation of Wave Equation  High-frequency approximation  Fast  Dependent on model complexity  Methods: Image Source/Beam Tracing, Frustum Tracing, Ray Tracing/Phonon Tracing

12. Beam Tracing for Sound Propagation © Copyright 2009 Anish Chandak [Funkhouser,1998] Demo Input: point sound source, point listener, scene geometry with acoustic properties Output: pressure impulse response (IR) Rendering: convolve IR with audio signal of sound source Note: audio signal is a function of pressure

13. Beam Tracing for Sound Propagation © Copyright 2009 Anish Chandak [Funkhouser,1998] Acoustic Geometry -- surface simplification Acoustic Material -- absorption coefficient -- scattering coefficient Source Modeling -- area source -- emitting characteristics -- sound signal Propagation Personalized HRTFs for 3D sound Late Reverberation Digital Signal Processing

14. Example: Input © Copyright 2009 Anish Chandak [Funkhouser,1998]

15. Step 1 (pre-processing) Spatial Subdivision Partition 3D space into convex regions (BSP Tree). Build adjacency graph. © Copyright 2009 Anish Chandak [Wikipedia, Binary space partitioning][Funkhouser,1998]

16. Example: Step 1 © Copyright 2009 Anish Chandak [Funkhouser,1998]

17. Step 2 (pre-processing) Beam Tracing © Copyright 2009 Anish Chandak Compute Beam Tree Node Information  Cell ID  Beam and its apex  Cell boundary  Parent node ID  Attenuation

18. Example: Step 2 © Copyright 2009 Anish Chandak [Funkhouser,1998]

19. Example: Step 2 © Copyright 2009 Anish Chandak [Funkhouser,1998]

20. Example: Step 2 © Copyright 2009 Anish Chandak [Funkhouser,1998]

21. Example: Step 2 © Copyright 2009 Anish Chandak [Funkhouser,1998]

22. Example: Step 2 © Copyright 2009 Anish Chandak [Funkhouser,1998]

23. Example: Step 2 © Copyright 2009 Anish Chandak [Funkhouser,1998]

24. Example: Step 2 © Copyright 2009 Anish Chandak [Funkhouser,1998]

25. Example: Step 2 © Copyright 2009 Anish Chandak [Funkhouser,1998]

26. Step 3 (interactive) Path Generation © Copyright 2009 Anish Chandak Find cell, C, containing listener (log N) For each beam in C check for listener is inside it Yes, then a path exist Attenuation, path length, and direction can be computed quickly Construct path by traversing the beam tree Compute Impulse Response (IR)

27. Example: Step 3 © Copyright 2009 Anish Chandak [Funkhouser,1998]

28. Example: Step 3 © Copyright 2009 Anish Chandak [Funkhouser,1998]

29. Example: Step 3 © Copyright 2009 Anish Chandak [Funkhouser,1998]

30. Step 4 (interactive) Auralization Convolve IR with input sound signal Use the directional paths to simulate 3D audio using HRTFs © Copyright 2009 Anish Chandak Impulse Response (IR) Sound Signal* = Output Audio *=

31. Ray Tracing for Sound Propagation © Copyright 2009 Anish Chandak [Krokstad,1968] [Kulowski,1984] Input: spherical sound source, spherical listener, scene geometry with acoustic properties Output: energy impulse response (IR) Rendering:  convert energy IR into pressure IR  convolve IR with audio signal of sound source Note: audio signal is a function of pressure

32. Shoot Sound Rays (Step 1) © Copyright 2009 Anish Chandak Scene Geometry S Sound Source Listener L Shoot Rays From Source

33. Trace Sound Rays (Step 2) © Copyright 2009 Anish Chandak S L

34. Specular Reflection (Step 3) © Copyright 2009 Anish Chandak S L Based on Reflection Coefficient Annihilate Or Energy Based

35. Diffuse Reflection (Step 3) © Copyright 2009 Anish Chandak S L Based on Scattering Coefficient Annihilate or Choose a random direction

36. Construct Energy Histogram (Step 3) © Copyright 2009 Anish Chandak S L Collect Rays at the Listener

37. Construct Pressure IR from Energy Histogram (Step 4) © Copyright 2009 Anish Chandak To compute sound signal at a point add sound pressure of all contributions Phase angles of p n and p m are different and for quite a large number of components [Kuttruff,2007]

38. Auralization (Step 4) Convolve IR with input sound signal Use the directional paths to simulate 3D audio using HRTFs © Copyright 2009 Anish Chandak Impulse Response (IR) Sound Signal* = Output Audio *=

39. Advanced Topic: Acoustic Rendering Equation © Copyright 2009 Anish Chandak Equivalent to rendering equation in computer graphics [Kajiya, 1986] Time dependent equation Typically solved in frequency space Very recent development [ Siltanen, 2007] Lot of potential to apply graphics techniques of rendering to acoustic rendering equation

40. Advanced Topic: Acoustic Surface Simplification © Copyright 2009 Anish Chandak Visual Geometry Acoustic Geometry [Vorländer,2007]

41. Advanced Topic: HRTFs for 3D sound © Copyright 2009 Anish Chandak Inter-aural Level Difference (ILD) Inter-aural Time Difference (ITD) HRTF = Head Related Transfer Function. Encodes ILD, ITD, and much more.

42. COMP 770 Course Project Suggestions © Copyright 2009 Anish Chandak Sound + Visual (Parameterized Sound)  Integrating Sounds and Motions in Virtual Environments (http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=289953&site=ehost- live)http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=289953&site=ehost- live  Presence: Teleoperators & Virtual Environments (Journal) (http://search.ebscohost.com/login.aspx?direct=true&db=aph&jid=VTE&site=ehost-live)http://search.ebscohost.com/login.aspx?direct=true&db=aph&jid=VTE&site=ehost-live Acoustic Radiosity Simulation  A modified radiosity algorithm for integrated visual and auditory rendering (doi:10.1016/0097-8493(93)90112-M)doi:10.1016/0097-8493(93)90112-M  Samuel Siltanen, Tapio Lokki, and Lauri Savioja, “ACOUSTIC RADIANCE TRANSFER METHOD,” in The 19th International Congress on Acoustics (ICA) (Madrid, 2007), http://www.sea-acustica.es/WEB_ICA_07/fchrs/papers/rba-05-008.pdf. http://www.sea-acustica.es/WEB_ICA_07/fchrs/papers/rba-05-008.pdf Building Evacuation Using Sound Cues  Virtual Acoustic Technology: Its Role in the Development of an Auditory Navigation Beacon for Building Evacuation  Building Acoustics (Journal) (http://www.ingentaconnect.com/content/mscp/bac)http://www.ingentaconnect.com/content/mscp/bac

43. Reading List © Copyright 2009 Anish Chandak 1. Rudolf Rabenstein, Oliver Schips, and Er Stenger, “Acoustic rendering of buildings,” in In 5th International Conference on Building Simulation, 1997, 8—10 2. Funkhouser, T., Carlbom, I., Elko, G., Pingali, G., Sondhi, M., and West, J. 1998. A beam tracing approach to acoustic modeling for interactive virtual environments. In Proceedings of the 25th Annual Conference on Computer Graphics and interactive Techniques SIGGRAPH '98. ACM, New York, NY, 21-32. 3. Peter Svensson, "The Early History of Ray Tracing in Room Acoustics". 4. Funkhouser, Thomas and Tsingos, Nicolas and Jot, Jean-Marc, "Survey of Methods for Modeling Sound Propagation in Interactive Virtual Environment Systems," Presence and Teleoperation, 2003.

44. Additional References © Copyright 2009 Anish Chandak Michael Vorländer, Auralization: Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality, 2007. Samuel Siltanen et al., “The room acoustic rendering equation,” The Journal of the Acoustical Society of America 122, no. 3 (2007): 1624-1635, doi:10.1121/1.2766781. Kajiya, J. T. 1986. The rendering equation. In Proceedings of the 13th Annual Conference on Computer Graphics and interactive Techniques D. C. Evans and R. J. Athay, Eds. SIGGRAPH '86. U. Krockstadt. Calculating the acoustical room response by the use of a ray tracing technique. Journal of Sound and Vibrations, 8(18):118-125, 1968. U. Kulowski. Algorithmic representation of the ray tracing technique. Applied Acoustics, 18:449-469, 1984. Heinrich Kuttruff, Acoustics, 2007.

45. Questions? © Copyright 2009 Anish Chandak

46. Modeling Sound Material [Embrechts,2001] [Christensen,2005] [Tsingos,2007] © Copyright 2009 Anish Chandak

47. Sound Source Modeling Volumetric Sound Source Complex Vibration Source Directional Sound Source © Copyright 2009 Anish Chandak