(The University of Tokyo) (Hokkaido University) Real-time Rendering of Aerodynamic Sound Using Sound Textures based on Computational Fluid Dynamics Tsuyoshi Yamamoto Tomoyuki Nishita (The University of Tokyo) Yoshinori Dobashi (Hokkaido University) 北海道大学の土橋です｡ 雲および大気の散乱光を考慮した稲妻の効率的なレンダリング法について発表させていただきます。

Examples of aerodynamic sound Hokkaido University http://nis-ei.eng.hokudai.ac.jp/~doba Sound of wind Sound generated by swinging objects quickly

Overview Introduction Related Work Principle and Prediction of Aerodynamic Sound Basic Idea of Our Method Computation of Sound Texture Real-time Sound Rendering Examples Conclusions

Overview Introduction Related Work Principle and Prediction of Aerodynamic Sound Basic Idea of Our Method Computation of Sound Texture Real-time Sound Rendering Examples Conclusions

Introduction Simulation of virtual environments Sound: important element voice, contact sound, etc. Improving reality of virtual environments Use of recorded sound need to find suitable sound quality depends on environment

Introduction Physically-based sound synthesis compute waves based on physical simulation generate sound automatically according to object motion Limited to sound due to solid objects Sound due to fluid wind(aerodynamic sound), water, explosion ...

Real-time rendering of aerodynamic sound Goal and Feature Real-time rendering of aerodynamic sound source is not oscillation of solid objects creating sound textures for aerodynamic sound rendering sound in real-time according to object motion

Real-time sound rendering Goal and Feature Sound by swinging sword and club Real-time sound rendering Sound synthesis depending on shapes and motion of objects

Overview Introduction Related Work Principle and Prediction of Aerodynamic Sound Basic Idea of Our Method Computation of Sound Texture Real-time Sound Rendering Examples Conclusions

No methods for aerodynamic sound Related Work in CG Propagation of sound [Takala92] [Funkhouser99] [Tsingos01] simulate reflection/absorption due to objects to compute sound taking into account geometric relation between source and receiver [Hahn95][O’Brien01] [O’Brien02] [van den doel01] Synthesis of sound waves compute sound waves by numerical analysis of subtle oscillation of objects No methods for aerodynamic sound

Related Work in CFD Prediction of aerodynamic sound Our method… [Lele97] to reduce noise due to high-speed transportation facilities, etc. complex numerical fluid simulation not appropriate for real-time applications Our method… Makes use of methods developed in CFD Realizes real-time sound synthesis

Overview Introduction Related Work Principle and Prediction of Aerodynamic Sound Basic Idea of Our Method Computation of Sound Texture Real-time Sound Rendering Examples Conclusions

Principle and Prediction Source of aerodynamic sound cylinder　 flow　 vortices in air vortices subtle fluctuations of air pressure due to vortices Prediction method Lighthill’s basic theory in 1952 [Ligh52] numerical simulation of compressible Navier-Stokes equations → computationally expensive Curle’s model

Curle’s Model Prediction by behavior of air near object pa sound sound source field flow　 vortex　 object　 receiver q　 r center position o　 Incompressible fluid analysis

Curle’s Model Prediction by behavior of air near object

Curle’s Model Prediction by behavior of air near object normal sound source function (SSF) time amp. flow sound source field normal pressure g(t) (x component)

Curle’s Model Prediction by behavior of air near object pa g(t) sound source function (SSF) pa sound pressure receiver q sound source field flow r g(t) center position o

Curle’s Model Prediction by behavior of air near object sound source function (SSF) constraint: Size of object must be sufficiently small relative to wavelength of sound

Overview Introduction Related Work Principle and Prediction of Aerodynamic Sound Basic Idea of Our Method Computation of Sound Texture Real-time Sound Rendering Examples Conclusions

Basic Idea Use of Curle‘s model not applicable to large object subdivide object into small regions equivalent to assuming independent virtual point sound sources receiver q region 1 region 2 region n virtual sound source +

Basic Idea Computing sound texture (preprocess) Rendering aerodynamic sound (real-time)

sound texture: w(l, s, u, v) Basic Idea Computing sound texture (preprocess) fluid analysis → table of sound source func. Rendering aerodynamic sound (real-time) s : time in texture domain t : time in reality l sound texture: w(l, s, u, v) speed v time s sound source pos. l SSF table fluid analysis uniform flow direction u speed v

sound texture: w(l, s, u, v) Basic Idea Computing sound texture (preprocess) fluid analysis → table of sound source func. Rendering aerodynamic sound (real-time) sound texture: w(l, s, u, v) fluid analysis l flow direction u speed v

Basic Idea Computing sound texture (preprocess) fluid analysis → table of sound source func. Rendering aerodynamic sound (real-time) dir./speed →sound texture →values of SSF move v1 v2 vn l c1 c2 cn (sound texture) SSF values (g1, g2, …, gn) receiver pos. q

Basic Idea Computing sound texture (preprocess) fluid analysis → table of sound source func. Rendering aerodynamic sound (real-time) dir./speed →sound texture →values of SSF →Curle’s model → sound pressure SSF values (g1, g2, …, gn) sound wave Curle‘s model receiver pos. q

Basic Idea Computing sound texture (preprocess) fluid analysis → table of sound source func. Rendering aerodynamic sound (real-time) dir./speed →sound texture →values of SSF →Curle’s model → sound pressure SSF values (g1, g2, …, gn) sound wave Curle‘s model receiver pos. q

Overview Introduction Related Work Principle and Prediction of Aerodynamic Sound Basic Idea of Our Method Computation of Sound Texture Real-time Sound Rendering Examples Conclusions

Computation of Sound Texture Fluid analyses for many directions and speeds long computation time sound texture speed v time s sound source pos. l fluid analysis flow dir. u speed v

Computation of Sound Texture Properties of aerodynamic sound frequency ∝ flow speed v amplitude ∝ (flow speed v )6 Need only sound texture at base speed v0 sound texture speed v time s sound source pos. l fluid analysis flow dir. u Reduce computation time and memory requirement drastically v0

Choosing 2D or 3D Fluid Analysis Stick-like object sound source pos. speed v time v0 2D analysis cross section x y flow

Choosing 2D or 3D Fluid Analysis Stick-like object 2D analysis cross section x y flow speed v time v0 sound source pos. 1D sound tex.

Choosing 2D or 3D Fluid Analysis Stick-like object 1 2 direction time 3 point sound source 2 3 1 1D sound tex. 2D sound tex. 2D analysis Others speed v time sound source pos. v0 3D analysis point sound source flow 2D sound tex.

Choosing 2D or 3D Fluid Analysis Stick-like object 2D sound tex. 2D analysis point sound source 1D sound tex. 1 2 3 direction time 1 2 3 1 2 3 Others 2D sound tex. 3D analysis

Choosing 2D or 3D Fluid Analysis Stick-like object 2D sound tex. 2D analysis point sound source 1D sound tex. 1 2 3 direction time 3 direction time sound source pos. Others 2 1 3D sound tex. 2D sound tex. 3D analysis

Overview Introduction Related Work Principle and Prediction of Aerodynamic Sound Basic Idea of Our Method Computation of Sound Texture Real-time Sound Rendering Examples Conclusions

Real-time Sound Rendering Procedure - repeat for each time step Dt

Real-time Sound Rendering Procedure - repeat for each time step Dt 1. compute direction ci and speed vi move v1 v2 vn c1 c2 cn

Real-time Sound Rendering Procedure - repeat for each time step Dt direction (c1, c2, …, cn) speed (v1, v2, …, vn) w(l, s, u, v0) sound texture values of SSF (g1, g2, …, gn) 1. compute direction ci and speed vi 2. compute SSF gi

Real-time Sound Rendering Procedure - repeat for each time step Dt 1. compute direction ci and speed vi r1 r2 rn 2. compute SSF gi 3. compute distance ri to receiver

Real-time Sound Rendering Procedure - repeat for each time step Dt 1. compute direction ci and speed vi 2. compute SSF gi + 3. compute distance ri to receiver 4. compute sound pressure pv Curle‘s model

Real-time Sound Rendering Procedure - repeat for each time step Dt direction (c1, c2, …, cn) speed (v1, v2, …, vn) w(l, s, u, v0) sound texture values of SSF (g1, g2, …, gn) 1. compute direction ci and speed vi 2. compute SSF gi (texture for base speed) 3. compute distance ri to receiver 4. compute sound pressure pv

Computation of SSF Property vi freq. ∝ speed v amp. ∝ (speed v )6 Dt t actual speed freq. ∝ speed v amp. ∝ (speed v )6 w sound texture (base speed v0) s

Computation of SSF Property vi freq. ∝ speed v vi(k) amp. ∝ (speed v )6 vi actual speed vi(k) k Dt t different interval s w sound texture (base speed v0)

Computation of SSF Property vi freq. ∝ speed v vi(k) amp. ∝ (speed v )6 vi actual speed vi(k) k Dt t x(vi(k)/v0)6 s w vi(k)/v0xDt sound texture (base speed v0)

Computation of SSF Property Recurrence relation Periodical use vi freq. ∝ speed v amp. ∝ (speed v )6 vi actual speed Dt Recurrence relation t ï î í ì = + D - ) , ( / 6 1 v s l t k c w g overlap w s w Periodical use blending for smooth transition sound texture (base speed v0)

Overview Introduction Related Work Principle and Prediction of Aerodynamic Sound Basic Idea of Our Method Computation of Sound Texture Real-time Sound Rendering Examples Conclusions

Fluid Simulation Demo Sound texture for square prism for one direction of flow length 50cm, side length 2.0cm base speed 10 m/s 2D analysis finite difference

Real-time Sound Rendering Demo Rotating sphere wire has no effect on sound Doppler effect Cylinder thrown at receiver rotating as it approaching Doppler effect Sound by wind wind through fence draft through gap between windows

Application Character animation Bear swinging a huge club Warrior swinging two different swords (image by TAITO)

Conclusions Sound synthesis of fluid Real-time rendering of aerodynamic sound sound texture based on CFD synthesis of sound waves using Curle‘s model real-time New element to improve realistic simulation of virtual environments

Acknowledgement As for fluid analysis As for character animation Atsushi Kunimatu (TOSHIBA Corp., Japan), Tunemi Takahashi (TOSHIBA Corp., Japan), Naofumi Shibata (TOSHIBA Info. Systems Corp., Japan) As for character animation People of GARAKUTA STUDIO (TAITO Corp., Japan) People of Project BUJINGAI (TAITO Corp., Japan)