JASS J ava A udio S ynthesis S ystem SUMMARY JASS is a unit generator based audio synthesis programming environment written in pure Java. JASS is freely available with full source code. The environment is based on a foundation structure consisting of a small number of Java interfaces and abstract classes, and a potentially unlimited number of unit generators, which are created by extending the abstract classes and implementing a single method. Filter-graphs, sometimes called “patches”, are created by linking together unit generators in arbitrary complex graph structures. Patches can be rendered in real-time with special unit generators that communicate with the audio hardware, which we have implemented using the JavaSound API. We used JASS to develop algorithms for real-time synthesis of realistic sound effects for interactive simulations (e.g., games) and animation. These sound effects are produced automatically, from 3D models using dynamic simulation and user interaction. The algorithms are efficient, physically-based, and can be controlled by users in natural ways. Audio is simulated using modal vibration models (determined by material and shape) driven by contact forces (determined by surface roughness). The contact forces can be custom designed or can be computed from simulations. CONTACT FORCES From the local contact physics we determine contact speed and velocity. With a surface roughness model this gives us the audio-force, which drives surface vibrations. Kees van den Doel and Dinesh K. Pai Department of Computer Science, University of British Columbia MODAL MODEL Vibrations of solids are modeled as modal resonator banks. Parameters are computed, measured, or set manually. Mode shapes determine timbral variations. SOUND SYNTHESIS Audio is rendered with JASS synthesis toolkit in real-time from modal vibrations and sent to audio hardware. The user interacts with the simulation by looking at the display and listening to the sounds and also provides input through mouse, joystick, or other more immersive means. Unit Generator Templates Simulation of Models DYNAMICS SIMULATOR The simulator computes the parameters needed to drive the audio and the graphics. They include the contact forces for sustained contact, impulsive forces for collisions, the relative surface velocities, and the velocities of the contact points on the surfaces. The contact parameters are provided to JASS at the rate of simulation for micro-simulation of contact interactions, while the system state is interpolated at frame rate for rendering graphics. SourceOut SinkIn InOut UG2 UG1 UG3 ClassInterface extends Legend: InOut UG Template Unit Generators are subclasses of templates Out, In, and InOut, which implement inter- faces Source or Sink. The InOut Unit Generator produces audio buffers and receives audio buffers from external Sources. Getbuffer() retrieves an audio-buffer and updates attached Sources. v F p Contact Force Model UG Resonance Model UG Source Player UG JASS Graphic Display Simulator v, F, roughnessp, damping, loadModel(file) start() positions User display user input time buf 1750 | | | | | | | | | | | getBuffer(t) computeBuffer() Sink Out Source1 buf1 Source2 buf2 Source3 buf3 Source source1 source2 source3