1. Introduction To X3DAudio
© Allan C. Milne
Abertay University
v14.3.7

2. Agenda. Why do we need X3DAudio? What is X3DAudio? Basic concepts.
X3DAudio API.
Coding steps.

3. At Present We Can …
Use overall volume adjustment to simulate distance.
Use output matrix volume to simulate panning.
Apply filtering to represent some large-scale effect.
Use above to approximate a sound in 2-D horizontal space.

4. What Is The Problem?
Sound adjustments to determine position are arbitrary.
how do they relate to game world coordinates?
Does not reflect actual physics.
Different frequencies are attenuated differently over distance.
Does not reflect psycho-acoustics.
How the human ear/brain processes the sound.
No doppler effect for movement.
Requires fine pitch control.

5. Other Issues. Requires complex calculations.
Must be computed for every sound.
Where are we listening from?
position?
which way are we facing?
Must consider 3D (x,y,z) positioning, not just 2D.

6. X3DAudio …
An API to position sounds in 3D space in terms of the manipulations that must be performed on the audio.
This API does not actually perform the audio manipulations but rather computes the appropriate volume, frequency and filter settings;
these settings must then be explicitly applied to the XAudio2 source voice.

7. Basic Concepts. A listener: Emitters:
The 3D position, velocity and orientation of whatever is hearing the sound;
Like the camera concept in graphics.
Emitters:
3D position and velocity of the sounds;
one emitter for each sound source;
Often associated with a source voice.
X3DAudio calculates voice settings to apply given the relative position and velocity of the listener and one emitter.

8. Position and Velocity.
Listener and emitter positions are defined in 3D user world coordinates.
Each can also have a velocity:
This does not affect position;
It is used only to calculate doppler effects.
These are defined as X3DAUDIO_VECTOR structs.

9. Representations.
A listener is represented by an X3DAUDIO_LISTENER struct.
An emitter is represented by an X3DAUDIO_EMITTER struct.
These structs define the 3D position, orientation, velocity and other attributes.
The audio settings to apply are returned via an X3DAUDIO_DSP_SETTINGS struct.

10. The X3DAudio API. X3DAudioInitialize(…) X3DAudioCalculate(…)
initializes X3DAudio;
normally called only once;
initialises internal constants.
X3DAudioCalculate(…)
supply listener and emitter arguments;
supply Flags to control what computations to perform;
Computes volume and other settings to apply;
returns computed settings.

11. Use With XAudio2.
Settings returned from the calculate function are in a form that can be applied using Xaudio2 functions.
SetOutputMatrix(…)
For volume settings determining direction and distance.
SetFrequencyRatio (…)
For adjusting pitch.
SetFilterParameters (…)
For adding basic effects.
Other settings are applied via a reverb effect object.

12. So What?
Use game world (x,y,z) coordinates to position our audio objects. (Be wary of scaling when working with other coordinate systems).
Include information on orientation and velocity to create more realistic audio rendering.
Audio settings are now in the game world rather than using ad-hoc settings to simulate position.
X3DAudio provides a variety of XAudio2 settings to provide realistic 3D positioning.

13. Setting Up. Initialize X3DAudio: Set up the listener:
Get number of output channels;
Call X3DAudioInitialize(…).
Set up the listener:
define X3DAUDIO_LISTENER struct;.
Set orientation, position and velocity.
Set up emitters for all 3D sources:
Define X3DAUDIO_EMITTER structs;
Set channel, position and velocity.

14. For Each Emitter …
Create space for the DSP settings that will be returned:
define X3DAUDIO_DSP_SETTINGS
Initialize channel counts fields;
Allocate array space.
Calculate and apply DSP settings to the source voice:
call X3DAudioCalculate(…).
Call SetOutputMatrix(…);
Call setFrequencyRatio(…);
Call SetFilterParameters(…).

15. Includes. #include <Windows.h> #include <XAudio2.h>
#include <X3DAudio.h>

16. 1. Initialize X3DAudio.
DWORD channelMask;
XAUDIO2_DEVICE_DETAILS voiceDetails;
engine->GetDeviceDetails (0, &voiceDetails);
channelMask = voiceDetails.OutputFormat.dwChannelMask;
X3DAUDIO_HANDLE X3DInstance;
X3DAudioInitialize (channelMask, X3DAUDIO_SPEED_OF_SOUND, X3DInstance);
If using my framework then instantiating XACore does this initialization for you.
Then all we need do is
X3DInstance = XACore::GetInstance().Get3DHandle();
A different approach is required for the Windows 8 and XBox platforms.

17. 2. Setup the Listener.
the struct defines the 3D position and orientation.
analogous to the camera in graphics.
Can also set velocity.
X3DAUDIO_LISTENER listener = { 0 };
X3DAUDIO_VECTOR front = { 0.0f, 0.0f, 1.0f };
X3DAUDIO_VECTOR top = { 0.0f, 1.0f, 0.0f };
listener.OrientFront = front;
listener.OrientTop = top;
X3DAUDIO_VECTOR pos = { 0.0f, 0.0f, -3.0f };
listener.Position = pos;

18. 3. Setup Emitter Structs.
Define for each source voice to be rendered in 3D.
CurveDistanceScaler is set to reflect distance units;
e.g. =0.001f if units are Km.
XAUDIO2_VOICE_DETAILS details;
source->GetVoiceDetails(&details);
X3DAUDIO_EMITTER emitter = { 0 };
emitter.ChannelCount = details.InputChannels;
emitter.CurveDistanceScaler = 1.0f;
X3DAUDIO_VECTOR pos = { 10.0f, 0.0f, 30.0f };
emitter.Position = pos;
X3DAUDIO_VECTOR vel = { 0.0f, 0.0f, 0.0f };
emitter.Velocity =vel;

19. 4. Initialize DSP Settings.
Allocate space in DSP struct for fields that require arrays.
Calculated voice settings will be returned in this DSP struct.
X3DAUDIO_DSP_SETTINGS DSPSettings;
SecureZeroMemory (&DSPSettings, sizeof (X3DAUDIO_DSP_SETTINGS));
DSPSettings.SrcChannelCount = emitter.ChannelCount;
DSPSettings.DstChannelCount = XACore::GetInstance().GetChannelCount();
DSPSettings.pMatrixCoefficients = new FLOAT32 [DSPSettings.SrcChannelCount * DSPSettings.DstChannelCount];

20. 5. Apply DSP settings.
Calculate function flags define what calculations to perform.
The settings should then be applied to the source voice corresponding to the emitter struct used with the calculate function.
X3DAudioCalculate (X3DInstance, &listener, &emitter,
X3DAUDIO_CALCULATE_MATRIX | X3DAUDIO_CALCULATE_DOPPLER | X3DAUDIO_CALCULATE_LPF_DIRECT,
&DSPSettings );
source->SetOutputMatrix ( NULL, DSPSettings.SrcChannelCount, DSPSettings.DstChannelCount, DSPSettings.pMatrixCoefficients);
source->SetFrequencyRatio (DSPSettings.DopplerFactor);
XAUDIO2_FILTER_PARAMETERS FilterParameters = { LowPassFilter,
2.0f * sinf(X3DAUDIO_PI/6.0f * DSPSettings.LPFDirectCoefficient),
1.0f };
source->SetFilterParameters (&FilterParameters);

21. Encapsulation. Code examples are “raw” XAudio2.
A minimal encapsulation is exposed in the XACore framework.
On gaining experience with 3D audio programming you may wish to create richer 3D framework encapsulations.

22. XACore Encapsulation.
Initializing XACore also initializes X3DAudio and creates the 3D audio handle.
Retrieve this using Get3DHandle().
XACore exposes Apply3D(…) to calculate and apply 3D positional settings as described previously.

23. void Apply3D ( ) const IXAudio2SourceVoice* aVoice,
const X3DAUDIO_EMITTER* anEmitter,
const X3DAUDIO_LISTENER* aListener,
const unsigned int flags =X3DAUDIO_CALCULATE_MATRIX
) const

24. In Summary …
Listener defines position and orientation of what hears the sound.
Emitters define positions of where the 3D sounds come from.
for each emitter, X3DAudio calculates the
relative position of the sound; and
the voice settings to apply to simulate this.
These settings can then be applied using XAudio2.