Spatial Audio - Spatial Sphere Demo Explained Steven Wilssens Senior Program Manager Silicon Graphics and Media

Please watch the talk first. http://aka.ms/spatialaudiosessionbuild2016

What is Spatial Audio again?

What is Spatial Audio Again? Spatial audio gives the illusion that sounds are coming directly from an object in a 3-dimensional space. They are audible all around the user. When you hear something as if you would in real-life… coming from an application. That’s spatial audio.

Spatial Audio in Windows 10 @ //BUILD 2016 UWP developers can convincingly place sounds in a 3-dimensional space (sphere) by using Microsoft’s HoloLens HRTF technology. HRTF technologies work with all stereo headphones. You can now create spatial audio experiences in UWP apps and games on HoloLens, Desktop, Phone, and Xbox One.

What are HRTF’s again?

HRTF (Head-related transfer function) Humans have just two ears, but can locate sounds in three dimensions: Distance Direction (above/below) Front/Back, Left/Right This is possible because the brain, inner ear and the external ears (pinna) work together to make inferences about location.

HRTF (Head-related transfer function) We take sound localization cues from one ear (monaural cues), and compare them with cues received at both ears (difference cues or binaural cues). Left or right cues originate from differences in the time sound Up and down, front and back cues originate from spectral changes produced by the outer ear shape (pinnae). Distance cues originate from intensity and reflection differences. HRTF transfers a sound to how an ear would receive that sound from a point in space.

The AudioGraph 1.0 API

Hello Old Friend - AudioGraph 1.0 Easy Coding Mic In Speaker Out Play from File Save to File Low Latency Effects

AudioGraph 1.0 💠 🎤 🎤 📢 🔀 🔀 📄 💾 IBasicAudioEffect DeviceInputNode DeviceOutputNode SubmixNode 📄 💾 FileOutputNode FileInputNode

The AudioGraph 1.1 API

AudioGraph 1.1   💠 🎤 🎤 📢 🔀 🔀 💾 📄   AudioNodeListener AudioNodeEmitter IBasicAudioEffect 🎤 🎤 📢 🔀 🔀 DeviceOutputNode DeviceInputNode SubmixNode 💾 📄   FileOutputNode FileInputNode

Emitters and Listeners Microsoft Build 2016 7/30/2018 2:37 AM Emitters and Listeners Spatial audio source Can be input or submix node Supported # of emitters depends on multiple factors Orientation, Position, Velocity Emitter Spatial receiver of audio Only supports 1 listener/AudioGraph Orientation, Position, Velocity Listener © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

DEMO: Spatial Audio - AudioGraph {Put on headphones to experience Spatial Audio} DEMO: Spatial Audio - AudioGraph http://aka.ms/spatialspheredemocode

DEMO: Spatial Audio – AudioGraph   AudioNodeEmitter 📄 Gain: 1 📢 FileInputNode DeviceOutputNode 🔀 🔀 Gain: 0.125 💠 SubmixNode ReverbAudioEffect

HRTFs and Head Orientation Tracking HRTF-based audio rendering with headphones greatly improves when headphones can track head orientation Reduces: Front-Back confusion Sound Source Elevation confusion Head movements from 2° to 32° may be required to ensure that spatial information is conveyed with high accuracy.

DEMO: Head Tracking and AudioGraph Listener Position {Put on headphones to experience Spatial Audio} DEMO: Head Tracking and AudioGraph Listener Position Powered by: Jabra Intelligent Headset Advanced sensor pack and dynamic 3D audio - and exciting new apps platform! https://intelligentheadset.com/ http://aka.ms/spatialspheredemocode

DEMO: Spatial Audio – AudioGraph  AudioNodeListener AudioNodeEmitter 📄 Gain: 1 📢 FileInputNode DeviceOutputNode 🔀 🔀 Gain: 0.125 💠 SubmixNode ReverbAudioEffect

Summary Use AudioGraph to add spatial audio to UWP apps Emitters position sound sources in space Listeners where you hear the audio Head tracking increases the ability for users to localize sound It’s easy to enable spatial audio in your UWP apps!

Call to Action Download the Spatial Audio Sphere Sample (http://aka.ms/spatialspheredemocode) Add Spatial Audio Sound to your UWP today Re-visit Build on Channel 9. Continue your education at Microsoft Virtual Academy online.