This presentation will probably involve audience discussion, which will create action items. Use PowerPoint to keep track of these action items during your presentation In Slide Show, click on the right mouse button Select “Meeting Minder” Select the “Action Items” tab Type in action items as they come up Click OK to dismiss this box This will automatically create an Action Item slide at the end of your presentation with your points entered. Improved Headphone Sound Stage Background to ENCM515 Labs on Audio Channel Modeling on ADSP21061 Based on concepts in Bessinger Thesis Various copyrights M. Smith, E. Bessinger and L.E.Turner M. Smith, Electrical and Computer Engineering, University of Calgary, Canada smithmr @ ucalgary.ca
To be tackled today Audio channel modelling concepts as detailed in Bessinger’s thesis on improved sound stage Sound re-positioning through delay lines Post Lab. 0 Quiz -- Familiarization with VisualDSP1.0++ Tool set. Implementation of FM-STEREO demodulation (“C” and assembly) Lab. 1 -- Implementation in “C” (with and without pointers) Lab. 2 -- Implementation in “assembly”, with and without pointers, with specialized SHARC architecture (hardware circular buffers) Sound colouration through FIR filters Lab. 3 -- In “C”, assembly, custom assembly (hardware loops) and VERY custom assembly (highly parallel algorithm) Additional Audio Channel Modeling Lab. 4 -- Multi-tasking environment -- SHARC RTOS -- Room colouration through IIR filters -- Student project? 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Main reference material E. C. Bessinger, Localization of Sound Using Headphones M. Sc. thesis, Electrical and Computer Engineering, January 1994. Supervisor Dr. Turner Translated for SHARC by M. Smith 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Improved Head Sound Stage Actual Sound Source Perceived Sound Source 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Concept behind Labs. Sound Source -- fixed input data array OFF-LINE initial development using VisualDSP1++ simulator Sound Source -- fixed input data array Process Sound -- KNOWN ALGORITHM Sound Sink -- fixed output data array Testing -- plotting and comparing to theory ON-LINE testing using SHARC board Sound Source -- background task (or CODEC) Process Sound -- SAME KNOWN ALGORITHM Sound Sink -- CODEC background task Testing -- using aurally using GOOD earphones 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Labs. 1 and 2 -- delay line -- Concept No relative delay modelled into the audio channel -- then sound perceived in centre of head Modelling a relative delay into the right ear audio channel. Sound arrival will shift sound to left as “sound” seems to get to left ear first DELAY_LEFT DELAY 0 DELAY_RIGHT 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Delay Necessary -- Calculation Distance to Sound Sound Time Delay = -------------------------- Speed of Sound Algorithm Delay = Time Delay * Sampling Rate Relative Distance = 0.3 m Speed of sound = 300 m/s Sampling Rate = 44 kHz Delay needed in array = ???? Sound resolution = ???? Reasonable or NOT? 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
GLOSSARY FIFO First In -- First Out stack Delay Line -- Implementation using FIFO LIFO Last In -- First Out stack C-stack -- Implementation using LIFO FIR Finite Impulse Response IIR Infinite Impulse Response DFT Discrete Fourier Transform FFT -- Fast Fourier Transform which is an implementation of DFT 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Real-time N-pt Delay code concept void ProcessSound(int channel_one, int channel_two, int *left_channel, int *right_channel) { if ((sound_source & LEFT_DELAY) == LEFT_DELAY) { if ((sound_source & MOVEDELAY) == MOVEDELAY) MemoryMove_LeftDelay(process_var1, &channel_one); ---------------------------------------------------- void MemoryMove_LeftDelay(int N, int *channel_one) { float left_delayline[MAXDELAY + 1] = {0}; // Insert new value into the back of the FIFO delay line left_delayline[0 + N] = (float) *channel_one; // N delays implimented // Grab delayed value from the front of the FIFO delay line *channel_one = (int) left_delayline[0]; // Update the FIFO delay line using inefficient mass memory moves for (count = 0; count < N; count++) <--- Or N + 1 or N - 1? left_delayline[count] = left_delayline[count - 1]; } WHAT’S WRONG AND MISSING? WHAT’S WRONG 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Basic Algorithm -- Handle Delay <---------- LEFT DELAY ----------> Sample into channel <----------DELAY ----------> Requires 2 * (LD + RD - 2) memory operations Sample from channel 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
To be tackled today Audio channel modelling concepts as detailed in Bessinger’s thesis on improved sound stage Sound re-positioning through delay lines Post Lab. 0 Quiz -- Familiarization with VisualDSP1.0++ Tool set. Implementation of FM-STEREO demodulation (“C” and assembly) Lab. 1 -- Implementation in “C” (with and without pointers) Lab. 2 -- Implementation in “assembly”, with and without pointers, with specialized SHARC architecture (hardware circular buffers) Sound colouration through FIR filters Lab. 3 -- In “C”, assembly, custom assembly (hardware loops) and VERY custom assembly (highly parallel algorithm) Additional Audio Channel Modeling Lab. 4 -- Multi-tasking environment -- SHARC RTOS -- Room colouration through IIR filters -- Student project? 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
To be tackled today Audio channel modelling concepts as detailed in Bessinger’s thesis on improved sound stage Sound re-positioning through delay lines Lab. 0 -- Familiarization with VisualDSP1.0++ Tool set. Implementation of FM-STEREO demodulation (“C” and assembly Lab. 1 -- Implementation in “C” (with and without pointers) Lab. 2 -- Implementation in “assembly”, with and without pointers, with specialized SHARC architecture (hardware circular buffers) Sound colouration through FIR filters Lab. 3 -- In “C”, assembly, custom assembly (hardware loops) and VERY custom assembly (highly parallel algorithm) Additional Audio Channel Modeling Lab. 4 -- Multi-tasking environment -- SHARC RTOS -- Room colouration through IIR filters -- Student project? 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Labs. 1 and 2 -- delay line -- Concept Get “ambience” by taking into account constructive and destructive interference around the face. This implies knowing characteristics of “audio channel” and modelling using an FIR filter -- 2 FIR filters per speaker -- processing requirement increasing FILTER 30 LEFT FILTER 330 LEFT FILTER 330 RIGHT FILTER 30 RIGHT 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Delay is not only thing to take into account Four audio channels from 2 sources in front 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Lab. 3 -- Transfer function h30(f) 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Lab. 3 -- FIR transfer -- h330(f) 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Why different Audio Channels? Delay is not only positioning element -- take into account actual direction Transfer function from loudspeaker to ear modelled by FIR filterS Enough time? 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
FIR schematic diagram 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
FIR = Complex Delay Line 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Real-time FIR Filter void ProcessSound(int channel_one, int channel_two, int *left_channel, int *right_channel{ if ((sound_source & FIRFilter) FIRFilter(&channel_one, &channel2); float fircoeffs_30[], fircoeffs[330]; void FIRFilter(int *channel_one, int *channel_two) { // Insert new value into FIFO delay line left_delayline[0 + N] = (float) *channel_one; right_delayline[0 + N] = (float) *channel_two; channel_one_30 = channel_one 330 = 0; for (count = 0, count < FIRlength - 1, count++) { channel_one_30 = channel_one_30 + arrayleft[count] * fir_30(count); channel_one_330 = channel_one_330 + arrayright[count]* fir_330[count]; } *channel_one = (int) channel_one_30 + scale_factor * channel_one_30 ditto 2 Update Left Channel delay line; Update Right Channel Delay line } 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
To be tackled today Audio channel modelling concepts as detailed in Bessinger’s thesis on improved sound stage Sound re-positioning through delay lines Lab. 0 -- Familiarization with VisualDSP1.0++ Tool set. Implementation of FM-STEREO demodulation (“C” and assembly Lab. 1 -- Implementation in “C” (with and without pointers) Lab. 2 -- Implementation in “assembly”, with and without pointers, with specialized SHARC architecture (hardware circular buffers) Sound colouration through FIR filters Lab. 3 -- In “C”, assembly, custom assembly (hardware loops) and VERY custom assembly (highly parallel algorithm) Additional Audio Channel Modeling Lab. 4 -- Multi-tasking environment -- SHARC RTOS -- Room colouration through IIR filters -- FFT Blocks -- Student project? 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Lab. 4 -- Room Effects Need to have multiple sources more FIR operations Need to have reflections off the wall -- possibilities IIR filters, Comb, FFT (frequency domain) 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Standard IIR filter example -- biquad Normally in multi-stages Make parallel BETWEEN rather than IN stages -- see Smith (Micro Magazine article, 1992) SMALL DELAYS 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Real Time standard IIR code finalleftear = h0 * lefttemp1 + h1 * lefttemp2 + h3 * lefttemp3; finalrightear = h0 * rightemp1 + h1 * righttemp2 + h3 * righttemp3; newlefttemp1 = gain * leftear + h4 * lefttemp2 + h5 * lefttemp3; newrighttemp1 = gain * rightear + h4 * rightemp2 + h5 * righttemp3; lefttemp3 = lefttemp2; // Parallel ear calculations lefttemp2 = lefttemp1; // Parallel stages lefttemp1 = newlefttemp1; righttemp3 = righttemp2; // Optimization -- final exam question? righttemp2 = righttemp1; // Optimization righttemp1 = newrighttemp1; // Need other delay and FIR filters for audio channel characteristics 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Lab. 5 -- IIR -- comb filter -- different Long delay line 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Full Lab 4 -- FIR and IIR Time available if multi-task? 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Lab. 4 requires 7 comb filters BIG DELAYS (Use PACK instructions or decimation?) 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Lab. 4 Possibilities -- FFT? Implement filters in frequency domain rather than in time domain Frequency domain algorithms can be considerably faster than time domain algorithms especially when performing several filtering operations on one set of data Require more memory to process BLOCKS OF DATA 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Real Time FIR Convolution Filter via multiplication in frequency domain FFT is faster that FIR and IIR if memory available (2N log N + 4N) rather than 3 * N * N operations for 512 points // Signal Conditioning -- Windowing wleftear[n] = leftear[n] * window[n]; FFT(wleftear, leftfrequency, forward FFT) for 512 points // Signal convolution with FIR frequency domain response new_leftfrequency1[n] = leftfrequency[n] * filter1[n] + leftfrequency[n] * filter2[n] + leftfrequency[n] * filter3[n]; FFT(new_leftfrequency, newleftear, inverse FFT) 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
DSP 4 -- FFT -- Decimation Require special architecture for speed 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Major FFT implementation Reference Major reference on signal conditional and filter implementation in frequency domain FFT -- friscy Fourier Transforms? Smith (Microprocessors and Microsystems) 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
DIF -- FFT code -- in Fortran pseudo code DO 11 I = 1, 15 -- allows FFT size up to 2^15 M = I; N2 = 2 **I; IF (N.EQ.N2) go to 21 11 CONTINUE RETURN 21 N2 = N DO 30 I = 1, M N1 = N2; N2 = N2/2; I1 = 1; I2 = N/N1; -- indexes DO 30 J = 1, N2 --- Note that N2 gets smaller C= cosarray(I1); S = sinarray(I1); I1 = I1 + I2 -- Note indexing format DO 30 K = J, N step N1 L = K + N2 tempr = xreal(K) - xreal(L); xreal(K) = xreal(K) + xreal(L) tempi = ximag(K) - ximag(L); ximag(K) = ximag(K) + ximag(L) xreal(L) = C * tempr + S * tempi; ximag(L) = C * tempi + S * tempr; 30 CONTINUE BitReverseArrayManipulation(N, xreal, ximag) 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca
Tackled today Audio channel modelling concepts as detailed in Bessinger’s thesis on improved sound stage Sound re-positioning through delay lines Lab. 0 -- Familiarization with VisualDSP1.0++ Tool set. Implementation of FM-STEREO demodulation (“C” and assembly Lab. 1 -- Implementation in “C” (with and without pointers) Lab. 2 -- Implementation in “assembly”, with and without pointers, with specialized SHARC architecture (hardware circular buffers) Sound colouration through FIR filters Lab. 3 -- In “C”, assembly, custom assembly (hardware loops) and VERY custom assembly (highly parallel algorithm) Additional Audio Channel Modeling Lab. 4 -- Multi-tasking environment -- SHARC RTOS -- Room colouration through IIR filters -- Student project? 11/22/2018 ENCM515 -- Background to Audio Channel Modelling Copyright smithmr@ucalgary.ca