1. Extracting the pinna spectral notches Vikas C. Raykar | Ramani Duraiswami University of Maryland, CollegePark B. Yegnanaryana Indian Institute of Technology, Chennai, India

2. Plan of the talk  Human Spatial Hearing.  Role of the pinna.  Extracting the pinna spectral notches.  A few exploratory studies.

3.  Primary cues  Interaural Time Difference (ITD)  Interaural Level Difference (ILD)  Can explain only localization in the horizontal plane.  All points in the one half of the hyperboloid of revolution have the same ITD and IID.  [cone of confusion ]  Other cues  Pinna shape gives elevation cues for higher frequencies.  Torso and Head give elevation cues for lower frequencies. Human spatial hearing Intricate system to be completely modelled Source HEA D Left earRight ear

4. Head Related Transfer Function (HRTF)  The spectral filtering caused by the head, torso and the pinna can be described by the HRTF or HRIR.  Can experimentally measure HRTF for all elevation and azimuth for both ears for different persons. Convolve the source signal with the measured HRIR to create virtual audio

5. CIPIC Database  Public Domain HRIR Database  HRIRs sampled at 1250 points around the head  45 subjects including the KEMAR  Anthropometry measurements V. Ralph Algazi, Richard O. Duda, Dennis M. Thompson, Carlos Avendano,"The CIPIC HRTF database, "in WASSAP '01 (2001 IEEE ASSP Workshop on Applications of Signal Processing to Audio and Acoustics, Mohonk Mountain House, New Paltz, NY, Oct. 2001)."The CIPIC HRTF database, "

6. Interaural polar Coordinate system Azimuth Elevation

7. Sample HRTF

8. HRTF Visualization

9. Motivation for our work

11. HRTF Composition Direct Pulse Torso Reflection Knee Reflection Pinna Spectral notches Pinna Resonances Head Diffraction

12. HRTF Composition Good parametric models exist for the head and the torso effects. Role of pinna yet to be properly understood.

13. HRTFs for different pinnae

14. Features due to pinna  Pinna contributes to the sharp spectral notches and peaks.  Pinna spectral notches are important for elevation perception.  Substantial psychoacoustical, behavioral, and neurophysiological evidence.  We propose a method to automatically extract the frequencies of the pinna spectral notches.

15. Extracting the spectral notches  Direct application of Pole-Zero modelling techniques do not work.  All pole-model is very good at picking the dominant spectral peaks. Can use Linear Prediction analysis.  However Pole-Zero models are highly sensitive to noise.  The measured HRIR includes the combined effects of the head diffraction and shoulder, torso, and the knee reflection. Due to the combined effects it is difficult to isolate the notches due to pinna alone.  The order of the model is also not known.  We are not guaranteed that perceptually relevant nulls will be captured.  They do a good fit for the envelope of the signal (provided the order is high), but perceptually relevant features may not be captured.

16. Pole-Zero Modelling

17. Determine the initial onset  Slope of the unwrapped phase spectrum gives the initial delay.

18. Eliminate torso/knee reflections  Window the signal using a Hann window. Torso reflection Knee reflection

19. Effect of Delay

20. Effect of windowing Group delay function

21. In order to emphasize the spectral nulls we compute the group delay function. It is the negative first derivative of the phase response of the filter. The notch frequencies can be extracted by finding the extrema in the group delay function. The group delay function

22. Remove the spectral peaks The poles can be extracted by doing a Linear Prediction(LP)Analysis. LP analysis basically fits an all pole model of order p to the HRIR. From the given HRIR signal we can compute the LP residual by passing it through the inverse filter.

23. Window the LP residual LP residual Original signal

24. Autocorrelation of the windowed LP residual LP residual Autocorrelation

25. The complete algorithm

26. A particular subject

27. Different subjects

29.  Psychoacoustical experiments have shown that high frequencies are essential for vertical localization. [Gardner and Gardner, Hebrank and Wright, Musicant and Butler ]  When pinna cavities are occluded with plastic moulds localization ability decreases. [Gardner and Gardner, Hoffman et.al.]  There is a consensus that elevation perception is monaural. [Wightman and Kistler, Middlebrooks and Green] Pinna is essential for elevation perception

30.  Are the spectral notches perceptually significant ?  How are these spectral notches caused?  What can we do once we have these spectral notches?  Interpolation  Customization  Anthropometric studies Three questions

31.  Spectral peaks and notches are the dominant cues contributed by the pinna.  As elevation increases the notch frequency increases.  The spectral peaks do not show a definite trend with elevation.  Spectral notches can be detected and discriminated. [ Wright et.al and Moore et.al]  Experiments on cats suggest that single auditory nerve fibers are able to signal in their discharge rates the presence of a spectral notch. [Poon and Brugge]  Interestingly a moving notch is better, [implications for head movement]  Vertical illusion in cats [Tollin and Yin] The case for notches ?

32. Vertical Illusion in cats Tollin DJ and Yin TCT (2003). Spectral cues explain illusory elevation effects with stereo sounds in cats, J.Neurophysiol., 90:525-530.Spectral cues explain illusory elevation effects with stereo sounds in cats,

33. HRTFs generally measured for a finite sampling grid of elevation and azimuth. For implementing a virtual audio system we need to interpolate HRTFs. HRTF measurement is a tedious and time consuming process. Normally takes 2 to 4 hours. Subject must be immoblile. Problem 1: Interpolation

34. Time Domain Frequency Domain Principal Components domain These may not be the right way to do interpolation since they do not consider the perceptual aspect of human hearing. Ideally interpolation should be done in perceptually important feature space. [ Those features in the HRIR important for source localization ] Current approaches for interpolation

35. Interpolation in the frequency domain

36. Experimentally shown that with HRTF measured for a particular person if used for different persons elevation perception is very poor. Ear shape of each person is unique and also the anatomy. Each person’s localizing capabilites are tuned to the shape of their ear and anatomy. Problem 2: HRTF Customization

37. Complete Measurement. Database Matching. Numerical Modelling. Frequency Scaling. Current approaches for Customization

38. Pinna shape and notches Correlate the pinna spectral notches with elevation, azimuth and anthropometric measurements

39. Empirical approaches

40. The cause for notches ? Batteau’s reflection model Hebrank and wright- reflection from the posterior concha wall Lopez-Poveda and Meddis incorporated diffraction into the model. Shaws model for resonances

41. Reflection Model

43. Shaw’s modes

44. Extracted Pinna Resonances

45. Compensating for the pinna angles

46. Coordinate system alignment

47. Reduction in ISD

48.  Proposed a method to automatically extract the frequencies of the pinna spectral notches.  Pinna spectral notches are perceptually significant for source localization. So instead of convolving with the complete HRIR we can build simplified models based on the features extracted.  Interpolation can be done in the perceptually important feature domain.  The pinna spectral noches can be related to the shape and the size of the pinna and customization of the HRIR is possible. Scope of our work

49. How sensitive are the notch frequencies to the probe position? Given an image of the ear, can we hope to get the HRTF? What happens behind the ears? Are multiple spectral notches redundant? What is the role of crus helias? How are the spectral notches from the right and left ear integrated? Is analysing in terms of notches the right approach? Could there be a template matching approach. Probably open questions

50. Thank You ! | Questions ?