1. Applied Psychoacoustics Lecture: Binaural Hearing Jonas Braasch Jens Blauert

2. left ear 3D acoustic scene right ear Coding Decoding 3D auditory scene Eardrum signal s(t): distance azimuth elevation s l (t) s r (t) 4D 2x 1D 4D ?

3. Types of Binaural Models Localization Models Detection Models Sound-Source Separation Models Pink Models Black-Box Models

4. Types of Binaural Models Localization Models Detection Models Sound-Source Separation Models Pink Models Black-Box Models

5. Tasks to solve 1What cues are available to localize a sound source? 2How can we extract those cues in a Binaural Computational Model? 3How can we calculate the position of the sound source from the extracted binaural cues ?

6. Models regarding one sound source 1. What cues are available to localize a sound source?

7. Models regarding one sound source 1. What cues are available to localize a sound source? Head-Related Coordinate System 

8. Those cues are available: Interaural cues –Interaural Time Differences (ITD‘s) –Interaural Level Differences (ILD‘s) Monaural cues –Spectral Cues

9. Those cues are available: Interaural cues –Interaural Time Differences (ITD‘s) –Interaural Level Differences (ILD‘s) Monaural cues –Spectral Cues Rayleigh‘s Duplex Theorie

10. HRIR

11. HRTF R HRTF L

14. Lateralization=intra cranial Localization =extra cranial Lateralization figure from Jens Blauert Interaural axis Sideward deviation=1-D task

15. delay lines figure from Jens Blauert attenuators How to generate ITDs and ILDs ITDsILDs

16. figure from Jens Blauert Frequenc y Lateralization blur Gaussian tones sinusoid s Lateralization Blur for ILDs Lateralization blur=Lateralization experiments Minimal audible angle=Localization experiments

17. figure from Jens Blauert ILD induced Lateralization inter aural level differences perceived sideways deviation left stronger right stronge r left right broadband noise 600-Hz sinusoid (Sayers, 1964)

18. f carrier envelope or carrier carrier envelope total signal pure tones frequency band wide Gaussian tones figure from Jens Blauert Lateralization Blur for ITDs Gaussian tones: Gaussian enveloped sinusoids of critical band width

19. Envelope vs. Carrier Signals

20. inter aural phase differences perceived sideways deviation left earlierright earlier left right figure from Jens Blauert ITD induced Lateralization

21. Localization Curves level difference (left louder) time difference (left earlier) direction of auditory event φ→ figure from Jens Blauert

22. Tasks to solve 1What cues are available to localize a sound source? 2How can we extract those cues in a Binaural Computational Model? 3How can we calculate the position of the sound source from the extracted binaural cues ?

23. How can we extract those cues in a Binaural Computational Model? Extracting ITD‘S –Jeffress Model –Cross-Correlation Models Extracting ILD‘s –Excitation-Inhibition cells

24. The Jeffress Model (1948)

25.   R L  Jeffress model (1948) Estimation of ITDs

26.   R L  Jeffress model (1948)

27.   R L  Estimation of ITDs Jeffress model (1948)

28.   R L  Estimation of ITDs Jeffress model (1948)

29.   R L  Estimation of ITDs Jeffress model (1948)

30.   R L  Estimation of ITDs Jeffress model (1948)

31.   R L    Interaural cross correlation Estimation of ITDs Jeffress model (1948)

32. Model Structure HRLP Halfwave rectification Lowpass filter

33. Cross-Correlation Models  Y (  )= 1/(t 1 -t 0 ) Y l (t)Y r (t+  )  t=t 0 t1t1 Cherry (1959)

34. Cross-Correlation Models  Y (  )= 1/(t 1 -t 0 ) Y l (t)Y r (t+  )  t=t 0 t1t1 Similarity to Jeffress‘ Coincidence Model :  k  k+1  k+2  k+3  k+4  k+5  k+6  k+7  k+8

35. Bandpass Filterbank Fletcher (1940) Patterson et al. (1995)

36. Model Structure Blauert und Cobben (1978)

37. Testsound 1 ff=500 Hz Time Frequency

38. Cross-correlation Band 7 (527 Hz)

39. Cross-correlation Band 11 (3809 Hz)

40. L R Uncertainty in High Frequencies

41. L R ? ? ? Uncertainty in High Frequencies

42. Testsound 1 ff=500 Hz modulated

43. Model Structure

44. HRLP Halfwave rectification Lowpass filter

45. Cross-correlation Band 21 (3809 Hz)

46. Estimating ILDs using EI-cells Reed and Blum (1990) Breebaart et al. (2001) E(  )=exp((10  /40  P l -10 -  /40  P r ) 2 )  ILD

47. Estimating ILDs using EI-cells Reed and Blum (1990) Breebaart et al. (2001) E(  )=exp((10  /40  P l -10 -  /40  P r ) 2 )  ILD        R L

48. EI model Band 25 (6281 Hz)

49. Tasks to solve 1What cues are available to localize a sound source? 2How can we extract those cues in a Binaural Computational Model? 3How can we calculate the position of the sound source from the extracted binaural cues ?

50. 3. How can we calculate the position of the sound source from the extracted binaural cues ?

53. Remapping

55. 30° 0° 90° Model based on EI-cells

56. left ear 3D acoustic scene right ear Coding Decoding 3D auditory scene Eardrum signal s(t): distance azimuth elevation s l (t) s r (t) 4D 2x 1D 4D ITDs ILDs monaural cues

57. Head-related Coordinate System frontal plane median plane horizontal plane backward φ =180 °  = 0 ° forward φ =0 °  = 0 ° figure from Jens Blauert

58. Head-related transfer functions Frequency [Hz]

59. Head-related transfer functions left right

60. Frequency [Hz] Head-related transfer functions Interaural time differences [ms]

61. Frequency [Hz] Head-related transfer functions Interaural level differences [dB]

62. Localization in the Median Plane Blauert 1969/70 Monaural Cues directional bands boosted bands Signal: 1/3 oct. Band noise level differences rel. judgement

63. Localization of a single sound source

64. Types of accompaning sound sources Non-coherent sound sources –independent sound sources (e.g. street noise, concurrent speakers, accompaning musical instruments) Coherent sound sources –wall reflections –electronically processed sound sources (e.g., loudspeaker arrays)

65. Part I Localization of a single sound source Part II Localization in the presence of a non- coherent sound source Part III Localization in the presence of coherent sound sources

66. target distracter 200ms 100ms Time Course

67. Methods Virtual auditory sound sources Individual HRTF 11 listeners, 10 repetitions Test sound and distracter: –noise (200 - 14 kHz) –T/D-ratio 0... - 15 dB GELP

68. Localization Results

69. Listener 6: anechoic condition Localization Results

70. Localization model

71. Lateralization shifts at 0 dB T/D-ratio listeners distracter

72. 0 dB-10 dB 60 dB Running interaural cross-correlation frequency band: 5

73. Localization Experiment target distracter target distracter

74. Arguments for the cross-correlation difference hypothesis two noise bursts are perceived as one auditory event, if their envelope is identical and they overlap in spectrum. This can be observed, even if the noise burst have different spatial positions and if they are uncorrelated. the auditory event of the target depends strongly on the exposure time of the masker before the target onset. existing models fail at very low SNRs.

75. The interaural cross-correlation difference function TT  D‘  D 200ms 100ms  T =  A -  D  D‘  D  T‘   A -  D‘ step 1: step 2:  TT TT TT TT  T‘  D‘ DD AA AA time

76. sig: 30° dis: 0° Distracter Total signal Total signal - Distracter Target

77. Lateralisation shifts Simulation using subtraction factor g:  T =  A -g(t)  D‘ with a) g(0)=0; b) g(x 0 )=1; Meunier et al. (1996)

78. Including a detection algorithm SNR=-15 dB

79. Conclusion The model is able to simulate localization and detection of broadband noise in broadband noise It allows localization at very low T/D-ratios The model explains a number of psychoacoustical phenomena (e.g. shifts of auditory events, clustering of responses) It can be extended to more than two sound sources

80. Part I Localization of a single sound source Part II Localization in the presence of a non- coherent sound source Part III Localization in the presence of coherent sound sources

81. The precedence effect (Blauert, 1983)

82. Time course

83. Methods Stimulus presentation via headphones Lead and lag pair: –Bandpass filtered noise (500 Hz cf) –100 Hz, 400 Hz or 800 Hz frequency range –Lead: 300 ms ITD, lag: —300 ms and vice versa –ISI 0, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.5 ms 6 listeners Acoustic pointer

84. Psychoacoustial results

85. delay Δt of the lag speaker lead lag Revised precedence effect curve for narrow-band signals Blauert & Braasch 2004

86. ILD analysis ISI [ms]

87. Specialized Models Combining Several Cues –Centrality and Straightness (Stern et al., 1988) –HRTF-adjustment (Gaik-Lindemann, 1990) –Neuronal Networks (z.B. Janko et al. 1996) Localizing more than one sound source –Contralateral Inhibition (Lindemann, 1986) –Bayes Classification (Nix, Hohmann, 1999) –Cross-Correlation Difference (Braasch, 2001)

88. Importance of Head movements Jonghees and van der Veer 1958

89. level differences between both loudspeaker signals Median values and variations between listeners azimuth angle φ of auditory event figure from Jens Blauert