21 st February 2013 The voices in your head & utilizing head movement in hearing-aid signal processing Alan Boyd (CeSIP/IHR) Supervisors Prof. John Soraghan (CeSIP) & Wm. Whitmer (IHR)

Internalization/externalization

Initial aims and objectives find the causes of internalization in hearing-impaired listeners study the effect of hearing-aid signal processing on internalization determine the prevalence of the perception of internalization in the hearing-impaired population improve hearing-aid signal processing to reduce internalization

Externalization I Externalized condition Internalized condition Render headphones ‘acoustically transparent’ 30° ← 3 m → 30° ← 3 m → No HRTF but still ITD

Results for 1 talker Boyd et. al, JASA (2012)

Results for 4 talkers Boyd et. al, JASA (2012)

Externalization II 2 m Listener with/without hearing-aids Continuous single male talker 30° 2 m Impulsive noise bursts

Externalization II Normal hearingHearing impaired

Internalization survey Participants 267 respondents: 70 without hearing aids; 122 unilaterally aided; 75 bilaterally aided. Prevalence Percentage of respondents who experience internalization: 15.7% without hearing aids 19.6% unilaterally-aided 35.1% bilaterally-aided

Psychoacoustic research summary Hearing-impaired have a compressed perception of externalization Results suggest that internalization occurs when auditory cues are not prominent, with no effect of the hearing aid. Survey results show an increase in the prominence of internalization with increasing number of hearing aids No scope for a signal processing solution...

f1-attentuation TOTAL FAILURE

Listeners move The position of the ears (and hearing aids) relative to a sound source can change due to head movements Hearing aids work best when static Performance of adaptive noise-reduction algorithms can be reduced by head movements or moving sources Head movements may provide useful information Changing user behaviour may require a different setting Algorithms could compensate for head movements Head movement & hearing aids

Hardware MEMS triple-axis gyroscope (ITG 3200) Provides angular velocity information Programmed via Arduino and USB serial Small enough to fit on a hearing aid Patents exist for basic use of a gyroscope in hearing aids gyroscopeArduino

Program selection based on user behaviour Directional programs are useful for listening to one sound source in noisy environments Omni-directional programs are useful for localizing sounds Gyroscope information is used to smoothly mix between programs: Directional when listener is stationary Omni-directional when listener is “searching” Version 1

Adaptive directional mic array Teutsch & Elko, 2001

Behavioural switching DIR A B OMNIGYRODIR A B A B GYRO A BB A

Version 3 SOURCE 1 DOA histogram bins Binaurally-linked mics Compensates for head-movements during direction of arrival (DOA) estimates Uses generalized cross-correlation (GCC-PHAT) SOURCE 2

STFT Cross-correlation Find correlation peak Over-sampled IFFT delay → angle of arrival Shift current histogram against rotated angle Find angle rotated since previous time-step Update histogram Gyroscope Mic 1Mic 2 Version 3 - System

Version 3 – One source Boyd et. al, Proc. ICA (2013)

Version 3 – Four sources in reverb Boyd et. al, Proc. ICA (2013)

Version 3 – One source in noise Boyd et. al, Proc. ICA (2013)

Version 4 Front/back detector GCC-PHAT measures ± 90° Measurement in front and rear hemisperes is the same Short measurement period Compares peak movement to head movement ↓ source angle + clockwise rotation = Position 1 ↑ source angle + clockwise rotation = Position 2 Cone of Confusion Position 1 Position 2

Improvements & combinations Improve gyroscope performance using accelerometers & magnetometers (Razor Attitude and Heading Reference System) Combine gyro system with Navin Chatlani’s steerable beamformer Record interaural time differences, level differences and head movements in real-world situations (ISRA 2013)

Conclusion Perception of internalization of sound occurs in a significant minority of the hearing-impaired population The hearing aid may not cause it, however it may exacerbate the perception if it already exists Head movement information can be used for; Automatic program selection based on behaviour Robust DOA estimates Longer timescale techniques

Thanks for listening Any questions?

Externalization over headphones

Head movement & hearing aids DIROMNIGYRODIRGYRO A B A B A B A B A B Standard adaptive beamformer Gyroscopically guided beamformer