1. Localizing a speech target in a multitalker mixture
Norbert Kopčo 1, Virginia Best 2, and Simon Carlile 2
1Technical University of Košice, Košice, Slovakia
2School of Medical Sciences, University of Sydney, Sydney, Australia
May 21, 2009
ASA 09 Portland 1

2. Introduction Spatial separation of sources enhances speech perception
In complex environments (e.g., with multiple talkers), spatial perception also important for “sorting” acoustic scene into objects and focusing attention on sources of interest (Brungart et al 2001; Freyman et al 1999; Kidd et al 2005; Best et al 2007; Shinn-Cunningham 2008)
Relatively few studies actually measured localization of speech in a multitalker environment (Yost et al., 1996; Hawley et al.1999; Drullman and Bronkhorst 2000; Brungart et al. 2006)
It is well known that spatial perception improves our ability to understand speech. Even in a simple scene with one talker and one noise masker, separation of the target from the masker improves intelligibility of the target speech.
In more complex scenes, e.g., when multiple people talk at the same time, spatial perception is also important for dividing the acoustic scene into objects and for focusing attention on the objects of interest
Despite the clear importance of spatial perception in these tasks, relatively few studies …
Even though we need to localize the target source in order to focus our attention in its direction, relatively few studies looked at speech localization in multitalker settings.
And, in particular, no previous study has looked at how uncertainty about the masker locations (or, equivalently, a priori knowledge of the masker locations), influences localization of the target speech.

3. Experiment and Goals
Study horizontal localization of speech in a multitalker environment
Question 1: How does presence of maskers influence localization performance?
Evaluate the effect of maskers on biases/variability in responses.
Question 2: Is performance affected by a priori knowledge / uncertainty about distribution of masker locations?
Compare performance when masker distribution fixed vs. varied from trial to trial.
Here, we performed an experiment that measured horizontal localization of speech in a multitalker environment. Our goal was to answer the following questions:
First, we wanted to know how the presence of maskers influences target localization in terms of bias and variability in responses.
Second, we wanted to compare the effect of the maskers when the masker distribution is known vs. when it is not. To achieve this goal, we compared the effect of a given masker pattern on target localization when the masker pattern was fixed within a block of trials vs. when the trials using the same pattern were mixed with trials using other patterns.
Third, it is known that the presence of maskers can mask the target so that it is undetectable. Since we wanted to separate this effect of maskers on target detectability from their effect on target localization, we asked the listeners to only respond when they heard the target.

4. Setup and masker patterns
The study was performed in a quiet semi-reverberant room. 11 speakers were distributed in a semi circular array in front of the listener. The speakers had 10-degree separation, covering the azimuth of -50 to +50degs.
On each trial the target was presented along with 4 maskers, each masker coming from a different speaker. The maskers were distributed in one of five patterns as shown in these schematics.

5. Methods
Stimuli: Target: word “two” spoken by a female talker Maskers: 4 different monosyllabic words, spoken by 4 male talkers (all longer than target) Target-to-Masker energy ratios: 0 dB or -5 dB
Task: Subjects pointed head to perceived target location Subjects asked to indicate location only if target heard (5 catch trials with no target per block to monitor obedience)
Conditions (separate blocks): - Control: No masker - Fixed: Masker pattern fixed across block of trials - Mixed: Masker pattern randomly chosen for each trial
Analysis: - evaluate the effect of the maskers on the bias and st.dev. re. control . Plot across-subject means + standard errors
The target stimulus in this study was the word “two” spoken by a female talker. The four maskers were four different monosyllabic words randomly changing from trial to trial and spoken by 4 different male talkers.
We explored two different Target to masker energy ratios: at 0 dB localization was relatively easy, while at -5 dB it was more challenging.
The subjects’ task was to point their head to the perceived location of the target. If they did not hear the target, they were instructed to point above the speaker array. We added several catch trials to assess whether they followed these instructions.
The study was divided into blocks of three types. In control blocks no maskers were presented. In the fixed condition blocks, the masker pattern was fixed throughout the block and the subject was informed about the masker locations at the beginning of the block. In the mixed condition blocks, a different masker pattern was chosen for every trial.
The data were analyzed by computing means and standard deviations in responses to each target location for each condition, and by plotting for each of the statistics the difference between the values when the taregets were masked vs when they were not (i.e. relative to the control condition ).

6. Detection
To assess how the presence of maskers influenced target detectability, here we plot the miss rate and the false alram rate observed in the study. False alarm rate is particularly important because it shows how likely the subjects were to respond to the target even if they did not hear it.
The false alarm rate was very low at 0 dB TMR and slightly higher at -5 dB TMR. Therefore only for the effects observed at both 0 and -5 dB TMR can we be certain that they are not due to subjects’ guessing the target location even if they did not hear the target.
On the other hand, masker locaiton uncertainty had no effect on False alarms or on misses at either TMRs, suggesting that any effect of masker uncertainty indicates a change in localization performance, not a change in its detectability.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
detectability very good at 0 dB TMR,
slightly worse at -5 dB, but still only 1 out of 5 responses on catch trials
-> differences between 0 dB & -5 dB might be due to detectability
-> trends similar across TMRs not likely due to decrease in detectability
IMPORTANT detectability independent of uncertainty
Detection worse at lower TMR, similar in both uncertainty conditions

7. Bias due to Maskers Leftward   Rightward
Let’s first look at the effect of the maskers on the localization biases.
To show such biases more clearly, the following figure plots the differences in responses relative to the control condition.
Now every vertical pair of plots shows data for one masker pattern, indicated at the bottom. The top row shows the data for TMR of 0 dB, the bottom row for -5 dB. The data for pattern 1 are ploted in the left column. We see that there was a rightward bias in responses to targets originating near the maskers, and a much smaller bias towards the maskers for targets originating on the other side.
A similar compressive effect of the maskers was observed for all the patterns, strongest for pattern #4 and weakest for pattern #3. In general, the compressive bias was strong for all targets presented from the locations near the peripheral maskers and weak for other locations. And, since the effect was similar at both TMRs, it is unlikely to be due to limited target detectability.
Finally, this figure also shows small but systematic differences between the fixed and mixed data. In particular, the pattern 4 panel shows that the subjects responses were more compressed when the maskers were fixed (blue) than when they were mixed (red).
Four-way repeated measures ANOVA was performed on the data, using the factors of target location, masking pattern, TMR and uncertainty. This ANOVA found that the uncertainty factor was significant, but that it was independent of the TMR. So, to specifically address the effect of uncertainty, the following slide shows the same data, collapsed across the TMR.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Plot of all patterns
Compression strongest for pattern 4, none for patt3
Compression strong near peripheral maskers (patts 1,2,4,5), weak when maskers contralateral (1,2) or central (patt3)
Trends independent of TMR -> independent of detectability
Also, differences between F&M: small but significant; not significantly different across the 2 different TMRs  collapsed x-TMR
Compression strongest for targets near peripheral maskers

8. Masker Uncertainty and Bias
Bias due to Masker Uncertainty Left   Right
The data collapsed across the TMRs are shown in the top row. In addition, the bottom row shows the effect of uncertainty directly, by plotting the difference between the mixed-condition data and the fixed-condition data (shown in the above panel in red and blue, respectively).
The graph, and additional statistical analysis, showed that a priori knowledge of the masker caused:
Rightward bias independent of target location when the maskers were on the left (shown in patt 1)
Leftward bias independent of target location when the maskers were on the right (patt 2)
Target-location dependent bias for patts 3 and 4, consistent with:
Compression of space when the maskers were on the sides (patt4), and
Expansion of space when the maskers were in the middle (patt3)
no effect when the maskers were evenly distributed
The small but significant effects of masker uncertainty are consistent with a mechanism that assumes that whenever the maskers are fixed in a certain pattern and the subject knows the masker locations, his/her responses are biased away from the maskers. Thus, give examples…
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
strongest for for patt4 – fixed responses more compressed:
Patts 1&2, pos/neg, independent of targ location
Patts 3&4, significantly changing w/ target location
Patt 5: no effect
Common mechanism: responses biased away from maskers when pattern fixed
Again, F&M equally detectable (and effect present at both TMRs), so not due to change in detectability
When masker pattern fixed throughout a block, responses biased away from maskers

9. Response Variability
Next, we analyzed the effect of the maskers on the standard deviations in localization responses. These effects are plotted here in a format similar to the previous plots. (point to graph
The plot again shows a complex effect of target location, masking pattern, uncertainty and TMR on response variability. The effect was confirmed by a 4-way ANOVA which found a significant 4-way interaction between these factors.
Looking at the data as a function of target location, the effect of the maskers depends on whether the targets are co-located with maskers or not. For example, for the pattern 1 at -5 dB, Fixed-maskers performance is more variable for on-masker targets and less variable for off-masker targets. And, the effect is reversed when the maskers are fixed.
Therefore, we summarize the results by collapsing the data across the target locations in two ways: either we collapsed across all the target locations, or we devided the target range into two regions and looked separately at the responses to targets coming from the on-masker locations and separately at the responses from the off-masker locations.
Overall, the effects shown in this figure are fairly variable as a function of the target location.
to summarize the effect we first collapsed the data across the target locations. However, as is shown, e..g, in the bottom left graph, for certain patterns the effect of the maskers depended strongly both on masker uncertainty and on the target location. This interaction would be hidden in data collapsed across the target location. On the other hand, the interaction is visible if the data are colapsed separately across the on-masker target locations and separately across the off-masker target locations.
The following slide shows the data collapsed both ways.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Specifically, we are showing the effect of the maskers on the standard deviation as a function of the target location, separately for each pattern, TMR and uncertainty conditions
for the mean localization responses
shown, respectively, by different columns, rows, and graph color.)
across the target locations as well as the effect of uncertainty shown separately for the data collapsed across the on-masker locations and separately for the data collapsed across the off-masker locations.
Describe slide: x-y axes, TMR, F/M, pattern
Complex effect of target location, masking pattern, uncertainty and TMR (confirmed by ANOVA 4way interaction)
E.g., for patt5 increase in stds present at all locations and for both F&M,
for patt12, std at increases only at some locations,
For patt4, std increase more for mixed than fixed
If F/M relation changes from pos to neg, the change tends to depend on whether targets on-maskers or off-maskers.
To bring out the main effects, we collapsed across the on-/off-masker target locations (or across all target locations)
Complex effect of target location, masking pattern, uncertainty and TMR

10. Uncertainty and Response Variability
Masker Uncertainty Helps   Hurts
The left panel shows the increases in standard deviations caused by the maskers, collapsed across the target locations, and separately for each pattern, for the fixed and mixed data (which are color coded), and for the two different TMRs (the filled bars showing data for TMR of 0 dB, and the open bars behind each filled bar showing the corresponding data for TMR of -5 dB).
The right panel shows the effect of masker uncertainty on standard deviations by plotting the difference between the mixed- and fixed-condition standard deviations, collapsed separately across the on-masker and separately across the off-masker target locations. Again, the filled bars represent the TMR of 0 dB and the open bars the TMR of -5 dB. Since this graph shows the difference between the mixed and fixed standard deviations, positive values in th graph mean that masker uncertainty hurt performance while negative values mean that masker uncertainty resulted in improved performance.
First let’s focus on patterns 1 through 3 in which all the maskers were grouped together, either on the left, or on the right, or in the middle. The filled bars on the left show that at 0 dB TMR, there was a slight increase in standard deviations, similar across all three patterns and across the two uncertainty conditions. The righthand panel shows that the difference between the effect of mixed and fixed patterns is very small, both at the on- and off-masker target locations.
The open bars on the left show that at -5 dB TMR the overall effect of the maskers was much stronger, again fairly similar for the two uncertainty conditions. However, the open bars on the right-hand side show that the effect of masker uncertainty was very different for the on- and off-masker targets. For the off-masker targets the effect of uncertainty was as expected: the standard deviations were larger when the masker pattern varied than when it was fixed within a block. On the other hand, for the on-masker targets the effect of uncertainty was reversed: changing the masker pattern randomly from trial to trial helped performance, reducing response variability compared to when the maskers were fixed.
from the previous slide. In this panel, each bar shows data
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Left: collapsed across target locations, separately for FIX and MIX
Right: difference between MIX and FIX, collapsed separately across the on- and off-masker target locations
At 0 dB TMR (filled bars): slight increase in stds due to maskers, independent of FIX/MIX or of on-/off-masker
At -5 dB:
Overall effect stronger (left) and - very different for on- and off-masker targets (right):
For off-maskers locations, masker uncertainty hurts, i.e., knowing that there are no maskers at these locations in a given block results in better target localization
For on-masker locations: MASKER UNCERTINTY HELPS, i.e., the effect of the maskers on co-located target localization is worse when the maskers are presented from the same locations on every trial
The effect on the left could be due to decreased detectability, but not the effect on the right
Patts 1-3 (grouped maskers): Averaged x-location, no Mixed-Fixed diff. If not averaged, fixing pattern - helps for off-masker targets - hurts for on-masker targets

11. Uncertainty and Response Variability
Masker Uncertainty Helps   Hurts
Now let’s look at the masker patterns 4&5 in which the maskers were not grouped near each other.
The bars in the left-hand panel show that these masking patterns caused a larger increase in response variability than the first three patterns; and that that was the case both TMRs
However, these bars also show that masker uncertainty had the opposite effect for pattern 4 compared to pattern 5. Specifically, masker uncertainty had the expected effect of increasing standard deviations in pattern 4.
On the other hand, masker uncertainty had the unexpected effect of decreasing standard deviations in pattern 5.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
at TMR of 0 dB (show) and of -5 dB (show).
. This increase was observed at both TMRs, and it was independent of the target location, as seen by comparing the bars for pattern 4 in the left-hand panel, and by the positive difference-bars for pattern 4 in the righthand panel.
Again, this decrease was observed at both TMRs and it was independent of the target location as seen when comparing the bars for pattern 5 in the left-hand panel, and by looking at the negative difference bars for pattern 5 in the righthand panel.
When maskers are distributed (either at the extremes, or evenly), the overall effect is worse at BOTH TMRs (left) (compared to patts 1-3).
Statistical analysis performed separately on patts 4&5 shows that:
For both patterns 4 and 5, the effect of uncrtainty is independent of the target location or of TMR:
For pattern 4, uncertainty always hurts
For patt 5, uncertainty always helps
Again, not due to detectability because that’s equal for F/M
Patts 1-3 (grouped maskers): Averaged x-location, no Mixed-Fixed diff. If not averaged, fixing pattern - helps for off-masker targets - hurts for on-masker targets
Patts 4-5 (distributed maskers): Effect of uncertainty independent of loc. or TMR - patt 4: uncertainty hurts performance - patt 5: uncertainty helps performance

12. Summary
1. Mixture has complex effects on localization bias and variability - depending on masker pattern, location of target re. maskers, and TMR compression of mean localization responses near peripheral maskers - increases in standard deviations, in particular when maskers distributed
2. Trial-to-trial randomization in the distribution of speech maskers (i.e., masker location uncertainty) modulates the effect of masking:
- sometimes exaggerating it (as expected) - but sometimes reducing it (unexpected)
These modulatory effects could be due to - changes in strategy - adaptation
To summarize: we have shown that the presence of a 4-talker mixture has a complex effect on the bias and variability in localization of a speech target, and that the effect depends on the specific masker pattern, on the target location relative to the maskers, as well as on the TMR.
These effects were modulated by trial-to-trial randomization in the distribution of the speech maskers. The effect of masker uncertainty on the biases was such that the responses were more away from the maskers when the maskers were fixed than when they were mixed. On the other hand, the effect of masker uncertainty on response variability was more complex: for some target-masker configurations masker uncertainty degraded performance, as expected, while for others it improved performance, contrary to our expectations.
At least two mechanisms can partially explain this complex effect of masker uncertainty. First, it is possible that the listners changed their response strategy when the maskers were fixed, re-distributing their resources towards the off-masker locations and away from the on-masker locations. Second, the observed effects might have been caused by short-term adaptation of the auditory representation of the masked locations, which were persistently activated by the maskers in the fixed condition.
However, neither of these hypotheses can explain all the observed effects.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%s
3. Speech maskers degrade speech localization per se (after separating it from effects of detection)
Finally, our results show that the speech masker mixture degrades speech localization by directly affecting the mean and standard deviation in localization responses, not by making the target non-detectible.
The main observed effect in terms of bias was in that the response range was compressed towards the median plane when the maskers were present, and in particular for the targets near peripheral maskers.
The maskers also increased response variability, most dramatically when they were arranged in patterns in which the maskers were not near each other.
In particular, it is not clear why masker uncertainty had a consistently detrimental effect on response variability for pattern 4 and a consistently beneficial effect for pattern 5.
1. Mixture has complex effects on response bias and variability, depending on masker pattern, location of target re. maskers, and TMR. Probably due to:
- main effect on biases: compression in responses to peripheral targets near maskers
- main effect on st.devs.: increases, in particular for distr.maskers
- energetic masking of targets, primarily from nearest maskers, - masking of binaural cues, dependent on masker distribution (pattern)
2. Trial-to-trial randomization in the distribution of speech maskers (i.e., masker location uncertainty) modulates the effect of masking:
- sometimes exaggerating it (as expected) - but sometimes reducing it (unexpected)
These modulatory effects could be due to - changes in strategy (re-distribution of resources based on a priori knowledge), - adaptation (persistent activation of masker locations in Fixed condition), - gradual calibration to target characteristics when target is off-masker but not when on-masker
3. Speech maskers degrade speech localization per se (after separating it from effects of detection) - detectability very good at 0 dB TMR, worse at -5 dB, - most effects of maskers have similar trends at both TMRs (amplified at -5 dB), - detectability independent of uncertainty