1. Why do we hear what we hear? James D. Johnston Chief Scientist DTS, Inc.

2. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. First, some notes The talk I’m about to give presents ideas gathered from a variety of papers and experiments, done by many people, over a long period of time. –It is not inviolate. –It is a discussion of phenomena –The mechanism is, in most cases, unknown, once one gets beyond the basilar membrane –There will be revisions as time goes on. There will ALWAYS be revisions

3. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. The auditory system Periph ery CNS

4. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. What am I calling “peripheral” HRTF’s, including ear canal and middle ear functions Cochlear analysis Reduction of sound into partial loudnesses as a function of time

5. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. And Part of the CNS? Everything else –Reduction from partial loudness to auditory features –Reduction of auditory features to auditory objects –Storage in short-term and long-term memory

6. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. Anything more about the CNS? It’s extremely flexible –It can consciously change what it does (leaving aside for now the definition of consciousness) –Its “output” is what finally matters to us –It evolved to do an extremely, distinctly excellent job of associating information from all senses and knowledge into the final result. All the time Everywhere

7. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. What actually gets to the CNS? Whatever is detected by the auditory periphery –We will leave out extremely intense LF and VHF signals, which can be detected by other means, these are extreme conditions and should not generally be experienced by a listener. How does the auditory periphery deal with the sound waves in the atmosphere?

8. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. What does the periphery do? First the periphery adds directional information via HRTF and ITD Then, the cochlea does a time/frequency analysis The time/frequency analysis is converted into loudness via compression in each “band”, introducing –Differences between loudness and intensity –The Haas (precedence) effect The set of loudnesses across frequency is encoded into a kind of biological PPM and transmitted across the auditory nerve. (No, it’s really not that simple, but it will do for now.)

9. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. A Key Point or Two The auditory periphery analyzes all signals in a time/frequency tiling called “ERB’s” or “Barks”. Due to the mechanics of the cochlea, first arrivals have very strong, seemingly disproportionate influence on what you actually hear –But this is actually useful in the real world Signals inside an ERB mutually compress Signals outside an ERB do not mutually compress.

10. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. Then what? The short-term loudness, called partial loudness, is, roughly speaking, integrated across a short amount of time (200 milliseconds or less) –Level Roving Experiments show that when delays of over 200 milliseconds exist between two sources, the ability to discern fine differences in loudness or timbre is reduced.

11. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. What happens after this Loudness Memory? Deep inside the CNS, in a fashion that I would not even care to speculate on, it seems clear that these partial loudnesses are analyzed into both monaural and binaural auditory features: –There is a great deal of data “loss” at this juncture –This memory can last “seconds or so” –The analysis from loudnesses to features can be very strongly guided by learning, experience, and cognition

12. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. And then? These features are turned into what I refer to as “auditory objects” –These can be committed to long-term memory –There is another substantial reduction in data rate –This process can be entirely steered by attention, cognition, other stimulii, etc.

13. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. A schematic of sorts: Loudness “integration ” Mbits/second Featur e Analys is Auditor y Object Analysi s Mbit/sec Kb/sec bit/sec Cognative and other Feedback

14. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. What does this imply? 1.If you listen to something differently (for different features or objects) a)You will REMEMBER different things b)This is not an illusion 2.If you have reason to assume things may be different a)You will most likely listen differently b)Therefore, you will remember different things

15. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. So what? What this all means, in effect, is that any test of auditory stimulii that wants to distinguish only in terms of the auditory stimulii must: 1.Have a falsifiable nature (i.e. be able to distinguish between perception and an actual effect) 2.Must isolate the subject from changes in other stimulii than audio 3.Must be time-proximate 4.Must have Controls 5.Must have trained, comfortable listeners

16. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. Controls? What? NOW what are you on about? A control is a test condition that tests the test. There can be many kinds of controls: –A positive control This is a condition that a subject should be able to detect. If they don’t, you have a problem. –A negative control A vs. A is the classical negative control If your subject hears a difference, you have a problem –Anchoring elements Conditions that relate scoring of this test to results in other tests These can vary depending on need, and may not be obligatory

17. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. Do I have to have controls? YES Well, unless you don’t want to know how good your test is, of course. 

18. Information on this slide is the confidential property of DTS. Any unauthorized copying is strictly prohibited. Copyright 2009 DTS, Inc. Is that all? Not even close, but we’re talking about basics today. Now, Poppy will talk quite a bit more about a variety of issues and effects in the auditory system and CNS.