Auditory Scene Analysis (ASA)

Auditory Demonstrations Albert S. Bregman / Pierre A. Ahad “Demonstration of Auditory Scene Analysis, The perceptual Organisation of Sound” ( CD in library) For a comprehensive view of Auditory Scene Analysis: Bregman, A. S. (1990) Auditory scene analysis: the perceptual organisation of sound. Cambridge, Mass.: The MIT Press (in library) Other books on auditory perception also give descriptions of ASA (check in library)

Some terms Source – the physical entity that gives rise to the sound pressure waves e.g. a violin being played Stream – the percept of a group of successive and/or simultaneous sounds as a coherent whole appearing to come from a single source The sounds we hear at any one time usually come from a number of different sources. In most cases we can hear and identify each of the different sound sources as having its own pitch, timbre, loudness and location.

Auditory Scene Analysis The inner ear separates sound into its frequency components At some point in the auditory system these components need to be assigned to the appropriate sound source Often called “perceptual grouping”, or “auditory scene analysis” Two aspects: simultaneous grouping and sequential grouping

Auditory Scene Analysis Simultaneous grouping – the grouping together of the simultaneous frequency components that come from a single source Sequential grouping – the connecting over time of the changing frequencies that a single source produces from one moment to the next First we will consider some of the physical cues used to achieve simultaneous grouping These cues are usually studied in isolation

Simultaneous grouping Some cues: Fundamental Frequency and Spectral Regularity Onset Timing Correlated changes in Amplitude or Frequency Sound Location Important: A single cue may not be effective all the time – these cues work together for perceptual organisation of the input sound

Fundamental Frequency and Spectral Regularity Consider two musical instruments each playing a note simultaneously It is easier to hear each note and each instrument if they are playing different notes (have different fundamental frequencies) Simultaneous sounds are more likely to fuse if they have the same fundamental frequency ** Has been shown that a pair of simultaneously presented vowels are easier to identify if their fundamental frequencies differ

Fundamental Frequency and Spectral Regularity Perceptual fusion of the frequency components from a harmonic sound – harmonicity – heard as a single sound If a frequency component does not form part of the harmonic series it tends to be heard out separately – as if part of a different source Demonstration 18: Isolation of a frequency component based on mistuning 3 rd harmonic is mistuned – number of steps to hear it out - % mistuning for perceptual segregation

Onset disparities Perceptual separation on tones enhanced by onset asynchrony. A frequency component that stops or starts at a different time from the complex sound is less likely to be heard as part of it than if it is simultaneous with it Rasch, R. A. (1978). The perception of simultaneous notes such as in polyphonic music, Acustica 40, 21-33

Onset disparities We can hear each of two ‘simultaneously’ played notes easier if there is a small onset difference between them These onset asynchronies are up to 30ms – so the percept is still of the notes sounding together The auditory system can exploit these onset differences even though we are not consciously aware of them Ensemble playing – completely synchronised?

Onset disparities Demonstration 21: effect of rate of onset on segregation Cluster of four overlapping pure tones The order of the onsets is either M H L M or M L H M (Medium High Low in frequency) The rise times are made shorter with each presentation – the asynchrony of the onsets is left unchanged affects the clarity with which the order of the tones can be heard

Onset disparities Shorter rise times – easier to hear the order of the tones Generally, sounds with abrupt onsets (shorter rise time) stand out better from a background of other sounds than do slow-rising sounds Shorter rise times – aids the perceptual segregation of sounds – to tell them apart Rapid onset sounds – e.g. notes from plucked or struck instruments

Correlated Changes in Amplitude or Frequency A sound may be perceptually segregated from an unchanging background if its components are modulated in amplitude or frequency Demonstration 19: Fusion based on common frequency modulation Hear harmonic complex tone Harmonics 1, 3, 5, 6, 7 remain steady Harmonics 2, 4, and 8 rise and fall in frequency four times Hear the two sets as separate sounds

Correlated Changes in Amplitude or Frequency Demonstration 20: Fusion by common frequency modulation All harmonics are frequency modulated First all harmonics change in frequency at the same rate – one sound is heard Then the harmonics are divided: harmonics 3, 4, and 5 undergo slow changes in frequency and the first harmonic undergoes fast changes in frequency Hear two sounds

Sound Location Sounds coming from different locations in space are generally assumed to be from different sources

Gestalt Principles of Organisation Gestalt psychology founded in the early 20 th century A group of psychologists: Max Wertheimer ( ), Wolfgang Köhler ( ) and Kurt Koffka ( ) formed the Gestalt school Gestalt theory – the perceptual whole is more than the sum of its parts Put forward a set of Gestalt grouping rules that describe which elements in an image belong together to form an object – mostly described in relation to vision

Gestalt Principles of Organisation These principles can also apply to hearing / auditory perception Application of these principles generally results in a grouping of the parts of the input sound that come from the same source - segregating those that don’t Discuss each of the principles separately Important: These principles work together – to arrive at a correct interpretation of the input sound – no single rule will always work

Gestalt Principles of Organisation Similarity Good Continuation Common Fate Disjoint Allocation Closure The Figure-Ground Phenomenon and Attention

Similarity Sounds are grouped into a single perceptual stream if they are similar in pitch, timbre, loudness or subjective location Demonstration 17: Failure of crossing trajectories to cross perceptually Falling and rising sequence are interleaved – tones from the rising and falling sequence are alternated in time. How easy is it to hear out each of the four standards Grouping by timbre and frequency region