SPPA 4030 Speech Science Stephen M. Tasko Ph.D. CCC-SLP

Topic 1: The Speech Chain Learning Objectives Outline the general sequence of biological/physical events that occur from speech formulation to speech perception. Describe the different types of information content embedded within the speech signal. Know and describe the different branches of physics and biology used to inform basic mechanisms of speech production and perception.

The Speech Chain (Denes & Pinson, 1993)

What information is embedded in the speech signal? Phonetic information Affective information Personal information Transmittal information Diagnostic Information

Branches of science employed to understand speech communication Physics Acoustics Aerodynamics Kinematics Dynamics Biology Anatomy – Gross anatomy – Microscopic anatomy – Molecular biology – Neuroimaging Physiology – Electrophysiology

Physical Quantities Review An Independent Learning Activity Learning Objectives Distinguish between basic and derived units Distinguish between scalar and vector quantities Define a range of derived quantities with special emphasis on displacement, velocity, acceleration, force, pressure, intensity, resistance and their physical relationship

Assignment 1 See Assignments section of course website Due September 12, 2013

Topic 2: The Source-Filter Theory of Speech Production: An Introduction Learning Objectives Outline the key assumptions of the source filter theory of speech production Distinguish between the source signal, filter characteristics, and the output signal Use a range of examples to demonstrate understanding of the source filter theory Distinguish to role that different vocal tract structures play in speech sound generation and speech sound filtering

Producing Speech The vocal tract can be conceived as a set of interconnecting tubes and valves. Speech production is achieved through the systematic regulation of air pressures and flows within the vocal tract.

Source-Filter Theory of Speech Production The sounds we hear as speech is the product of a sound source that has undergone filtering by the vocal tract source and the filter may be considered to be independent of each other

Vocal tract is a tube that can vary its shape From Titze (1994)

Source Filter Theory Source (Phonation) Filter (Resonator) Speech (What We Hear) Input Spectrum Frequency Response Curve Output Spectrum

Same Source, Different Filter

Different Source, Same Filters White Noise

Different Source, Same Filters (Human) burp

Different Source, Same Filters (Human) snore

Different Source, Same Filters (Human) Lip buzz

Different Source, Same Filters (Human) ?

Different Source, Same Filters (Non-Human) sheep

Different Source, Same Filters (Non-Human) accordion

Different Source, Same Filters (Non-Human) If it quacks like a duck…

Source Filter Theory Applied: Alaryngeal Speech

Source-Filter Theory Applied: Esophageal Insufflation Test

Source-Filter Theory Applied: Tracheoesophageal (TE) Speech

Source Filter Theory Applied: The Talkbox

Source-Filter Theory Applied: The Talkbox

Topic 3: A Brief Review of Physical Acoustics Learning Objectives Outline the physical processes underlying simple harmonic motion using the mass-spring model Describe the molecular basis of sound wave propagation Define the key characteristics of sinusoidal motion including – Amplitude: instantaneous, peak, peak-to-peak, root-mean-square (RMS), the decibel scale – Frequency/period including units of measure – Phase – Wavelength Briefly describe the relation between the sine wave and uniform circular motion Outline the relationship between the frequency and wavelength of a sound wave

Spring Mass Model Mass (inertia) – Newton’s first law of motion – Opposition to acceleration/deceleration Elasticity – Opposition to displacement – Rest position – Recoil force Friction

What is sound? It may be defined as the propagation of a pressure wave in space and time. Sound must propagate through a medium

Sound-conducting media Medium is composed of molecules Molecules have “wiggle room” Molecules exhibit random motion Molecules can exert pressure AB

Model of air molecule vibration (Time 1) Rest positions Air molecules sitting side by side

Model of air molecule vibration (Time 2)

Model of air molecule vibration (Time 3)

Model of air molecule vibration (Time 4)

Model of air molecule vibration (Time 5)

Model of air molecule vibration Time Distance abcd

Wave action of molecular motion Time Distance

Amplitude waveform Position Time

Amplitude waveform Amplitude Time

Where is pressure in this model? Time Pressure measuring device at a specific location

Pressure waveform Time Sound Pressure Ambient Pressure + - 0

Measuring Sound Amplitude Frequency Phase Wavelength

Measuring Sound: Signal Amplitude Ways to measure it Instantaneous Peak Peak-to-peak Root mean square (RMS) Decibel –see later Time Sound Pressure + - 0

Measuring Sound: Signal Amplitude Root mean square (RMS)

What units do we use to measure signal amplitude? Pressure: Force/area Intensity = Power/area where power=work/time & work=Force*distance Intensity is proportionate to Pressure 2

Brief Review: The decibel scale decibel scale typically used to represent signal amplitude Many common measurement scales are absolute and linear However, the decibel scale is relative and logarithmic

Absolute vs. relative measurement Relative measures are a ratio of a measure to some reference Relative scales can be referenced to anything you want. decibel scale doesn’t measure amplitude (intensity or pressure) absolutely, but as a ratio of some reference value.

Typical reference values Intensity – watts/m 2 – Threshold for normal hearing at 1000 Hz Sound Pressure Level (SPL) – 20 micropascals

However… You can reference intensity/pressure to anything you want For example, Post therapy to pre therapy Sick people to healthy people Sound A to sound B

Linear vs. logarithmic Linear scale: 1,2,3… For example, the difference between 2 and 4 is the same as the difference between 8 and 10. We say these are additive

Linear vs. logarithmic Logarithmic scales are multiplicative Recall from high school math and hearing science 10 = 10 1 = 10 x = 10 2 = 10 x = 10 3 = 10 x 10 x = = 1/10 x 1 Logarithmic scales use the exponents for the number scale log = 1 log = 2 log =3 log = -1

Logarithmic Scale base doesn’t have to be 10 In the natural sciences, the base is often 2.7… or e

Logarithmic Scale Why use such a complicated scale? – logarithmic scale squeezes a very wide range of magnitudes into a relatively compact scale – this is roughly how our hearing works in that a logarithmic scales matches our perception of loudness change

Combining the idea of logarithmic and relative… bel= log 10 (I m / I r ) I m –measured intensity I r – reference intensity A bel is pretty big, so we tend to use decibel where deci is 1/10. So 10 decibels makes one bel dB IL = 10log 10 (I m / I r )

Intensity vs. Pressure Intensity is difficult to measure. Pressure is easy to measure – a microphone is a pressure measuring device. Intensity is proportionate to Pressure 2

Extending the formula to pressure Using some logrithmic tricks, this translates our equation for the decibel to dB SPL = (2)(10)log 10 (P m / P r ) = 20log 10 (P m / P r )

Measuring Sound: Frequency/period Period (T): duration of a single cycle Frequency (F): rate that cycle repeats itself (1/T) Time Sound Pressure Period (T)

Measuring Sound: Frequency/period Absolute measure – Cycles-per-second: Hertz (Hz) Relative measure – Octave (double or halving of frequency) – Semitones (12 semitones = 1 octave)

Phase: Uniform Circular Motion

Initiating a sound waves that differ only in phase A force is applied to molecule at frequency f and time t same force applied at frequency f at time t+a where a < the period of vibration

Spatial variation in pressure wave wavelength ( ) is the distance covering adjacent high and low pressure regions

Spatial variation in pressure wave Time

Spatial variation in pressure wave

Relation between frequency and wavelength =c/F where : wavelength F: is the frequency c: is sound speed in medium (35,000 cm/sec)

Learning Objectives Draw and describe time-domain and frequency-domain representation of sound Distinguish between simple and complex sound sounds with regard to physical characteristics and graphical representations Distinguish between periodic and aperiodic sounds with specific emphasis on terms such as fundamental frequency/period, harmonics, and overtones Distinguish between continuous and transient sounds Describe how waves sum, define Fourier's theorem and be able to describe the basics of Fourier analysis

Graphic representation of sound Time domain – Called a waveform – Amplitude plotted as a function of time Frequency domain – Called a spectrum – Amplitude spectrum amplitude vs. frequency – Phase spectrum phase vs. frequency – May be measured using a variety of “window” sizes

Same sound, different graphs Time domain Frequency domain From Hillenbrand

Classification of sounds Number of frequency components – Simple – Complex Relationship of frequency components – Periodic – Aperiodic Duration – Continuous – Transient

Simple periodic sound Simple: one frequency component Periodic: repeating pattern Completely characterized by – amplitude – period (frequency) – phase Other names: sinusoid, simple harmonic motion, pure tone

Simple periodic sound: Graphic appearance From Hillenbrand

Complex periodic sounds Complex: > one frequency component Periodic: repeating pattern Continuous Frequencies components have a special relation – Lowest frequency: fundamental frequency Symbol: f o Frequency component with longest period – Higher frequency components: harmonics integer (whole number) multiples of the f o

Complex periodic sounds: Graphic appearance Time domain: – repeating pattern of pressure change – within the cycle, things look complex Frequency domain: – spectral peaks at evenly spaced frequency intervals Auditory impression: sounds ‘musical’

Complex periodic sounds: Graphic appearance From Hillenbrand

Glottal Source Time Frequency Amplitude

Amplitude vs. Phase Spectrum Amplitude spectrum: different Phase spectrum: same From Hillenbrand

Amplitude vs. Phase Spectrum Amplitude spectrum: same Phase spectrum: different From Hillenbrand

(Complex) Aperiodic sounds Complex: > one frequency component Aperiodic: Does not repeat itself Frequency components are not systematically related May be – Continuous – Transient

Aperiodic sounds: Graphic appearance Time domain: – no repeating pattern of pressure change Frequency domain: – the spectrum is dense – No “picket fence” Auditory impression: sounds ‘noisy’

Aperiodic sounds: Graphic appearance From Hillenbrand

Analysis of complex waves Waves can be summed Complex waves are the sum of simple waves Fourier: French Mathematician: – Any complex waveform may be formed by summing sinusoids of various frequency, amplitude and phase Fourier Analysis – Provides a unique (only one) solution for a given sound signal – Is reflected in the amplitude and phase spectrum of the signal – Reveals the building blocks of complex waves, which are sinusoids

Learning Objectives Draw and differentiate the waveform and the waveform envelope Draw and differentiate the amplitude spectrum, the phase spectrum and the spectrum envelope Differential between short-term spectra and long-term average spectra.

The “envelope” of a sound wave Waveform envelope: – imaginary smooth line that follows the peak of the amplitude of a sound pressure waveform Spectrum envelope: – Imaginary smooth line drawn on top of the amplitude spectrum

Waveform envelope From Hillenbrand

Waveform envelope Time

Spectrum envelope Frequency Amplitude

Thought Question Can an aperiodic and complex periodic sound have identical spectrum envelopes?

Amplitude Spectrum: Window Size “short-term” vs. “long-term average” amplitude spectrum

“Instantaneous” Amplitude Spectra

(Long Term) Average Amplitude Spectrum

Learning Objectives Describe how the amplitude spectrum and the spectrogram are related. Identify the axis units of the spectrogram. Provide some advantages of the spectrogram over the amplitude spectrum. Distinguish between a wide band and narrow band spectrogram and outline the different information each provides. Distinguish between a harmonic and a formant on a spectrogram. Be able to draw stylized (highly simplified) spectrograms based on spectra and spectrum envelopes.

The Spectrogram

Rotate 90 degrees F A F A Building a spectrogram

Rotate it so that The amplitude is Coming out of the page F A This is really narrow because it is a slice in time F Time Building a spectrogram

Dark bands = amplitude Peaks Time Frequency Building a spectrogram

Two main types of spectrograms Narrow-band spectrograms – Akin to amplitude spectrums “lined up” – Frequency resolution is really sharp Wide-band spectrograms – Akin to spectrum envelopes “lined up” – Frequency resolution not so sharp

Highlights harmonic structure Highlights spectrum envelope Wide vs. Narrow Band Spectrograms

Learning Objectives Define an acoustic filter Draw and label a frequency response curve Draw and differentiate different types of acoustic filters Define terms such as cutoff frequency, center frequency, roll off rate, gain, and bandwidth Define and draw a basic filter system and relate that to the source-filter theory of speech production

What is an “Acoustic” Filter holds back (attenuates) certain sounds and lets other sounds through - selective.

Why might we be interested in filters? Human vocal tract acts like a frequency selective acoustic filter Human auditory system behaves as a frequency selective filter helps us understand how speech is produced and perceived.

Kinds of frequency selective filters Low-pass filters – Lets low frequencies “pass through” and attenuates high frequencies High-pass filters – Lets high frequencies “pass through” and attenuates low frequencies Band-pass filters – Lets a particular frequency range “pass through” and attenuates other frequencies

Low Pass Filters Frequency lowhigh Gain + -

High Pass Filters Frequency lowhigh Gain + -

Band Pass Filter Frequency lowhigh Gain + -

Frequency Response Curve (FRC) Frequency lowhigh Gain + - Center frequency lower cutoff frequency upper cutoff frequency passband 3 dB

Operation of a filter on a signal NOTE: Amplitude spectrum describes a sound Frequency response curve describes a filter

Learning Objectives Define resonance, free and forced vibration Describe how the pendulum and spring mass models can help explain resonance. Outline how mass and stiffness influences the resonant frequency of a mass spring system. Outline how acoustic resonators behave like acoustic filters. Calculate resonant frequencies of a uniform tube based on its physical dimensions. Describe how the wavelength of the sound determines the resonant frequency of tube.

Free vibration objects tend to vibrate at a characteristic or resonant frequency (RF)

Forced vibration A vibrating system can force a nearby system into vibration The efficiency with which this is accomplished is related to the similarity in the resonant frequency (RF) of the two systems

Forced vibration If the RF of the two systems are the same, the amplitude of forced vibration will be large If the RF of the two systems are quite different, the amplitude of forced vibration will be small or nonexistent

Resonance refers to The tendency for an object to vibrate at a particular frequency or frequencies. The ability of a vibrating system to force another system into vibration.

Back to the mass spring model Vibratory frequency of the mass spring determined by – Mass – Stiffness of the spring

Acoustic Resonance Ideas from mechanical resonance also applies to acoustic systems Acoustic chambers will transmit sound frequencies with more or less efficiency, depending upon the physical characteristics Therefore, they act as filters, passing through (and even amplifying) some frequencies and attentuating others.

Acoustic Resonance And since they act as filters, they have most of the same features of a filter, even though we might use different names. Center frequency is often termed the resonant frequency. Frequency response curve often termed the resonance curve.

Helmholtz Resonator

Actions of a Helmholtz Resonator

Stephen M. Tasko Other Acoustic Resonators: Tube Resonators Uniform tubes: Factors that influence resonance – Length. – Cross-sectional area along its length. – Whether it is closed at either or both ends.

Uniform tube, closed at one end

Uniform tube closed at one end First resonance or formant F 1 = c/4 l Where c=speed of sound (35,000 cm/sec) l = length of the tube males ~ 17.5 cm females ~ 14 cm Higher resonant/formant frequencies are odd multiples of F 1 For example, F 1 = (c/4 l )*1 F 2 = (c/4 l )*3 F 3 = (c/4 l )*5 Stephen M. Tasko

Comparing Helmholtz and tube resonators

Resonator Features Sharply tuned Broadly tuned

Resonator Features An example of the resonance characteristics of the human vocal tract Frequency Gain