The Speech Chain (Denes & Pinson, 1993) Tasko SPPA 6010 Advanced Speech Science

What information is embedded within the speech acoustic signal? Phonetic information Affective information Personal information Transmittal information Diagnostic Information Tasko SPPA 6010 Advanced Speech Science

Branches of science employed to understand speech communication Physics Acoustics Aerodynamics Kinematics Dynamics Biology Anatomy Gross anatomy Microscopic anatomy Molecular biology Neuroimaging Physiology Electrophysiology Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Physical Quantities Basic vs. Derived Scalar vs. Vector Area Volume Displacement Velocity Acceleration Force Pressure Work Power Intensity Resistance Ohm’s Law (V=IR) Tasko SPPA 6010 Advanced Speech Science

Speech anatomy as “tubes” and “valves” Speech production is achieved through the systematic regulation of air pressures and flows within the lungs and vocal tract. Tasko SPPA 6010 Advanced Speech Science

Source-Filter Theory of Speech Production The sounds we hear as speech is the product of a sound source that has undergone filtering by the vocal tract source and the filter may be considered to be independent of each other Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Source-Filter Theory Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Source-Filter Theory Tasko SPPA 6010 Advanced Speech Science

Sound: Acoustics review What is sound? Graphic representation of sound Classifying sounds Filters Resonance The decibel Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science What is sound? It may be defined as the propagation of a pressure wave in space and time. propagates through a medium We are going to need to talk more about the medium through which sound propagates. This will seem a little bit like a digression, but it should vastly improve your grasp of the material that we go into later. So we need to talk some physics and physical quantities. Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science What is sound? Mass-spring model Tasko SPPA 6010 Advanced Speech Science

Wave action of molecular motion Time 1 2 3 4 5 Distance Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Amplitude waveform Position Time Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Amplitude waveform Question: How long will this last? Amplitude Time Tasko SPPA 6010 Advanced Speech Science

Model of air molecule vibration Time 1 2 3 4 5 Distance a b c d Tasko SPPA 6010 Advanced Speech Science

Simple Harmonic Motion: Sine Wave Features Amplitude Period Frequency Hz octave Phase Pressure Time Tasko SPPA 6010 Advanced Speech Science

Graphic representation of sound Time domain Called a waveform Amplitude v. time Frequency domain Called a spectrum Amplitude spectrum amplitude vs. frequency Phase spectrum phase vs. frequency May be measured using a variety of “window” sizes Spectrogram frequency v. amplitude v. time Tasko SPPA 6010 Advanced Speech Science

Same sound, different graphs Time domain Frequency domain Tasko SPPA 6010 Advanced Speech Science From Hillenbrand

Are all sound waves simply sinusoids? NO! Waves can be summed Simple waves can combine to produce complex waves Fourier: French Mathematician: Any complex waveform may be formed by summing sinusoids of various frequency, amplitude and phase Fourier Analysis Provides a unique (only one) solution for a given sound signal Is reflected in the amplitude and phase spectrum of the signal Reveals the building blocks of complex waves, which are sinusoids Tasko SPPA 6010 Advanced Speech Science

Classification of sounds Number of frequency components Simple Complex Relationship of frequency components Periodic Aperiodic Duration Continuous Transient Tasko SPPA 6010 Advanced Speech Science

Complex periodic sounds: Graphic appearance Tasko SPPA 6010 Advanced Speech Science From Hillenbrand

Complex periodic sounds: Graphic appearance Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Brief Digression Tasko SPPA 6010 Advanced Speech Science

Amplitude vs. Phase Spectrum Amplitude spectrum: different Phase spectrum: same Tasko SPPA 6010 Advanced Speech Science

Amplitude vs. Phase Spectrum Amplitude spectrum: same Phase spectrum: different Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Digression concluded Tasko SPPA 6010 Advanced Speech Science

Aperiodic sounds: Graphic appearance Tasko SPPA 6010 Advanced Speech Science From Hillenbrand

What “class” of sound is speech? Tasko SPPA 6010 Advanced Speech Science

The “envelope” of a sound wave Amplitude envelope Spectrum envelope Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Amplitude envelope Tasko SPPA 6010 Advanced Speech Science From Hillenbrand

Tasko SPPA 6010 Advanced Speech Science Spectrum envelope Tasko SPPA 6010 Advanced Speech Science From Hillenbrand

Amplitude Spectrum: Window Size “instantaneous” amplitude spectrum (long term) average amplitude spectrum Tasko SPPA 6010 Advanced Speech Science

“Instantaneous” Amplitude Spectra Tasko SPPA 6010 Advanced Speech Science

(Long Term) Average Amplitude Spectrum Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science The Spectrogram Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science F A Rotate 90 degrees F A Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science F F Rotate it so that The amplitude is Coming out of the page Time A This is really narrow because it is a slice in time Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Dark bands = amplitude Peaks F Time Tasko SPPA 6010 Advanced Speech Science

Two main types of spectrograms Wide-band spectrograms Akin to spectrum envelopes “lined up” Frequency resolution not so sharp Narrow-band spectrograms Akin to amplitude spectrums “lined up” Frequency resolution is really sharp Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Highlights harmonic structure Highlights spectrum envelope Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Filters What is a filter? How are they relevant to speech? Frequency response curve Representing filter operation Types of filters Tasko SPPA 6010 Advanced Speech Science

Frequency Response Curve (FRC) low high Gain + - Center frequency lower cutoff frequency upper cutoff passband 3 dB Tasko SPPA 6010 Advanced Speech Science

Operation of a filter on a signal NOTE: Amplitude spectrum describes a sound Frequency response curve describes a filter Tasko SPPA 6010 Advanced Speech Science

Source-Filter Theory revisited Tasko SPPA 6010 Advanced Speech Science

Some frequency selective filters Low-pass filters High-pass filters Band-pass filters Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Resonance What is resonance? Free vibration Forced vibration Acoustic resonators Resonance and speech Resonators as frequency selective filters Tasko SPPA 6010 Advanced Speech Science

Tasko SPPA 6010 Advanced Speech Science Resonance and Speech Tasko SPPA 6010 Advanced Speech Science

Resonators as frequency selective filters Tasko SPPA 6010 Advanced Speech Science

Measuring signal amplitude Amplitude vs. loudness Sound intensity vs. sound pressure Decibel scale Linear vs. logarithmic Absolute vs. relative Reference values Deriving the equations Tasko SPPA 6010 Advanced Speech Science