1. Analyzing the Speech Signal
Julia Hirschberg
CS 6998
11/10/2018

2. Basic Acoustics What is sound?
Pressure fluctuations in the air caused by a musical instrument, a car horn, a voice
Cause eardrum to move
Auditory system translates into neural impulses
Brain interprets as sound
How does it travel?
Via sound wave of air molecules that ‘travels’ thru air
11/10/2018

3. Molecules don’t travel but pressure fluctuations do
But sound waves lose energy as they travel --it takes energy to move those molecules
And molecules also move for reasons other than e.g. the sound of my voice: noise
Ratio of speech-generated molecular motion to other motion: signal-to-noise ratio
11/10/2018

4. Types of Sound: Periodic Waves
Simple Periodic Waves (sine waves) defined by
Frequency: how often does pattern repeat per time unit
Cycle: one repetition
Period: duration of cycle
Frequency=# cycles per time unit, e.g.
Frequency in Hz=1sec/period_in_sec
Horizontal axis of waveform
Amplitude: peak deviation of pressure from normal atmospheric pressure
11/10/2018

5. Phase: timing of waveform relative to a reference point
Complex periodic waves (eg)
Cyclic but composed of two or more sine waves
Fundamental frequency (F0): rate at which largest pattern repeats (also GCD of component freqs)
Components not always easily identifiable: power spectrum graphs amplitude vs. frequency
11/10/2018

6. Fourier’s Theorem
Any complex waveform can be analyzed into a set of sine waves with their own frequencies, amplitudes, and phases
Fourier analysis produces power spectrum from complex periodic wave
Potential problems:
Assumes infinite waveform when we have only a small window for analysis
Waveform itself may be inaccurately represented
11/10/2018

7. Types of Sound: Aperiodic Waves
Waveforms with random or non-repeating patterns (eg)
Random aperiodic waveforms: white noise
Flat spectrum: equal amplitude for all frequency components
Transients: sudden bursts of pressure (clicks, pops, door slams)
Waveform shows a single impulse
Fourier analysis shows a flat spectrum
11/10/2018

8. Sample Analyses
wavesurfer
11/10/2018

9. Filters Acoustic filters block out certain frequencies of sounds
Low-pass filter blocks high frequency components of a waveform
High-pass filter blocks low frequencies
Reject band (what to block) vs. pass band (what to let through)
11/10/2018

10. Production of Speech Voiced and voiceless sounds
Vocal fold vibration produces complex periodic waveform
Cycles per sec of lowest frequency component of signal = fundamental frequency (F0)
Fourier analysis yields power spectrum with component frequencies and amplitudes
F0 is first (lowest frequency) peak
Harmonics are multiples of F0
Vocal tract filters simple voicing waveform to create complex wave
11/10/2018

11. Digital Signal Processing
Analog devices store and analyze continuous air pressure variations (speech) as a continuous signal
Digital devices (e.g. computers) first convert continuous signals into discrete signals (A-to-D conversion)
Sampling: how many time points in the signal to consider?
Quantization: how accurately do we want to measure amplitude at sampling points?
11/10/2018

12. Sampling Sampling rate: how often do we need to sample?
At least 2 samples per cycle to capture periodicity of a waveform component at a given frequency
100 Hz waveform needs 200 samples per sec
Nyquist frequency: highest-frequency component captured with a given sampling rate (half the sampling rate)
11/10/2018

13. Samping/storage tradeoff
Human hearing: 20K top frequency
But do we really need to store 40K samples per second of speech?
Telephone speech: 300-4K Hz (8K sampling)
But fricatives have energy above 4K
16-22K usually good enough
11/10/2018

14. Sampling Errors Aliasing:
Signal’s frequency higher than half the sampling rate
Solutions:
Increase the sampling rate
Filter out frequencies above half the sampling rate (anti-aliasing filter)
11/10/2018

15. Quantization
Measuring the amplitude at sampling points: what resolution to choose?
Integer representation
8, 12 or 16 bits per sample
Noise due to quantization steps avoided by higher resolution but requires more storage
Choice depends on what kind of analysis to be done
11/10/2018

16. But clipping occurs when input volume is greater than range representable in digitized waveform  transients
11/10/2018

17. Perception of Pitch
Auditory system’s perception of pitch is non-linear
Sounds at lower frequencies with same difference in absolute frequency sound more different than those at higher frequencies
Bark scale (Zwicker) models perceived difference
11/10/2018

18. Pitch-Tracking Autocorrelation techniques
Goal: Estimate F0 over time as fn of vocal fold vibration
A periodic waveform is correlated with itself
One period looks much like another (eg)
Find the period by finding the ‘lag’ (offset) between two windows on the signal for which the correlation of the windows is highest
Lag duration (T) is 1 period of waveform
Inverse is F0 (1/T)
11/10/2018

19. Halving: shortest lag calculated is too long (underestimate pitch)
Errors:
Halving: shortest lag calculated is too long (underestimate pitch)
Doubling: shortest lag too short (overestimate pitch)
11/10/2018

20. Pitch Track Headers version 1 type_code 4 frequency 12000.000000
samples
start_time
end_time
bandwidth
dimensions 1
maximum
minimum
time Sat Nov 2 15:55:
operation record: padding xxxxxxxxxxxx
11/10/2018

21. Pitch Track Data F0 Pvoicing Energy A/C Score
11/10/2018

22. RMS Amplitude
Energy closely correlated experimentally with perceived loudness
For each window, square the amplitude values of the samples, take their mean, and take the root of that mean
What size window?
Longer windows produce smoother amplitude traces but miss sudden acoustic events
11/10/2018

23. Perception of Loudness
Non-linear: Described in sones or decibels (dB)
Differences in soft sounds more salient than loud
Intensity proportional to square of amplitude so…intensity of sound with pressure x vs. reference sound with pressure r = x2/r2
bel: base 10 log of ratio
decibel: 10 bels
dB = 10log10 (x2/r2)
Absolute (20 Pa, lowest audible pressure fluctuation of 1000 Hz tone) or typical threshold level for tone at frequency
11/10/2018

24. Pressure of Common Sounds
Event Pressure Db
Absolute 20 0
Whisper
Quiet office 2K 40
Conversation 20K 60
Bus 200K 80
Subway 2M 100
Thunder 20M 120
*DAMAGE* 200M 140
11/10/2018

25. Speech Analysis Gives us Information
About variation in
Loudness
Pitch (contours, accent, phrasing, range)
Timing (rate, pauses)
Style (articulation, disfluencies)
This can be correlated with other features
Syntax, semantics, discourse context, words
11/10/2018

26. Next Week Read HLT96 (Ch. 5) Try out some TTS systems
Bring 3 discussion questions to class
Projects:
Sign up for appointment to discuss project topic
11/10/2018

27. Vocal fold vibration
[UCLA Phonetics Lab demo]
11/10/2018

28. Places of articulation
alveolar
post-alveolar/palatal
dental
velar
uvular
labial
pharyngeal
laryngeal/glottal
11/10/2018

29. More Sample Analyses
wavesurfer
11/10/2018