Spectral Analysis March 3, 2016

Mini-Rant I have succeeded in grading your course project reports. Things to keep in mind: A table of stop phonemes is a theory about how your language works. Your job is to back up that theory with evidence. Minimal pairs are not that important, but examples are. You are a better linguist than your consultant is. You cannot expect your consultant to know what a “minimal pair” is, or to be able to produce them on demand for you. Your consultant may very well say things about their language that are completely uninformed and wrong.

Mini-Rant, part II Do not tell me about the writing system for your language. Your job is not to find out how the language is written, but how it is spoken. Be conservative with your conclusions. Only tell me what can be justified with evidence from the data you’ve collected. The process is more important than the product. I would rather that you get comfortable with not knowing what the “right” answer is than for you to tell me what somebody else thinks the right answer is.

Moving On… Don’t forget: the next course project report is due on Tuesday! On Tuesday, we will also start adding mystery spectrograms into the mix. …and I will return your mid-terms to you. Today’s goal: let’s learn where spectrograms come from.

The Source The complex wave emitted from the glottis during voicing= The source of all voiced speech sounds. In speech (particularly in vowels), humans can shape this spectrum to make distinctive sounds. Some harmonics may be emphasized... Others may be diminished (damped) Different spectral shapes may be formed by particular articulatory configurations....but the process of spectral shaping requires the raw stuff of the source to work with.

Spectral Shaping Examples Certain spectral shapes seem to have particular vowel qualities.

Spectrograms A spectrogram represents: Time on the x-axis Frequency on the y-axis Intensity on the z-axis

Real Vowels

Ch-ch-ch-ch-changes Check out some spectrograms of sinewaves which change frequency over time:

The Whole Thing What happens when we put all three together? This is an example of sinewave speech.

The Real Thing Spectral change over time is the defining characteristic of speech sounds.  It is crucial to understand spectrographic representations for the acoustic analysis of speech.

Life’s Persistent Questions How do we get from here: To here? Answer: Fourier Analysis

Fourier’s Theorem Joseph Fourier ( ) French mathematician Studied heat and periodic motion His idea: any complex periodic wave can be constructed out of a combination of different sinewaves. The sinusoidal (sinewave) components of a complex periodic wave = harmonics

Fourier Analysis Building up a complex wave from sinewave components is straightforward… Breaking down a complex wave into its spectral shape is a little more complicated. In our particular case, we will look at: Discrete Fourier Transform (DFT) Also: Fast Fourier Transform (FFT) is used often in speech analysis Basically a more efficient, less accurate method of DFT for computers.

Spectral Slices The first step in Fourier Analysis is to window the signal. I.e., break it all up into a series of smaller, analyzable chunks. This is important because the spectral qualities of the signal change over time. a “window” Check out the typical window length in Praat.

The Basic Idea For the complex wave extracted from each window... Fourier Analysis determines the frequency and intensity of the sinewave components of that wave. Do this about 1000 times a second, turn the spectra on their sides, and you get a spectrogram.

Possible Problems What would happen if a waveform chunk was windowed like this? Remember, the goal is to determine the frequency and intensity of the sinewave components which make up that slice of the complex wave.

The Usual Solution The amplitude of the waveform at the edges of the window is normally reduced... by transforming the complex wave with a smoothing function before spectral analysis. Each function defines a particular window type. For example: the “Hanning” Window

There are lots of different window types... each with its own characteristic shape Hamming BartlettGaussian HanningWelchRectangular

Window Type Ramifications Play around with the different window types in Praat.

Ideas Once the waveform has been windowed, it can be boiled down into its component frequencies. Basic strategy: Determine whether the complex wave correlates with sine (and cosine!) waves of particular frequencies. Correlation measure: “dot product” = sum of the point-by-point products between waves. Interesting fact: Non-zero correlations only emerge between the complex wave and its harmonics! (This is Fourier’s great insight.)

A Not-So-Complex Example Let’s build up a complex wave from 8 samples of a 1 Hz sine wave and a 4 Hz cosine wave. Note: our sample rate is 8 Hz A1 Hz B4 Hz CSum: Check out a visualization.

Correlations, part 1 Let’s check the correlation between that wave and the 1 Hz sinewave component CSum: A1 Hz: C*ADot: The sum of the products of each sample is 4. This also happens to be the dot product of the 1 Hz wave with itself. = its “power”

Correlations, part 2 Let’s check the correlation between the complex wave and a 2 Hz sinewave (a non-component) CSum: D2 Hz: C*DDot: The sum of the products of each sample is 0.  We now know that 2 Hz was not a component frequency of the complex wave.

Correlations, part 3 Last but not least, let’s check the correlation between the complex wave and the 4 Hz cosine wave CSum: B4 Hz C*BDot: The sum of the products of each sample is 8. Yes, 8 happens to be the dot product of the 4 Hz wave with itself. its “power”

Mopping Up Our component analysis gave us the following dot products: C*A = 4(A = 1 Hz sinewave) C*D = 0(D = 2 Hz sinewave) C*B = 8(B = 4 Hz cosine wave) We have to “normalize” these products by dividing them by the power of the “reference” waves: power (A) = A*A = 4  C*A/A*A = 4/4 = 1 power (D) = D*D = 4  C*D/D*D = 0/4 = 0 power (B) = B*B = 8  C*B/B*B = 8/8 = 1 These ratios are the amplitudes of the component waves.

Let’s Try Another Let’s construct another example: 1 Hz sinewave + a 4 Hz cosine wave with half the amplitude A1 Hz *B4 Hz ESum: Let’s check the 1 Hz wave first: ESum: A1 Hz E*ADot: Sum = 4

Yet More Dots Another example: 1 Hz sinewave + a 4 Hz cosine wave with half the amplitude. Now let’s check the 4 Hz wave: ESum: B4 Hz E*BDot: The sum of these products is also 4. = half of the power of the 4 Hz cosine wave.  The 4 Hz component has half the amplitude of the 4 Hz cosine reference wave. (we know the reference wave has amplitude 1)

Mopping Up, Part 2 Our component analysis gave us the following dot products: E*A = 4(A = 1 Hz sinewave) E*B = 4(B = 4 Hz cosine wave) Let’s once again normalize these products by dividing them by the power of the “reference” waves: power (A) = A*A = 4  E*A/A*A = 4/4 = 1 power (B) = B*B = 8  E*B/B*B = 4/8 =.5 These ratios are the amplitudes of the component waves. The 1 Hz sinewave component has amplitude 1 The 4 Hz cosine wave component has amplitude.5

Footnote Sinewaves and cosine waves are orthogonal to each other.  The dot product of a sinewave and a cosine wave of the same frequency is Asin Fcos A*FDot: However, adding cosine and sine waves together simply shifts the phase of the complex wave. Check out different combos in Praat.

Problem #1 For any given window, we don’t know what the phase shift of each frequency component will be. Solution: 1.Calculate the correlation with the sinewave 2.Calculate the correlation with the cosine wave 3.Combine the resulting amplitudes with the pythagorean theorem: Take a look at the java applet online:

Sine + Cosine Example Let’s add a 1 Hz cosine wave, of amplitude.5, to our previous combination of 1 Hz sine and 4 Hz cosine waves C1+4: *Fcos GSum: Let’s check the 1 Hz sine wave again: GSum: A1 Hz G*ADot: Sum = 4

Sine + Cosine Example Now check the 1 Hz cosine wave: GSum: F1 Hz G*FDot: Sum = 2 Sinewave component amplitude = 4/4 = 1 Cosine wave component amplitude = 2/4 =.5 Total amplitude = Check out the amplitude of the combo in Praat.

In Sum To perform a Fourier analysis on each (smoothed) chunk of the waveform: 1.Determine the components of each chunk using the dot product-- Components yield a dot product that is not 0 Non-components yield a dot product that is 0 2.Normalize the amplitude values of the components  Divide the dot products by the power of the reference wave at that frequency 3.If there are both sine and cosine wave components at a particular frequency: Combine their amplitudes using the Pythagorean theorem