1. CS 591 S1 – Computational Audio -- Spring, 2017
Wayne Snyder Computer Science Department Boston University
Today
Multiplying (and squaring) Sine Waves – the foundation of frequency detection!
F0 Determination
Zero-Crossing Rate
Correlation and Autocorrelation
Next Time:
F0 Determination continued: peak picking and refinements
Pitch Tracking 1

2. Digital Audio Fundamentals: Multiplying/Squaring Signals
We have spent nearly a month learning how to add sine waves together; we will spend the next month learning what happens when we multiply!
In fact, sophisticated ways of detecting frequencies in a signal will depend on what happens when we multiply one signal by another. So let’s investigate!
Ring Modulation consisted of multiplying a carrier wave by a modulating wave, generally of different frequencies:
The result is a signal, whose spectrum can be calculated from the carrier and modulating signals:
Carrier wave fc is split into two sideband frequencies fc ± fm of amplitude (Ac+Am) / 2. 2

3. Digital Audio Fundamentals: Multiplying/Squaring Signals
Important Digression: In a musical signal, the values alternate between positive and negative. As long as there is an integral number of periods within the window, this means that the average sample value (or the sum of the samples) is very close to 0.0:
sum = 0
for k in range(len(X)):
sum += X[k]
print( sum / len(X) )
Average Sample Value:
e-17
e-17
e-17 3

4. Digital Audio Fundamentals: Multiplying/Squaring Signals
Definition: A (relatively) small slice of a signal is called a window and the window length W is the duration in samples or in seconds.
Definition: For a window of length W samples, a window frequency is one whose period P is such that W = P * k for some integer k, i.e., an integral number of periods exactly fit within the window; alternately, it begins and ends at same instantaneous phase.
Fact 1: For any particular window, the window frequencies form a harmonic series. 4

5. Digital Audio Fundamentals: Multiplying/Squaring Signals
Fact 2: The sum (or average) of the samples in window of a sine wave which has a window frequency will be very close to 0.0. The phase does not matter.
Examples: Windows of 1 sec with waves (3.0,1.0,0.0) and (3.0,1.0,2.0):
Sum
e-15
e-07 5

6. Digital Audio Fundamentals: Multiplying/Squaring Signals
If the wave is not at a window frequency (i.e., it ends at a different instantaneous phase than it began), then the sum could be positive or negative, due to the last, incomplete period:
If we take the average sample value over larger and larger windows of a sine wave, we get a gradual convergence to a limit of 0.0:
Sum
49,645, 6

7. Digital Audio Fundamentals: Multiplying/Squaring Signals
Fact 3: When you add together two window frequency components (i.e., Additive Synthesis), you will get a signal whose sum (and average) is very close to 0.0.
Average Sample Value:
e-13
e-13
e-12 + = 7

8. Digital Audio Fundamentals: Multiplying/Squaring Signals
Fact 4: When you multiply two window frequencies fc and fm (i.e., Ring Modulation), you will get an average sample value very close to 0.0 as long as fc ≠ fm.
Average Sample Value:
e-13
e-13
e-08 * = 8 8

9. Digital Audio Fundamentals: Multiplying/Squaring Signals
Fact 4: When you multiply two window frequencies fc and fm (i.e., Ring Modulation), you will get an average sample value very close to 0.0 as long as fc ≠ fm.
WHY? Recall that Ring Modulation with (fc, Ac, _) * (fm, Am, _) removes the carrier frequency and replaces it with two sideband frequencies (fc ± fm, (Ac, Am) / 2, _ ); thus any Ring Modulation signal is identical to one produced by Additive Synthesis. * = 9 9

10. Digital Audio Fundamentals: Multiplying/Squaring Signals
But what happens in multiplying two sine waves when the frequencies of the two sine waves is the same? Why does this result not hold?
Multiplying two identical sine waves (i.e., squaring it) produces a sine wave of twice the frequency but shifted in phase and amplitude:
Average Sample:
0.0
0.25 + -
This is consistent with the “half angle formula” for cos:
(Obvious) Note: All the values in the squared wave are ≥ 0. Average is > 0. + - + + 10

11. Digital Audio Fundamentals: Multiplying/Squaring Signals
These results are consistent with the spectral analysis of Ring Modulation: Ring Modulation with carrier ( fc, Ac, 0 ) and modulator ( fm, Am, 0 ) with fc = fm
produces:
( 0, (Ac*Am)/2, 0 ) and ( 2*fc, (Ac*Am)/2, 0 )
where the 0 Hz wave is a constant offset of (Ac*Am)/2 to the wave:

12. Digital Audio Fundamentals: Multiplying/Squaring Signals
Ok, let’s summarize what happens…..
Let’s consider the product of two sine waves ( 5.0, 1.0, 0.0) and ( f, 1.0, 0.0 )
for f in [0.0, 0.2, 0.4, …, 4.6, 4.8, 5.0, 5.2, 5.4, …, 9.8 ]: 12

13. Digital Audio Fundamentals: Multiplying/Squaring Signals
Alternately, we could use the formula for the product of sine waves:
sin(x)*sin(y) = ½ [ cos(x-y) – cos(x+y) ]
so in the case where x and y differ by δ:
sin(x+δ)*sin(x) =
½ [ cos(δ) –cos(2x+δ) ]
When δ = 0, we have
½ [ –cos(2x) ]
Amplitude displacement in range [ ]
twice-frequency wave with shifted phase and inverted amplitude 13

14. Digital Audio Fundamentals: Multiplying/Squaring Signals
The MOST IMPORTANT thing to remember, however, is that when you multiply a sine wave by itself, all the values in the squared wave are ≥ 0 and the sum (or average) sample value is > 0.
Average Sample:
0.0
0.25 14

15. Digital Audio Fundamentals: Multiplying/Squaring Signals
What happens when the signal is composite (not a simple sine wave)?
Let’s track the average sample value when multiplying a composite wave created from [ ( 3.0,0.5,0.0), (5.0,0.3,0.0), (10,0.2,0.0) ] by a “probe wave”
created from [ ( f, 1.0, 0.0 ) ] for various frequencies f … 15

16. Digital Audio Fundamentals: Multiplying/Squaring Signals
What happens when the signal is composite (not a simple sine wave)?
Let’s track the average sample value when multiplying a composite wave created from [ ( 3.0,0.5,0.0), (5.0,0.3,0.0), (10,0.2,0.0) ] by a “probe wave”
created from [ ( f, 1.0, 0.0 ) ] for various frequencies f … 16

17. Digital Audio Fundamentals: Multiplying/Squaring Signals
What happens when the signal is composite (not a simple sine wave)?
Let’s track the average sample value when multiplying a composite wave created from [ ( 3.0,0.5,0.0), (5.0,0.3,0.0), (10,0.2,0.0) ] by a “probe wave”
created from [ ( f, 1.0, 0.0 ) ] for various frequencies f … 17

18. Digital Audio Fundamentals: Multiplying/Squaring Signals
What happens when the signal is composite (not a simple sine wave)?
Let’s track the average sample value when multiplying a composite wave created from [ ( 3.0,0.5,0.0), (5.0,0.3,0.0), (10,0.2,0.0) ] by a “probe wave”
created from [ ( f, 1.0, 0.0 ) ] for various frequencies f …
Note that average values are exactly half the amplitudes in the original spectrum; when a “probe wave” matches the frequency and phase exactly, you get a bump in the average value, due to the squared part! 18

19. Digital Audio Fundamentals: Multiplying/Squaring Signals
What happens when the signal is composite (not a simple sine wave)?
You can think of this as a simple application of (pair-wise) plus and multiplication on vectors, which obeys the typical arithemetic rules
X3 = signal from [ ( 3.0, 0.5, 0.0 ) ]
X5 = signal from [ ( 5.0, 0.3, 0.0 ) ]
X10 = signal from [ ( 10.0, 0.2, 0.0 ) ]
P = “probe wave” = signal from [ (3.0, 1.0, 0.0) ]
X = X3 + X5 + X10
X5 * P average value will be close to 0.0
X3 * P average value will be close to 0.25 = half amplitude of X3
X * P = P * (X3 + X5 + X10) = ( P * X3 ) + (P * X5 ) + (P * X10) avg close to 0.25
avg: 19

20. Digital Audio Fundamentals: Multiplying/Squaring Signals
Conclusions on Multiplying Signals in a Window:
We (for now) assume that all sine waves are at window frequencies.
If a “probe wave” EXACTLY matches a component of a signal in frequency AND phase, the average sample value will be half the amplitude of the component.
If a “probe wave” does not match a component, the average sample value will be very close to 0.0.
This gives us a way to detect frequencies in a signal…. sort of…..
Problems:
Frequency must match exactly
Phase must match exactly
Signal must be composed entirely of window frequencies.
We will deal with these problems in two phases, one as part of pitch determination and one as part of understanding the Fourier Transform…. 20

21. Digital Audio Fundamentals: F0 (lowest frequency) Detection
Detecting the fundamental frequency of a signal (“pitch detection”, more properly “F0 detection”) is a good place to start in our study of musical signal analysis. For some signals it is easy to see their fundamental frequency:
– =
1/ = 220 Hz
100 Hz
These signals have a clear sense of a dominant pitch. 21

22. Digital Audio Fundamentals: F0 Detection
Other sounds are harder to determine or have no pitch:
[Example: Beethoven’s Seventh Symphony.]
[Higher love, percussion opening.]
Interestingly, although we are not always conscious of it, speech sounds do usually have a dominant pitch, although some consonants (“s”) are practically pitchless.
[Genesis01.wav] 22

23. Digital Audio Fundamentals: F0 Detection with ZCR
The simplest pitch detection is the Zero-Crossing Rate (ZCR) Algorithm; since a sine wave crosses the 0 line twice in its period, you could just count the number of times the signal changes sign (+ to – or – to +) in a signal window and divide by 2*length of the window:
4/(2*0.1s) = 20 Hz 23

24. Digital Audio Fundamentals: F0 Detection with ZCR
There are various problems with the approach, first, that it can be off by half a period:
4/(2*0.1s) = 20 Hz but the signal is 23.2 Hz 24

25. Digital Audio Fundamentals: F0 Detection with ZCR
There are various problems with the approach, first, that it can be off by half a period:
Works better when frequency is large relative to window size….. 25

26. Digital Audio Fundamentals: F0 Detection with ZCR
The second problem is that higher harmonics in the signal may create too many zero crossings!
The solution is typically to smooth the signal
using a “low-pass filter” that filters out the higher
harmonics….. this is the approach taken by cheap
“chromatic tuners for guitars”…. 26

27. Digital Audio Fundamentals: Correlation and AutoCorrelation
How do we find the period of a periodic signal? In fact, what we need is to determine at what distance the signal is periodic….
Recall: a signal is periodic with period P if X[k] == X[k+P] for all k < len(X) – P: P
X[k] X[k+P] 27

28. Digital Audio Fundamentals: Correlation and AutoCorrelation
In other words, we need to find the correlation between the signal, and the signal shifted by P samples:
When we find the correlation of a signal with a shifted version of itself, this is called
Auto-Correlation
X[k]
Signal shifted by P samples 28
X[k+P]

29. Digital Audio Fundamentals: Correlation and AutoCorrelation
Correlation measures how similar two random variables (or vectors or signals) are:
Pearson’s Product-Moment Coefficient for random variables X and Y:
But for signals, there are several simplifying conditions:
Since we will be interested only in comparing magnitudes (e.g., finding the maximum), we do not need to normalize to the range [-1..1], and so can eliminate the denominator;
Since signals have a mean very close to 0, we do not need to standardize by subtracting the means (already standardized!)
So we would have: average sample value = E( X * Y ) = Σ(Xi * Yi) / len(X) but, again, as in (1), we do not need the denominator!
We have simply: Σ(Xi * Yi) = dotProduct(X,Y) = sum of product of X and Y 29

30. Digital Audio Fundamentals: Correlation and AutoCorrelation
In fact, the correlation varies as a sine wave, with a maximum at 0 and 2π and a minimum at π, and 0 at π/2 and 3π/2 (effectively a cosine): 30

31. Digital Audio Fundamentals: Correlation and AutoCorrelation
Autocorrelation is the same thing as correlation, but the Y signal is a time-shifted version of the X signal:
Correlate X[ L : R ] with X[ L+lag: R+ lag ]
Σ(Xi * Xi+lag) 31
Lag

32. Pitch Determination using Autocorrelation
The code is (as usual) fairly simple:
def acorr(X, lag):
sum = 0
for i in range(len(X) - lag):
sum += X[i]*X[i+lag]
return sum/(len(X)-lag) 32
Lag

33. Pitch Determination using Autocorrelation
When you have a simple sine wave and the perfect lag, you are essentially squaring the wave and this maximizes the sum of products.
Lag 33

34. Pitch Determination using Autocorrelation
When we graph the autocorrelation coefficient for various lag times, the peaks in the curve show where the correlation between the lagged signal is strongest, which corresponds to frequency F0. Let’s look at some (artificial) examples.
Spectrum:
[ ( 441, 1.0, 0 ) ] 34

35. Pitch Determination using Autocorrelation
Spectrum:
[ ( 441, 1.0, pi ) ]
Note that the phase does not matter! 35

36. Pitch Determination using Autocorrelation
Spectrum:
[ ( 441, 1.0, 2.3 ) ]
Note that the phase does not matter! 36

37. Pitch Determination using Autocorrelation
Spectrum (with even harmonics):
[ (441,0.6,0),
(441*3,0.2,0),
(441*5,0.2,0) ] 37

38. Pitch Determination using Autocorrelation
Spectrum (with even harmonics):
[ (441,0.6,0),
(441*3,0.2,0),
(441*5,0.2,0) ]
Spectrum:
[ (110, 1.0, 0 )] 38

39. Pitch Determination using Autocorrelation
Bell Spectrum:
220 Hz 39

40. Pitch Determination using Autocorrelation
Steel String Spectrum:
220 Hz 40

41. Pitch Determination using Autocorrelation
Now let’s try some actual music signals….. Cello.wav at 11.35 41

42. Pitch Determination using Autocorrelation
WTC1_01.wav at 0.3 sec 42

43. Pitch Determination using Autocorrelation
SteelString.wav at 0.5 sec (220 Hz)
Bell Spectrum:
220 Hz 43

44. Pitch Determination using Autocorrelation
SteelString.wav at sec (220 Hz) 44

45. Pitch Determination using Autocorrelation
SteelString.wav at sec for seconds duration 45

46. Pitch Determination using Autocorrelation
SteelString.wav at sec for 0.01 seconds duration 46

47. Pitch Determination using Autocorrelation
Genesis01.wav at 2.87 sec 47

48. Pitch Determination using Autocorrelation
Beethoven.Seventh.wav at 3.125 48

49. Pitch Determination using Autocorrelation
Bell.wav at 0.03 49

50. Pitch Determination using Autocorrelation
Cymbal.wav at 0.3 sec 50

51. Pitch Determination using Autocorrelation
HigherLove.wav at sec 51

52. EXTRA SLIDES
But what happens when the frequencies of the two sine waves is the same?
Multiplying two sine waves of the same frequency but opposite phase produces the inverse of the product:
Average Sample:
0.0
- 0.25 + -
This makes sense if we think of signals as vectors which we can multiply pairwise:
X * X = X2
X * - X = - (X * X) = - X2 + - - - 52

53. Digital Audio Fundamentals: Multiplying/Squaring Signals
But what happens when the frequencies of the two sine waves is the same?
Multiplying two sine waves of the same frequency but off by π/2 (e.g., sin and cos) produces a (normal) sine wave (of twice the frequency).
This is consistent with the formula for the product of two sin expressions:
sin(x)*sin(y) = ½ [ cos(x-y) – cos(x+y) ]
so:
sin(x+φ)*sin(x+ξ) =
½ [ cos(φ+ξ) – cos(2x+(φ+ξ) ) ]
Amplitude displacement in range [ ]
twice-frequency wave with shifted phase and inverted amplitude 53

54. Digital Audio Fundamentals: Multiplying/Squaring Signals
The MOST IMPORTANT thing to remember, however, is that when you multiply a sine wave by itself, all the values in the squared wave are ≥ 0 and the sum (or average) sample value is > 0.
Average Sample:
0.0
0.25 54