1. Week 5 – Nyquist-Shannon ESE 250 – S’12 Kod & DeHon 1 ESE250: Digital Audio Basics Week 5 Feb. 9, 2012 Nyquist-Shannon Theorem

2. 2 Course Map Numbers correspond to course weeks 2,5 6 11 13 12 Today ESE 250 – S’12 Kod & DeHon Week 5 – Nyquist-Shannon

3. Where are we ? Week 2  Received signal is sampled & quantized  q = PCM[ r ] Week 3  Quantized Signal is Coded  c =code[ q ] Week 4  Sampled signal first transformed into frequency domain  Q = DFT[ q ] Week 5  signal oversampled & low pass filtered  Q = LPF[ DFT(q+n) ] Week 6  Transformed signal analyzed  Using human psychoaoustic models Week 7  Acoustically Interesting signal is “perceptually coded”  C = MP3[ Q] Over Sample DFT LPF DecodeProduce r(t)r(t) p(t)p(t) q + n C Perceptual Coding Store / Transmit Q + N Q Week 4 Week 6 Week 5Week 3 [Painter & Spanias. Proc.IEEE, 88(4):451–512, 2000] Week 5 – Nyquist-Shannon ESE 250 – S’12 Kod & DeHon 3

4. Reconstruction Acquisition Side  we’ve convinced ourselves  to sample,  then process,  then store and transmit  discretely sampled values On the other end?  how should the stored sound be produced?  ? send strings of sampled spl levels o off to the sound card o at what rate?  ? Interpolate o with what family of interpolants? Week 4: harmonic reconstruction  Gets better with more terms  But need more samples to compute them  and error seems unpredictable This week: general reconstruction  introduce further assumptions about signal o to guarantee exact finite reconstruction o with appropriate basis functions  introduce another processing step  to achieve those assumptions Generic Digital Signal Processor Sample r(t)r(t) q Code c Store/ Transmit DecodeProduce p(t)p(t) Week 5 – Nyquist-Shannon 4 ESE 250 – S’12 Kod & DeHon

5. Week 5 – Nyquist-Shannon ESE 250 – S’12 Kod & DeHon 5 Shannon’s Theorem M. Unser. Proc. IEEE, 88(4):569–587, 2000 Questions: are the real functions countable after all? what is “frequency content”? b0(t)b0(t) Hypothesis  given a (sufficiently “nice”) real function, r(t)  whose frequency content does not exceed  M = 2  f M Conclusion  There is a set of “sampling” basis functions (we’ll not discuss) B S = { …, b -2 (t), b -1 (t), b 0 (t), b 1 (t), b 2 (t), … }  which can exactly reconstruct the function r(t) = … + r -2 b -2 (t) + r -1 b -1 (t) + r 0 b 0 (t) + r 1 b 1 (t) + r 2 b 2 (t) + …  from samples r k = r(k T M )  taken at sampling intervals T M =  /  M = 1 / (2 f M )  called the “Nyquist” rate

6. Trial Application to Our Setting Psychoacoustic measurements  introduced in next (week 6) lecture  show human audition is bandlimited  at frequency  A = 2  f A for f A = 22 kHz Naïve Processing Strategy  sample received signal, r(t)  at intervals T A =  /  A = 1 /2 f A = [ 44 ¢ 10 3 ] -1 sec  reconstruct exactly as needed r(t) = … + r -2 b -2 (t) + r -1 b -1 (t) + r 0 b 0 (t) + r 1 b 1 (t) + r 2 b 2 (t) + …  from samples r = {…, r -2, r -1, r 0, r 1, r 2, … }  where r k = r(k T A ) Why bother with DFT at all?? Week 5 – Nyquist-Shannon 6 ESE 250 – S’12 Kod & DeHon

7. Poking the Trial Balloon In reality we receive a “mixed” signal  r(t) = q(t) + n(t)  q(t) q(t) o signal component of auditory interest o has bandwidth f A = 22 kHz  n(t) n(t) o noise (uninteresting information ) o has (typically) high frequency content  impurities associated with transmission and recording  background sounds We only get to sample  the received signal, r(t),  not the desired signal, q(t) !! Week 5 – Nyquist-Shannon 7 ESE 250 – S’12 Kod & DeHon

8. In class experiment:  assume a nominal band limit  at frequency  N = 2  f N for f N = 5/4 Hz [samples/sec]  Question: what is the Nyquist sample rate? o T N = 1/(2 f N ) = 2/5 sec [sample/sec ] -1 In this experiment, assume that  q(t) ´ 0 o signal component of auditory interest o satisfies bandwidth requirement of f N = 5/4 Hz  n(t) = Cos[9/5  N t] o noise (uninteresting information ) o has “high” frequency content o i.e., it is above (by 9/5) the presumed Nyquist rate Class will sample r(t) = q(t) + n(t)  over “window” - T 0 /2 < t < T 0 /2 for T 0 = 4 [sec]  at the presumed Nyquist rate  Questions: o what is the number of Nyquist samples?  n S ¢ T N = T 0  Number of samples: n S = 2 f N T 0 = 10 o what are the Nyquist sample times? t 2 {-2,-(8/5),-(6/5),-(4/5),-(2/5),0,2/5,4/5,6/5,8/5,2} q(t)q(t) n(t)n(t) r(t) = q(t) + n(t) Bursting the Trial Balloon Week 5 – Nyquist-Shannon 8 ESE 250 – S’12 Kod & DeHon

9. Bursting the Trial Balloon Class will sample r(t) = q(t) + n(t)  over “window” - T 0 /2 < t < T 0 /2 for T 0 = 4  at the presumed Nyquist rate o Number of samples: n S = 2 f N T 0 = 10 o Sample instants: t 2 {-2,-(8/5),-(6/5),-(4/5),- (2/5),0,2/5,4/5,6/5,8/5,2} and get an “aliased” version of the noise!  r(t) looks like a sampled version of  n(t) = Cos[1/5  N t] Week 5 – Nyquist-Shannon 9 ESE 250 – S’12 Kod & DeHon

10. Aliasing Widely familiar phenomenon  Wikipedia aliasing article Wikipedia aliasing article  “wagon wheel” effect “wagon wheel” effect  Wikipedia Nyquist-Shannon Theorem Wikipedia Nyquist-Shannon Theorem “Folding:”  another general manifestation of aliasing  premise: r(t) = q(t) + n(t) o we expect q(t) with bandwidth less than  N = 2  f N o but turns out that n(t) is some “ superharmonic residue”  n(t) = cos[ m  N (1+  ) t ]  where 0 <  <1  outcome: sample at “Nyquist rate” T N = 1/(2 f N ) o r k = r( k T N ) = q(k T N ) + n(k T N ) o where the noise n( k T N ) = cos[m  N (1+  ) k T N ] = cos[mk  (1+  ) ] = cos[ mk  ¢  cos[mk  ] - sin[ mk  ¢  sin[mk  ] = 1 ¢ cos[mk  ] - 0 ¢ sin[mk  ] = cos[   N k T N ] o “folds over” to act as if it were a low-band tone of frequency  f N Week 5 – Nyquist-Shannon 10 ESE 250 – S’12 Kod & DeHon

11. Anti-Aliasing Given the “mixed” signal  r(t) = q(t) + n(t)  q(t) – auditory signal with bandwidth f A = 22 kHz  n(t) – high frequency noise We require some “anti-aliasing” pre-process  that “smooths away” the noise  which will otherwise appear in the samples o r = {…, r -2, r -1, r 0, r 1, r 2, … } o where r k = r(k T A ) o and T A =  /  A = 1 /2 f A = [ 44 ¢ 10 3 ] -1 sec Question: how to do this? Week 5 – Nyquist-Shannon 11 ESE 250 – S’12 Kod & DeHon

12. Week 5 – Nyquist-Shannon ESE 250 – S’12 Kod & DeHon 12 Interlude: Visual Aliasing Wolfram: Drawing a Line on Digital Display Berkeley Course Demo Typography

13. Back to Shannon (new today) Hypothesis  given a (sufficiently “nice”) real function, r(t)  whose frequency content does not exceed  M = 2  f M Conclusion  There is a set of “sampling” basis functions (we’ll not show) B S = { …, b -2 (t), b -1 (t), b 0 (t), b 1 (t), b 2 (t), … }  which can exactly reconstruct the function r(t) = … + r -2 b -2 (t) + r -1 b -1 (t) + r 0 b 0 (t) + r 1 b 1 (t) + r 2 b 2 (t) + …  from samples r k = r(k T M )  taken at sampling intervals T M =  /  M = 1 /(2 f M )  called the “Nyquist” rate M. Unser. Proc. IEEE, 88(4):569–587, 2000 Questions: are the real functions countable after all? what is “frequency content”? what are the “basis functions” ? Week 5 – Nyquist-Shannon 13 ESE 250 – S’12 Kod & DeHon

14. Week 5 – Nyquist-Shannon ESE 250 – S’12 Kod & DeHon 14 Fourier’s Theorem (review) Hypothesis  given a (sufficiently “nice”) real function, r(t)  which is periodic with period T 0 r(t) = r(t + T 0 ) Conclusion  the set of “harmonic” basis functions B H = { …, h -2 (t), h -1 (t), h 0 (t), h 1 (t), h 2 (t), … }  at frequency  0 = 2  f 0 for f 0 = 1/T 0  defined by h k (t) = cos k  0 t (k>0) h k (t) = 1 (k>0) h k (t) = sin k  0 t (k<0)  can exactly reconstruct the function r(t) = … + R -2 h -2 (t) + R -1 h -1 (t) + R 0 h 0 (t) + R 1 h 1 (t) + R 2 h 2 (t) + … M. Unser. Proc. IEEE, 88(4):569–587, 2000 Question: are the real functions countable after all?

15. Clipping a Non-Periodic Function If  we are willing  to restrict attention  to a specific observation window -T 0 /2 · t · T 0 /2 Then  Fourier’s Theorem yields  exact reconstruction  of the periodic extension  of any function Answer: apparently, “periodic” (or, more practically, finite time) real functions are countable after all! - T 0 /2T 0 /2T0T0 -T 0 - T 0 /2T 0 /2T0T0 -T 0 Week 5 – Nyquist-Shannon 15 ESE 250 – S’12 Kod & DeHon

16. Fourier + Shannon Fourier: period T 0 ) exact harmonic reconstruction  at frequency  0 = 2  f 0 for f 0 = 1/T 0  r(t) = … + R -2 h -2 (t) + R -1 h -1 (t) + R 0 h 0 (t) + R 1 h 1 (t) + R 2 h 2 (t) + … Shannon: bandwidth  M ) exact “sampled” reconstruction  at frequency  M = 2  f M for f M = 1/(2T M )  r(t) = … + r -2 b -2 (t) + r -1 b -1 (t) + r -0 b 0 (t) + r 1 b 1 (t) + r 2 b 2 (t) + … clipped frequency forces finite harmonic series clipped time forces finite samples Week 5 – Nyquist-Shannon 16 ESE 250 – S’12 Kod & DeHon

17. Fourier + Shannon:  period T 0 & bandwidth  M = 2  f M ) finite exact harmonic reconstruction  with n M = Round[T 0 / T M ] samples  at “Maximal Rate” T M = 1 /(2 f M ) Algebra:  k  0 >  M ) R -k sin k  0 t ´ R k cos k  0 t ´ 0 (otherwise high frequency) ) R -k = R k = 0 (sin & cos never identically zero)  k > n M ) r (t) = R -n M h -n M (t) + … + R -2 h -2 (t) + R -1 h -1 (t) + R 0 h 0 (t) + R 1 h 1 (t) + R 2 h 2 (t) + … + R n M h n M (t) Now we’re ready to solve the anti-aliasing problem Fourier + Shannon ) Finite Series Week 5 – Nyquist-Shannon 17 ESE 250 – S’12 Kod & DeHon

18. Now postulate:  New signal, q(t), satisfying o period T 0 o psychoacoustic bandwidth  A = 2  f A for f A = 22 kHz o bounded by maximal bandwidth  A <  M  sampled with o n A = Round[T 0 / T A ] samples o at “Auditory Nyquist Rate” T A = 1 /(2 f A ) Fourier + Shannon:  period T 0 & bandwidth  A ) finite exact harmonic reconstruction q(t) = Q -n A h -n A (t) + … + Q -2 h -2 (t) + Q -1 h -1 (t) + Q 0 h 0 (t) + Q 1 h 1 (t) + Q 2 h 2 (t) + … + Q n A h n A (t)  vector representation: o Q = (Q -n A, …, Q -2, Q -1, Q 0, Q 1, Q 2, …, Q n A ) Shannon + Fourier  bandwidth  A & period T 0 ) finite exact “sampled” reconstruction o q(t) = q -n A b -n A (t) + … + q -2 b -2 (t) + q -1 b -1 (t) + q -0 b 0 (t) + q 1 b 1 (t) + q 2 b 2 (t) + … + q n A b n A (t)  vector representation: o q = (q -n A, …, q -2, q -1, q 0, q 1, q 2, …, q n A ) Question: what is the relationship between the representations  frequency domain, Q  time domain, q Answer: Q = DFT(q) Psychoacoustics meets Nyquist Week 5 – Nyquist-Shannon 18 ESE 250 – S’12 Kod & DeHon

19. Q+N = DFT(q) + DFT(n) = DFT(q+n) Assume we receive mixed signal  r(t) = q(t) + n(t)  q(t) q(t) o signal component of interest o has period T 0 & bandwidth  A  n(t) n(t) o noise (uninteresting information ) o has (for now) only high frequency content, , i.e., satisfies:  A <  ·  M Problem: how to remove noise? Frequency Domain approach  Take the received time-sampled representation, r = q + n r = (r -n M, …, r -2, r -1, r 0, r 1, r 2, …, r n M ) = (q -n M + n -n M, …, q -2 + n -2, q -1 + n -1, q 0 + n 0, q 1 + n 1, q 2 + n 2, …, q n M + n -n M )  Compute frequency domain representation o R = DFT(r) = DFT(q+n) = DFT(q)+ DFT(n) = Q + N o R = (R -n M, …, R -2, R -1, R 0, R 1, R 2, …, R n M ) = (Q -n M + N -n M, …, Q -2 + N -2, Q -1 + N -1, Q 0 + N 0, Q 1 + N 1, Q 2 + N 2, …, Q n M + N -n M )  Recall the meaning of this representation: for h k (t) = trig[k  0 t] and trig 2 {cos, sin} we have r(t) = (Q -n M + N -n M ) h -n M (t) + … + (Q -n A + N -n A ) h -n A (t) + … + (Q -n M + N -n M ) h -2 (t) + (Q -n M + N -n M ) h -1 (t) + (Q -n M + N -n M ) h 0 (t) + (Q -n M + N -n M ) h 1 (t) + (Q -n M + N -n M ) h 2 (t) + … + (Q -n A + N -n A ) h n A (t) + … + (Q -n M + N -n M ) h n M (t)  Now apply assumptions about frequency content! Week 5 – Nyquist-Shannon 19 ESE 250 – S’12 Kod & DeHon

20. R = DFT(r), r = DFT -1 (R ) Compute frequency domain representation  R = (R -n M, …, R -2, R -1, R 0, R 1, R 2, …, R n M ) = (Q -n M + N -n M, …, Q -2 + N -2, Q -1 + N -1, Q 0 + N 0, Q 1 + N 1, Q 2 + N 2, …, Q n M + N -n M ) Now introduce assumptions about frequency content:  k > n A =  A /  0 ) Q k = 0  k < n A =  A /  0 ) N k = 0 Interpret in entries of R = (Q -n M + N -n M, …, Q -2 + N -2, Q -1 + N -1, Q 0 + N 0, Q 1 + N 1, Q 2 + N 2, …, Q n M + N -n M ) = (0 + N -n M, …, Q n A + N -n A, …, Q -2 + 0, Q -1 + 0, Q 0 + 0, Q 1 + 0, Q 2 + 0, …, Q n A + N -n M, …, 0 + N -n M ) = (0,…, 0, Q n A …, Q -2, Q -1, Q 0, Q 1, Q 2, …, Q n A,,, 0,…, 0 ) + (N -n M,…, N -n A, 0, …, 0, 0, 0, 0, 0, …, 0, N n A,…, N n M ) Implement Algorithmically  Zero out the “higher integer” entries of S = ZERO n A (R)  Obtain the inverse transform, s = DFT -1 (S) Conclude that s = q Week 5 – Nyquist-Shannon 20 ESE 250 – S’12 Kod & DeHon

21. Visualizing a Low Frequency Subspace Low Frequency Signal  Corrupted by  High frequency noise Week 5 – Nyquist-Shannon ESE 250 – S’12 Kod & DeHon 21 h0(t)h0(t) h1(t)h1(t) h -1 (t)

22. Oversampled Pre-Filter Anti-aliasing  requires“oversampling” o n M = Round[T 0 / T M ] samples o at “Maximal Rate” T M = 1 /(2 f M ) o Where f M represents our best estimate of o the highest noise frequency  but how should we model o “noise” o and its likely bandwidth? Analog pre-processing  physical electro-mechanical world o traditionally piezo-electric microphone o increasingly MEMS  Has its own limited bandwidth Electronics need merely  sample up to  the microphone “cut off” frequency Low-Pass Filter FilterTransform r Q q ADXL001 Accelerometer Pre-Sampling r Low-Pass Week 5 – Nyquist-Shannon 22 ESE 250 – S’12 Kod & DeHon

23. Week 5 – Nyquist-Shannon ESE 250 – S’12 Kod & DeHon 23 To Read Further Tutorial on Shannon’s Theorem M. Unser. Sampling - 50 years after shannon. Proceedings of the IEEE, 88(4):569–587, 2000

24. Week 5 – Nyquist-Shannon ESE 250 – S’12 Kod & DeHon 24 ESE250: Digital Audio Basics Week 5 Feb. 9, 2012 Nyquist-Shannon Theorem