Nov'04CS3291 Sectn 71 UNIVERSITY of MANCHESTER Department of Computer Science CS3291 : Digital Signal Processing ‘05 Section 7 Sampling & Reconstruction

Nov'04CS3291 Sectn 72 This section concerns DSP systems for operating on analogue signals which must first be sampled and digitised. Resulting digital signals often need to be converted back to analogue form or “reconstructed”. Before starting, review some facts about analogue signals.

Nov'04CS3291 Sectn Review of some analogue signals and systems theory: 1. Analogue Fourier Transform (FT) & its inverse: 2. Unit impulse  (t) has following properties: (i)  (t) = 0 for t  0 3. If  (t) applied to LTI analogue system, output is impulse-resp. h(t). Frequency-response is FT of h(t). 4. Fourier series for periodic signal x(t) with period T:

Nov'04CS3291 Sectn 74 Consider periodic signal, s(t) = repeat T {  (t)} s(t) = repeat T {  (t)} t T-T 0 T 2T3T Its Fourier series is:

Nov'04CS3291 Sectn 75 End of preliminaries

Nov'04CS3291 Sectn : Sampling an analogue signal Given x a (t) with Fourier Transform X a (j  ). Sample x a (t) at intervals T to obtain {x[n]}: {..., x[-1], x[0], x[1], x[2], x[3],... } t xa(t) T 2T 3T 4T 5T-T -3T -2T x[1] x[2] x[-1] x[-2] x[5] X[0] x[3] x[-3] x[1] = x a (T), x[2] = x a (2T), etc.

Nov'04CS3291 Sectn 77 x S (t) is a succession of delayed impulses, each multiplied by a sample value of {x[n]}. Define a new analogue signal:

Nov'04CS3291 Sectn 78 Note that: x S (t)

Nov'04CS3291 Sectn 79 Note that: x S (t)

Nov'04CS3291 Sectn 710 Fourier transform of x S (t) is It follows that: X S (j  ) = X(e j  ) with  =  T.  Fourier transform of x S (t) = DTFT of {x[n]}

Nov'04CS3291 Sectn : Relative frequency  =  T =  /f S radians per sample When  = , true frequency is f S / 2 Hz DTFT as function of  DTFT of {x[n]} = Fourier transform of x S (t) X(e j  ) = X S (j  ) with  =  T.

Nov'04CS3291 Sectn Relating DTFT X(e j  ) of {x[n]} to FT X a (j  ) of x a (t) We have related X(e j  ) to FT of x S (t), but not yet to FT of x a (t) x a (t) x S (t){x[n]}    X a (j  )  ?  X S (j  )=X(e j  ) t t {…, 3, -2, -2, 1, 1, -1, … } x a (t)x S (t) {x[n]} =

Nov'04CS3291 Sectn 713 Convenient form of the SAMPLING THEOREM: FT of sample T {x a (t)} is (1/T)repeat 2  /T {X a (j  )}   x S (t) X S (j  ) 'sample T {x a (t)}' already defined 'repeat 2  /T {X a (j  )}' means (loosely speaking) X a (j  ) repeated at frequency intervals of 2  /T.   /T -  /T 2  /T 4  /T -4  /T -2  /T X S (j  ) X a (j  )

Nov'04CS3291 Sectn 714 Proof of Sampling Theorem Re-express x s (t) as follows:

Nov'04CS3291 Sectn 715 Periodic s(t) has complex Fourier series: (Proved earlier)

Nov'04CS3291 Sectn 716 Remember: Complex Fourier series for s(t):- Periodic with frequency 1/T Hz. Thus:

Nov'04CS3291 Sectn 717

Nov'04CS3291 Sectn 718 TIMES EQUALS

Nov'04CS3291 Sectn 719  x S (t) = x a (t) s(t) with Taking Fourier transform: = (1/T) repeat 2  /T { X a (j  ) }

Nov'04CS3291 Sectn 720 Taking just 3 terms: Valid for any analogue signal x a (t) Significance of Sampling Theorem Next slide

Nov'04CS3291 Sectn 721 If x a (t) bandlimited between  /T rad/s (  f s /2 Hz): |X|X(j  )|  a   Spectrum of x S (t): |Xs(j  )|   

Nov'04CS3291 Sectn 722 So what?? X a (j  ), X a (j(  - 2  /T)), & X a (j(  + 2  /T) do not overlap Therefore: X S (j  ) = (1/T) X a (j  ) for -  /T <  <  /T Given x S (t) with FT X S (j  ) we can recover x a (t) exactly, by filtering off everything above  /T radians/second Aliasing distorton Next slide

Nov'04CS3291 Sectn 723 If X a (j  ) not bandlimited to  /T, overlap occurs: |Xa(j  )|    |Xs(j  )|     

Nov'04CS3291 Sectn 724 Effect of the overlap Now X S (j  )  X a (j  )/T for -  /T <  <  /T. X S (j  ) is distorted version of X a (j  )/T. This is “Aliasing” distortion. repeat 2/  {X(j  )} is sum of identical copies of (1/T)X a (j  ) each shifted in frequency by a multiple of  /T

Nov'04CS3291 Sectn 725 Given analogue signal x a (t) with FT X a (j  ) Let {x[n]} be obtained by sampling x a (t) at intervals of T seconds. X(e j  ) = DTFT of {x[n]} Let x S (t) = sample T {x a (t)} Then X S (j  ) = (1/T)repeat 2  /T {X a (j  )} And X S (j  ) = X(e j  ) with  =  T Properties of X(e j  ) related to X a (j  )

Nov'04CS3291 Sectn 726    X a (j  )  !  X S (j  )= X(e j  ) t t {…, 3, -2, -2, 1, 1, -1, … }x a (t)x S (t) {x[n]} =     /T-  /T  /T -  /T  --

Nov'04CS3291 Sectn 727 Properties of X(e j  ): summary i. If {x[n]} obtained by sampling x a (t) at f S = 1/T Hz when x a (t) is bandlimited to  f S /2 then: X(e j  ) = (1/T) X a (j  ) for -  <  <  with  =  T ii. X(e j  ) = X S (j  ) with  =  T. iii. Spectrum X(e j  ) repeats at intervals of 2 . iv. X(e - j  ) = X * (e j  ) for real signals.

Nov'04CS3291 Sectn Anti-aliasing filter:- To avoid aliasing, low-pass filter x a (t) to band-limit to  f S /2. It then satisfies “Nyquist sampling criterion”. Example 7.1: x a (t) has sinusoidal component at 7 kHz: It is sampled at 10 kHz without an antialiasing filter. What happens?

Nov'04CS3291 Sectn 729 Solution It becomes 3 kHz sine-wave & distorts signal. |X S (j2  f)| 5 k k f

Nov'04CS3291 Sectn 730 Example 7.2 : anti-aliasing filter specification An analogue signal of bandwidth  3 kHz is received with non- bandlimited noise added to it. It is to be sampled at 10 kHz. Analogue low-pass filter required to attenuate, by at least 24 dB, signal components likely to cause aliasing distortion to the 3 kHz signal. What order of Butterworth filter is required? Solution: Exercise (solution in notes)

Nov'04CS3291 Sectn : Reconstruction of x a (t) Let {x[n]} be x a (t) sampled at f S Hz with x a (t) band-limited to  f S /2: {x[n]} = { …, 1.1, 7.34, -23.6, -4, 0, 2.99, -5, 14.3, 8,… } Can we obtain x a (t) exactly from {x[n]} ?

Nov'04CS3291 Sectn 732 Need to construct an analogue signal whose FT is equal to T.X(e j  ) with  =  T for -  /T <  <  /T, & zero outside this range. X S (j  ) = X(e j  ) for  in range  / T &  =  T, but is not zero outside this range.

Nov'04CS3291 Sectn 733 |Xs(j  )|     |Xa(j  )|    Its magnitude spectrum is: If we use an ideal lowpass filter to remove everything outside  /T we get:

Nov'04CS3291 Sectn 734 Ideal reconstruction: In theory, to obtain x a (t) exactly from {x[n]}: 1) Construct x S (t) with ideal impulses. 2) Ideal low-pass filter with cut-off  /T to remove images. 3) Multiply by T. In practice cannot have ideal impulses, or an ideal low-pass filter. Each impulse approximated by pulse of finite voltage & non-zero duration:

Nov'04CS3291 Sectn 735 Easiest to use sample and hold (S/H) circuit. Produces a voltage  x[n] at t = nT. Holds this fixed until next sample at t = (n+1)T. Produces a “staircase waveform” x P (t) say. t x P (t) T 2T 3T t x S (t) /T 6/T 9/T 5/T

Nov'04CS3291 Sectn 736 Standard ADC produces x P (t) instead of x S (t). Pretend x P (t) produced by generating x S (t) & passing it through a filter with impulse response: t T h a (t) 1/T

Nov'04CS3291 Sectn 737 In theory (pretend):-

Nov'04CS3291 Sectn 738 In Reality:- x P (t) T 2T 3T 4/T 6/T 9/T 5/T {..., 4, 6, 9, 5, 3,.... } t ADC 3/T x P (t) will in practice be scaled.

Nov'04CS3291 Sectn 739 Effect of S/H approximation:- S/H filter has impulse response: Frequency response of S/H filter:

Nov'04CS3291 Sectn 740 Gain at  = 0 is 0 dB. Gain at  /T is 20 log 10 (2/  ) = dB. Using S/H incurs loss which  3.92 dB as    /T.

Nov'04CS3291 Sectn 741 Phase response linear: delay of T/2 seconds.

Nov'04CS3291 Sectn 742 In frequency-domain, instead of: |Xs(j  )|     we get: |X P (j  )|     shape changes as    /T

Nov'04CS3291 Sectn 743 Sometimes loss is disregarded. Analogue (or digital) compensation filter may cancel out loss.  /T 1 2/   /2 G(  ) 

Nov'04CS3291 Sectn 744 Idea for digital filter to compensate for S/H reconstruction loss. Put before A-to-D converter. FIR design by windowing.  1 2/   /2 G(  ) 

Nov'04CS3291 Sectn 745 Exercise: Consider effect of reconstructing x a (t) using pulses of same height as in S/H case but only T/2 seconds. wide. 2/T 4/T 3/T 1/T t V T02T3T t h(t) 0 1/T T/2 Impulse response

Nov'04CS3291 Sectn 746 Solution: For new S/H filter:

Nov'04CS3291 Sectn 747 Conclusions from this example Gain now only 0.5 (i.e. -6dB) at  = 0. Gain at  =  /T is 20 log 10 (  2/  ) = -6.9 dB. Roll off now  only 0.9 dB as    /T. Less compensation required. Any disadvantages? Yes. Less gain,  noise has greater effect.

Nov'04CS3291 Sectn Quantisation error: Conversion of samples of x a (t) to binary numbers produces a digital signal. Approximate to nearest quantisation level: V      

Nov'04CS3291 Sectn 749 ‘m-bit’ uniform ADC has 2 m levels  volts apart. Rounding true samples {x[n]} produces: x^[n] = x[n] + e[n] (Quantised) (True) (Error) Normally e[n] lies between -  /2 & +  /2

Nov'04CS3291 Sectn 750 Ideal reconstruction from x^[n] will produce: x a (t) + e a (t) instead of x a (t). where e a (t) arises from {e[n]} & is band-limited to  f S /2 Assumptions about e a (t): i) Samples of e a (t) random & uniformly distributed between  /2. ii) Power spectral density of e a (t) evenly spread in frequency range -f s /2 to f s /2.

Nov'04CS3291 Sectn 751 Probability density of e[n] e[n]     Power spectral density of e a (t)  

Nov'04CS3291 Sectn 752 Power of e a (t) = mean square value of {e[n]} = variance as mean is zero.  /2 = (1/  ) [ e 3 /3] -  /2 =  2 / 12 (quantisation noise power in watts)

Nov'04CS3291 Sectn 753 Signal to quantisation noise ratio (SQNR): To maximise SQNR, signal must be large enough to use all quantisation levels without overflow. Amplification required before A/D conversion.

Nov'04CS3291 Sectn 754 SQNR for sinusoid: Given m-bit ADC with step-size , maximum sine-wave amplitude is 2 m-1 . Power of A sin(  t) : A 2 /2.  maximum possible SQNR is:- = m dB. ( i.e. approx 6 dB per bit)

Nov'04CS3291 Sectn 755 This formula often assumed for signals which are approx. sinusoidal. Example 7.4: (a) How many bits are required to achieve a SQNR of 60 dB with sine-waves amplified to occupy full range of uniformly quantising A/D converter? (b) What SQNR is achievable with a 16-bit uniformly quantising A/D converter applied to sinusoidally shaped signals? Solution: (a) About ten bits. (b) 97.8 dB.

Nov'04CS3291 Sectn Block diag of DSP system for analogue signal processing

Nov'04CS3291 Sectn 757 Antialiasing LPF: Analogue lowpass filter, cut-off < f S /2 to remove any spectral energy which would be aliased into signal band. Analogue S/H: Holds input steady for ADC. A/D conv: Converts voltages to binary numbers of specified word-length. Quantisation error incurred. Samples taken at f S Hz. Digital processor: Controls S/H & ADC to determine f S Reads samples from ADC, processes them & outputs to DAC. D/A conv: Converts binary numbers to analogue voltages. Stair-case waveforms normally produced. S/H compensation: Compensates S/H reconstruction loss of up to 4 dB by boosting spectrum as it approaches f S /2. Reconstruction LPF: Removes images of  fs/2 band produced by S/H reconstruction.

Nov'04CS3291 Sectn Choice of sampling rate To process x a (t) band-limited to  F with F = 20kHz. In theory, we could choose f S = 2F Hz. e.g. 40 kHz. There are two related problems with this choice. (1) Need very sharp analogue anti-aliasing filter to remove everything above F Hz. (2) Need very sharp analogue reconstruction filter to eliminate images (ghosts)

Nov'04CS3291 Sectn 759 |Xs(j2  f)| Hz -F 2F F REMOVE fs/2 - f Slightly over-sampling: Increase f S, to 44.1 kHz when F=20 kHz: |Xs(j2  f)| Hz -F 2F F REMOVE fs/2 - f

Nov'04CS3291 Sectn 760 Analogue filtering now easier. Need only remove everything above f S - F Hz. With F=20kHz, need to filter out everything above 24.1 kHz* without affecting 0 to 20 kHz. * NOT f S /2, it’s a bit better than that.

Nov'04CS3291 Sectn 761 Higher degrees of over-sampling: We wish to digitally process signals bandlimited to  F Hz Instead of taking 2F samples per second, sample at 4F Hz. Anti-aliasing input filter now needs to filter out only components above 3F (not 2F) without distorting 0 to F. Reconstruction simplified as images start at  3F. Images easier to remove without affecting signal in frequency range  F.

Nov'04CS3291 Sectn 762 |Xs(j2 f)| Hz -F 2F F REMOVE fs/2 - -2F 4F - 3F  f f S =4F f f S /2 8F f S =8F 4F -8F

Nov'04CS3291 Sectn 763 f f S /2 8F f S =12F 4F -8F 12F -12F -4F

Nov'04CS3291 Sectn 764 Over-sampling simplifies analogue filters. But what is the effect on the SQNR ? Does SQNR (a) reduce, (b) remain unchanged or (c) increase ? Consider f S =2F

Nov'04CS3291 Sectn 765 As  and m remain unchanged the maximum achievable SQNR Hz is unaffected by this increase in sampling rate. However, quantisation noise power is assumed to be evenly distributed in frequency range  f S /2, & f S has now been doubled. Same amount of quantisation noise power now more thinly spread in frequency range  2F Hz rather than  F Hz.

Nov'04CS3291 Sectn 766 Same noise power, more thinly spread.

Nov'04CS3291 Sectn 767 Increasing f S keeps noise power unchanged, but increases noise bandwidth. As signal bandwidth does not change, keep analogue filter cut- offs unchanged i.e. keep pass-band at  F, not f S /2 now. Assuming quantisation noise evenly spread, this will remove half the noise power. Adds 3 dB to the maximum SQNR.  over-sampling can increase the SQNR. Four-times & 256-time oversampling used.

Nov'04CS3291 Sectn 768 Advantages of oversampling: Simpler antialiasing and reconstruction filters. Reduction in SQNR of 3dB per doubling of f S. Reduces S/H reconstructn roll-off effect. Disadvantages: Increases no. of bits per second (ADC word-length x f S ), Increases cost of processing, storage/transmission Faster ADC needed.

Nov'04CS3291 Sectn 769 Effect of increasing m rather than f S 3 dB in SQNR gained by doubling f S. Doubles the bit-rate. Doubling number of ADC bits, m, also doubles the bit-rate. Increases the max SQNR by 6m dB i.e. 48 dB if 8-bit ADC replaced by 16-bit ADC. Example: with F = 5 kHz:- m f S Max SQNRbit-rate 1010 kHz60 dB100 k kHz63 dB200 k 1210 kHz72 dB120 k Conclusion: Oversampling simplifies analogue electronics, but is less economical with digital processing, storage & transmission resources.

Nov'04CS3291 Sectn 770 Digital anti-aliasing & reconstruction filters Can get the best of both worlds. Sample at higher rate, 4F say, with analogue filter to remove components beyond  3F. Digital filter to remove components beyond  F. Down-sample (decimate) to reduce f S to 2F. Simply omit alternate samples. To reconstruct: Up-sample by placing zeros between each sample: e.g. { …, 1, 2, 3, 4, …} with f S = 10 kHz becomes {…, 1, 0, 2, 0, 3, 0, 4, 0, …} at 20 kHz. Creates images (ghosts) in the digital signal. Removed by digital filter. Then reconstruct as normal but at higher f S.

Nov'04CS3291 Sectn 771 CD format: 20 kHz bandwidth stored at 44.1 kHz. Player up-samples by inserting zeros & digitally filtering. Four, 8 or 16 times over-sampling was used. Bit-stream: up- (over-) samples to such a degree (256) that a one bit ADC is all that is required. Produces high quantisation noise, but very thinly spread in frequency-domain. Most of it filtered off by very simple analogue filter. 256 times over-sampling gain only 3 x 8 dB.  some more tricks needed: noise-shaping.

Nov'04CS3291 Sectn 772 Example: A DSP system for processing sinusoidal signals in the range 0 Hz to 4 kHz samples at 20 kHz with an 8-bit ADC. If the input signal is always amplified to use the full dynamic range of the ADC, estimate the SQNR in the range 0 to 4 kHz. How would the SQNR be affected by decreasing f S to 10 kHz and replacing the 8-bit ADC by a 10-bit device? Are there any disadvantages in doing this?