1. Random Processes and LSI Systems What happedns when a random signal is processed by an LSI system? This is illustrated below, where x(n) and y(n) are random signals, and h(n) is a deterministic (i.e., nonrandom) LSI system. The input, x(n) is a random signal, so y(n) is, too. Random in, random out. x(n) y(n)h(n)

2. Random Processes and LSI Systems The LSI system, h(n) does exactly the same thing as it would if h(n) were deterministic, so: x(n) y(n)h(n) or, in the frequency domain,

3. Random Processes and LSI Systems We can take the square of the magnitude, Recall (from the previous lecture) that S xx (  ), the spectral density of x(n), is related to by

4. Random Processes and LSI Systems so obviously and this can be rewritten as:

5. so obviously and this can be rewritten as: Random Processes and LSI Systems Since the LSI system is deterministic, we can take it outside the expected value:

6. Random Processes and LSI Systems What’s left inside the expected value is S xx (  ), so isn’t that nice?

7. Random Processes and LSI Systems It can be shown that, if x(n) is a zero-mean sequence (which it is) and h(n) is an LSI system (which it is), then y(n) is also zero mean. This means we can find the variance of y (the average output power by: This means we can reduce the average power of a random signal (i.e., reduce the noise power) by attenuating parts of its spectral density function.

8. Random Processes and LSI Systems The above equation shows that we can attenuate parts of the output noise spectral density by making the system frequency response such that the system rejects those parts. The power of a wideband noise source can be reduced by lowpass (or bandpass) filtering the noise.

9. Random Processes and LSI Systems -- S xx (  ) Suppose x(n) is a white noise sequence as shown. It’s average power is:

10. Random Processes and LSI Systems -- H(  ) Now, suppose h(n) is an ideal lowpass filter,  c =  /4 - 

11. Random Processes and LSI Systems -- So now the output spectral density is this: -  S yy (  ) S xx (  ) And the average power of the output sequence is:

12. Random Processes and LSI Systems So reducing the bandwidth by 75% also reduced the noise power by 75%.

13. Suppose the input to our LSI system h(n) is the sum of two random sequences, z(n) and g(n): Random Processes and LSI Systems x(n) y(n)h(n)  z(n) g(n) If we know S zz (n) and S gg (n), can we find S yy (n)? Yes!

14. We already know that Random Processes and LSI Systems So we need to find S xx (n) in terms of S zz (n) and S gg (n). recall that: and

15. Random Processes and LSI Systems so

16. Random Processes and LSI Systems Two random processes are independent if the outcome of one does not influence the other. For example, rolling two dice. Two random processes are dependent if the outcome of one can influence the other. Example: drawing a 5- card poker hand.

17. Random Processes and LSI Systems If the two processes that produce z(n) and g(n) are independent, we can simplify the last expression: and if they’re zero-mean processes, this further reduces to:

18. Random Processes and LSI Systems and this means that the spectral density of x(n) is So, for two sequences z(n) and g(n), which are generated by independent, zero-mean random processes, summed together to form the input sequence x(n) to an LSI system h(n), the output noise spectral density is

19. Random Processes and LSI Systems Finally, the average power of the output sequence y(n) is:

20. White Noise A zero-mean white noise sequence x(n) has the following proprerties: E[x(n)] = 0 x(k) and x(k+n) are independent if n is nonzero. White noise is independent with respect to any other sequence.

21. White Noise If two random variables x and y are independent, So, for white noise, But E[x(n)] = 0, so

22. White Noise We’ve previously seen that So we’ve just shown that for white noise, The power spectral density is the DTFT of the autocorrelation function, so for zero-mean white noise, which is constant across the entire spectrum (white).

23. We can generate a zero-mean white noise sequence x(n) by randomly choosing, for each n, a real number between –  /2 and  /2. Each number must be independent of the choice of all others. This is a uniformly distributed, zero mean, white noise signal. White Noise

24. Quantization Noise and Oversampling Here’s a VERY practical example of exactly this type of signal: quantization noise. Suppose we have an N-bit A/D converter with an input we designate x a (t). x a (t) can range from a minimum of -V volts to a maximum of +V volts. The A/D converter samples x a (t) every T seconds, and produces an N-bit binary number which approximates the quantized value of the sample x a (nT)

25. Quantization Noise and Oversampling An N-bit binary number is used to represent the sampled input, x a (nT). This binary number has a finite number of possible values: 2 N. Each of these values represents a range of possible input voltages, and there is one such range for each possible N-bit number. Thus, the total input range, is divided into 2N smaller ranges.

26. Quantization Noise and Oversampling Each of these smaller ranges can be expressed as where M is the actual N-bit number representing a particular sample. The following figure illustrates these relationships for a 4-bit A/D converter (N = 4).

27. Quantization Noise and Oversampling xaxa M M=0 M=1 4-bit A/D converter

28. Quantization Noise and Oversampling If the output of the A/D converter is M, x a (nT) is somwhere in the range So, if the output of the A/D converter is M, this says that the input is

29. Quantization Noise and Oversampling If the output of the A/D converter is M, x a (nT) is somwhere in the range So, if the output of the A/D converter is M, this says that the input is

30. Quantization Noise and Oversampling If we let the A/D converter output be the center of the range, we can rewrite the previous relationship as

31. Quantization Noise and Oversampling or where x(n) is the actual input signal to the A/D converter, and the A/D output is We can express the quantization error as an error (or noise) sequence, e(n), yielding this:

32. Quantization Noise and Oversampling Written this way, The A/D converter’s output can be thought of as the sum of two sequences: a signal sequence equal to the input signal sampled by an ideal, infinite-precision A/D converter, and the quantization noise sequence, e(n). Note that the quantization noise can have any value between –  /2 and  /2, and its probability density function is uniform.

33. Quantization Noise and Oversampling Consider the signal sequence, x(n). It’s average power is The quantization noise sequence has this average power: so the A/D converter’s signal to noise ratio (SNR) is:

34. Quantization Noise and Oversampling We can write the SNR in terms of decibels: Naturally, we want the SNR to be as large as possible. This means we make the signal power as large as possible, by using the entire input range of the A/D converter. We also minimize the noise power. One way to do this is to increase the number of bits, which may or may not be practical. We would like to have another way.

35. Quantization Noise and Oversampling To see if there is another way, let’s investigate e(n). We know that e(n) is a random variable with values in the range and it’s uniformly distributed in that range. Positive and negative values are equally likely, so it’s a zero- mean process.

36. Quantization Noise and Oversampling It’s average power is:

37. Quantization Noise and Oversampling With this knowledge, we can write the SNR as Or, in dB:

38. Quantization Noise and Oversampling If N > 8, we can use this approximation: substituting this in the previous expression for SNR gives us:

39. Quantization Noise and Oversampling Which shows that each additional bit of precision improves the SNR by about 6 dB. This makes sense, since it cuts  in half. It’s not too much of a stretch to assume that quantization errors occurring at different times are independent. If this is assumed, then the quantization noise sequence, e(n), satisfies the conditions to be modeled as uniformly distributed white noise.

40. Quantization Noise and Oversampling -- S ee (  ) e2e2 We’ve already seen that the power spectral density of a white noise sequence x(n) is so the power spectral density of e(n) is as shown below:

41. Quantization Noise and Oversampling So the average power of the quantization noise sequence e(n) is spread over the discrete time frequency range from –  to . Remember that  radians per sample in the discrete time domain maps to f s /2 in the continuous time domain, so the quantizing noise power is spread over the range – f s /2 to f s /2. - f s /2 S ee (  fT ) e2e2

42. Quantization Noise and Oversampling If, for example, we quadruple the sampling frequency, but leave N alone, we take the quantizing noise power and spread it over a band four times as wide. Suppose we have a bandlimited signal (bandwidth = B) with power spectral density shown in the next slide:

43. If we sample it at the minimum sample rate, f s = 2B, the spectral densities of the signal sequence and the quantization noise sequence are as shown next: S xx ( f )  Quantization Noise and Oversampling f 

44. If we double the sample rate (f s = 4B), the signal and noise spectral densities are as shown: S xx ( f )  Quantization Noise and Oversampling f  ee

45. If we double the sample rate again (f s = 8B), the signal and noise spectral densities are as shown: S xx ( f )  Quantization Noise and Oversampling f  ee 

46. Note that the true signal power and the true noise power are the same in all three, the apparent difference merely serves to show that the noise power is spread over a wider band. S xx ( f )  Quantization Noise and Oversampling f  ee 

47. This can be generalized for f s = 2MB, as shown below: S xx ( f )  Quantization Noise and Oversampling f  ee  Notice that if M > 1 (i.e., if the signal is oversampled) some of the noise power is outside the signal bandwidth.

48. We can eliminate the portion of the noise power which is outside the signal bandwidth by using a digital lowpass filter to attenuate it. Since this filter will not attenuate anything in the signal band, the signal (and the information it conveys) is unaffected. Unfortunately, the in-band noise is also not affected, but getting rid of the out-of-band noise is a good thing. Quantization Noise and Oversampling

49. If M, the oversampling factor, is 1, we are sampling at the minimum rate, and the SNR is given by Quantization Noise and Oversampling If we let M = 2 (oversampling by a factor of 2), and use an ideal lowpass filter to eliminate out-of-band noise, the result is as shown next:

50. Quantization Noise and Oversampling S xx ( f )  f  ee  This eliminates half the noise power, doubling the signal to noise ratio. If we let M = 4 (oversampling by 4), the SNR is doubled again:

51. Quantization Noise and Oversampling S xx ( f )  f  ee  In general, every time we double the sample rate, we can double the SNR. This is the same as increasing the SNR by 3 dB for every doubling of the sample rate.

52. Quantization Noise and Oversampling The resolution of an A/D converter is the effective number of bits which, with M=1, would yield it’s signal to noise ratio. The SNR of an A/D converter (with M = 1) is given by: so oversampling by 4 effectively adds a bit to the converter’s resolution. If a digital filter eliminates the out-of-band quantization noise, that is.

53. Our oversampling system is as shown below: Quantization Noise and Oversampling x a (t) H(e j  ) A/D f s = 2MB Lowpass  c =  /M The filter output, g(n), can be thought of as the sum of two components:

54. The filter doesn’t change the signal, but it does affect the noise by eliminating the out-of-band portion. Notice that the noise at the filter input is denoted e(n), while the output noise is Quantization Noise and Oversampling Since the signal and noise components of the filter input were assumed to be independent, the spectral density of the filter output is given by:

55. The signal power is still given by Quantization Noise and Oversampling The output noise power is

56. Quantization Noise and Oversampling So the output SNR is: So oversampling by a factor of M multiplies the output SNR by M. In terms of dB,

57. Quantization Noise and Oversampling This shows that if M = 2, we add 3 dB to the SNR, and if M = 4, we add 6 dB. 6 dB is also the SNR improvement we get by adding one bit of resolution. In other words, quadrupling the sample rate is equivalent to adding one bit to the A/D converter. If we call the converter’s resolution in bits R, the relationship between R, N and M is:

58. Quantization Noise and Oversampling If we have an N – bit converter, and need R bits of resolution, we can solve for M: The dynamic range of an A/D converter is the SNR we get when the input is a full-scale sinusoid:

59. The power of this signal is V 2 /2. by subsituting this for  x 2 into Quantization Noise and Oversampling we get This shows that improving the resolution by 1 bit also improves the dynamic range by 6 dB.

60. The dynamic range of an oversampling A/D converter (and filter) is given by: Quantization Noise and Oversampling