1. Signals ---- 11 CY2G2/SE2A2 Information Theory and Signals Aims: To discuss further concepts in information theory and to introduce signal theory. Outcomes: An ability to quantify information transfer in noisy channels, continuous sources and communication systems. An understanding of coding to minimise errors and of fundamental concepts in signal theory. Book: M.J.Usher &C.G.Guy, “Information and Communication for Eengineers’’ Macmillan (Strongly recommended) Information Theory : Review of fundamental information theory, Matching source to channel, Information in noisy channels, Coding in noisy channels, Shannon’s second theorem, Coding methods, Information in continuous sources, Idea communication theorem, Implications and applications.

2. Signals ---- 12 CY2G2/SE2A2 Information Theory and Signals Signal Theory : Random noise and its properties, Introduction to signal theory, time domain properties, frequency domain representations, Autocorrelation, Cross correlation, convolution and their properties, Fourier series, application to simple waveforms, complex form and application to pulse train, deductions and implications. Theory and properties of Fourier Transforms, and their applications including autocorrelation, power spectrum, convolution and linear systems, and sampling theory. Course work: Laboratory practicals 400 Electrical noise 417 Binary codes 418 Correlations and Convolution 419 Fourier Series Examination: CY2G2: Four questions in one section --- answer at least three. Please SE2A2 module description for division of exam questions.

3. Signals ---- 13 Noise Random noise occurs in any practical information transmission system. Its value at any instant is unpredictable. Received signal = Actual signal + Noise Random noise is an unwanted signal, producing errors or changes in amplitude of the actual signal (wanted), and reducing the overall information transfer.

4. Signals ---- 14 Most signal would have some kind of pattern, or regularity, or the plot of signal is smooth, e. g. sine wave. noise is unpredictable in value, as next time step could take any value in its range. The plot is not smooth. Received signal = Actual signal + Noise This results in signal in a mixed form. The predictability depends on the S/N ratio, the ratio Var(S)/Var(N). Random noise is an unwanted signal, producing errors or changes in amplitude of the actual signal (wanted), and reducing the overall information transfer. The noise level is represented by its variance or standard deviation. Given a signal, a noise with higher variance would corrupt the received signal to a higher extend than a noise with a lower variance.

5. Signals ---- 15 P(x) x x

6. Signals ---- 16 Properties of noise (i) Time domain The essential feature of electrical noise is it is unpredictable in the time domain. Its amplitude follows a Gaussian distribution.

7. Signals ---- 17 Frequency domain is a commonly used method of signal processing. Frequency response is used to describe a systems characteristics using its response to sinusoidal signal. If a sine wave is fed into a system (input), the output will also be a sine wave, but with different amplitude and usually have a phase shift. By changing the frequency of input signal, the system can be shown as how its amplitude and phase changes accordingly, which then defines a system. Frequency response: A power function (corresponds to amplitude) versus frequency plot is used. For instance a sinusoidal signal with a fixed frequency would be plotted as a peak at a certain frequency. A sinusoidal signal is totally predictable. 30Hz Volts 5 frequency V(t)= 5sin(60π t)

8. Signals ---- 18 (ii) Frequency domain Unpredictability in the time domain  Flatness in the frequency domain. (No peak means no periodicity, which means predictable) Noise can be compared to white light which has a flat frequency plot in spectrum.

9. Signals ---- 19 (iii) Representation Represent each slice by a sinusoidal oscillator with frequency Equal to that of centre of slice: We have infinite number of sinewave generators with same amplitude a i, random phase, Φi. So, the noise wave form is given by

10. Signals ---- 110 (iii) Addition of random generators

11. Signals ---- 111 Instantaneous sum Mean squares value Since two wave forms are independent, So the mean squares of the two noise adds up as the mean squares of v(t)

12. Signals ---- 112 Note that for signal of same frequency and phase, as shown below The root mean squares add as The root mean squares of s(t)

13. Signals ---- 113 Example:

14. Signals ---- 114 Homework: In Matlab: >> a=3*randn(1000,1); % generate a random Gaussian series of 1000 points, with standard deviation 3. >> b=4*randn(1000,1); % generate a random Gaussian series of 1000 points, with standard deviation 4. >> c=a+b; % generate a random Gaussian series of 1000 points as the sum of the above 2 series.

15. Signals ---- 115 >> sigma_a=std(a); % find the standard deviation of a. >> sigma_b=std(b); % find the standard deviation of b. >> sigma_c=std(c); % find the standard deviation of c (the sum of a and b). >> plot(a); % show you the random series as a figure. >> sigma_a % show you the standard deviation of a, should approximately be 3. >> sigma_b % show you the standard deviation of b, should approximately be 4. >> sigma_c % show you the standard deviation of c, (note that c should approximately be 5=, but not 7).