2. Postacademic Course on Telecommunications 20/4/00 p. 1 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Lecture-2: Limits of Communication Problem Statement: Given a communication channel (bandwidth B), and an amount of transmit power, what is the maximum achievable transmission bit-rate (bits/sec), for which the bit-error-rate is (can be) sufficiently (infinitely) small ? - Shannon theory (1948) - Recent topic: MIMO-transmission (e.g. V-BLAST 1998, see also Lecture-1)

3. Postacademic Course on Telecommunications 20/4/00 p. 2 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Overview `Just enough information about entropy’ (Lee & Messerschmitt 1994) self-information, entropy, mutual information,… Channel Capacity (frequency-flat channel) Channel Capacity (frequency-selective channel) example: multicarrier transmission MIMO Channel Capacity example: wireless MIMO

4. Postacademic Course on Telecommunications 20/4/00 p. 3 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA `Just enough information about entropy’(I) Consider a random variable X with sample space (`alphabet’) Self-information in an outcome is defined as where is probability for (Hartley 1928) `rare events (low probability) carry more information than common events’ `self-information is the amount of uncertainty removed after observing.’

5. Postacademic Course on Telecommunications 20/4/00 p. 4 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA `Just enough information about entropy’(II) Consider a random variable X with sample space (`alphabet’) Average information or entropy in X is defined as because of the log, information is measured in bits

6. Postacademic Course on Telecommunications 20/4/00 p. 5 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA `Just enough information about entropy’ (III) Example: sample space (`alphabet’) is {0,1} with entropy=1 bit if q=1/2 (`equiprobable symbols’) entropy=0 bit if q=0 or q=1 (`no info in certain events’) q H(X) 1 10

7. Postacademic Course on Telecommunications 20/4/00 p. 6 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA `Just enough information about entropy’ (IV) `Bits’ being a measure for entropy is slightly confusing (e.g. H(X)=0.456 bits??), but the theory leads to results, agreeing with our intuition (and with a `bit’ again being something that is either a `0’ or a `1’), and a spectacular theorem Example: alphabet with M=2^n equiprobable symbols : -> entropy = n bits i.e. every symbol carries n bits

8. Postacademic Course on Telecommunications 20/4/00 p. 7 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA `Just enough information about entropy’ (V) Consider a second random variable Y with sample space (`alphabet’) Y is viewed as a `channel output’, when X is the `channel input’. Observing Y, tells something about X: is the probability for after observing

9. Postacademic Course on Telecommunications 20/4/00 p. 8 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA `Just enough information about entropy’ (VI) Example-1 : Example-2 : (infinitely large alphabet size for Y) + noise decision device XY 00 01 10 11 + noise XY 00 01 10 11 00 01 10 11

10. Postacademic Course on Telecommunications 20/4/00 p. 9 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA `Just enough information about entropy’(VII) Average-information or entropy in X is defined as Conditional entropy in X is defined as Conditional entropy is a measure of the average uncertainty about the channel input X after observing the output Y

11. Postacademic Course on Telecommunications 20/4/00 p. 10 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA `Just enough information about entropy’(VIII) Average information or entropy in X is defined as Conditional entropy in X is defined as Average mutual information is defined as I(X|Y) is uncertainty about X that is removed by observing Y

12. Postacademic Course on Telecommunications 20/4/00 p. 11 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (I) Average mutual information is defined by -the channel, i.e. transition probabilities -but also by the input probabilities Channel capacity (`per symbol’ or `per channel use’) is defined as the maximum I(X|Y) for all possible choices of A remarkably simple result: For a real-valued additive Gaussian noise channel, and infinitely large alphabet for X (and Y), channel capacity is signal (noise) variances

13. Postacademic Course on Telecommunications 20/4/00 p. 12 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (II) A remarkable theorem (Shannon 1948): With R channel uses per second, and channel capacity C, a bit stream with bit-rate C*R (=capacity in bits/sec) can be transmitted with arbitrarily low probability of error = Upper bound for system performance !

14. Postacademic Course on Telecommunications 20/4/00 p. 13 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (II) For a real-valued additive Gaussian noise channel, and infinitely large alphabet for X (and Y), the channel capacity is For a complex-valued additive Gaussian noise channel, and infinitely large alphabet for X (and Y), the channel capacity is

15. Postacademic Course on Telecommunications 20/4/00 p. 14 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (III) Information I(X|Y) conveyed by a real-valued channel with additive white Gaussian noise, for different input alphabets, with all symbols in the alphabet equally likely (Ungerboeck 1982)

16. Postacademic Course on Telecommunications 20/4/00 p. 15 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (IV) Information I(X|Y) conveyed by a complex-valued channel with additive white Gaussian noise, for different input alphabets, with all symbols in the alphabet equally likely (Ungerboeck 1982)

17. Postacademic Course on Telecommunications 20/4/00 p. 16 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (V) This shows that, as long as the alphabet is sufficiently large, there is no significant loss in capacity by choosing a discrete input alphabet, hence justifies the usage of such alphabets ! The higher the SNR, the larger the required alphabet to approximate channel capacity

18. Postacademic Course on Telecommunications 20/4/00 p. 17 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (frequency-flat channels) Up till now we considered capacity `per symbol’ or `per channel use’ A continuous-time channel with bandwidth B (Hz) allows 2B (per second) channel uses (*), i.e. 2B symbols being transmitted per second, hence capacity is (*) This is Nyquist criterion `upside-down’ (see also Lecture-3) received signal (noise) power

19. Postacademic Course on Telecommunications 20/4/00 p. 18 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (frequency-flat channels) Example: AWGN baseband channel (additive white Gaussian noise channel) n(t) + channel s(t) r(t)=Ho.s(t)+n(t) Ho f H(f) B -B Ho here

20. Postacademic Course on Telecommunications 20/4/00 p. 19 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (frequency-flat channels) Example: AWGN passband channel passband channel with bandwidth B accommodates complex baseband signal with bandwidth B/2 (see Lecture-3) n(t) + channel s(t) r(t)=Ho.s(t)+n(t) Ho f H(f) x Ho x+B

21. Postacademic Course on Telecommunications 20/4/00 p. 20 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (frequency-selective channels) n(t) + channel s(t) R(f)=H(f).S(f)+N(f) H(f) Example: frequency-selective AWGN-channel received SNR is frequency-dependent! f H(f) B-B

22. Postacademic Course on Telecommunications 20/4/00 p. 21 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (frequency-selective channels) Divide bandwidth into small bins of width df, such that H(f) is approx. constant over df Capacity is optimal transmit power spectrum? f H(f) B-B 0 B

23. Postacademic Course on Telecommunications 20/4/00 p. 22 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (frequency-selective channels) Maximize subject to solution is `Water-pouring spectrum’ Available Power B L area

24. Postacademic Course on Telecommunications 20/4/00 p. 23 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Channel Capacity (frequency-selective channels) Example : multicarrier modulation available bandwidth is split up into different `tones’, every tone has a QAM-modulated carrier (modulation/demodulation by means of IFFT/FFT). In ADSL, e.g., every tone is given (+/-) the same power, such that an upper bound for capacity is (white noise case) (see Lecture-7/8)

25. Postacademic Course on Telecommunications 20/4/00 p. 24 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (I) SISO =`single-input/single output’ MIMO=`multiple-inputs/multiple-outputs’ Question: we usually think of channels with one transmitter and one receiver. Could there be any advantage in using multiple transmitters and/or receivers (e.g. multiple transmit/receive antennas in a wireless setting) ??? Answer: You bet..

26. Postacademic Course on Telecommunications 20/4/00 p. 25 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (II) 2-input/2-output example A B C D + + X1 X2Y2 Y1 N1 N2

27. Postacademic Course on Telecommunications 20/4/00 p. 26 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (III) Rules of the game: P transmitters means that the same total power is distributed over the available transmitters (no cheating) Q receivers means every receive signal is corrupted by the same amount of noise (no cheating) Noises on different receivers are often assumed to be uncorrelated (`spatially white’), for simplicity

28. Postacademic Course on Telecommunications 20/4/00 p. 27 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (IV) 2-in/2-out example, frequency-flat channels Ho 0 0 + + X1 X2Y2 Y1 N1 N2 first example/attempt

29. Postacademic Course on Telecommunications 20/4/00 p. 28 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (V) 2-in/2-out example, frequency-flat channels corresponds to two separate channels, each with input power and additive noise total capacity is room for improvement...

30. Postacademic Course on Telecommunications 20/4/00 p. 29 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (VI) 2-in/2-out example, frequency-flat channels Ho -Ho Ho + + X1 X2Y2 Y1 N1 N2 second example/attempt

31. Postacademic Course on Telecommunications 20/4/00 p. 30 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (VII) A little linear algebra….. Matrix V’

32. Postacademic Course on Telecommunications 20/4/00 p. 31 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (VIII) A little linear algebra…. (continued) Matrix V is `orthogonal’ (V’.V=I) which means that it represents a transformation that conserves energy/power Use as a transmitter pre-transformation then (use V’.V=I)... Dig up your linear algebra course notes...

33. Postacademic Course on Telecommunications 20/4/00 p. 32 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (IX) Then… + + Y2 Y1 N1 N2 + + X^1 X^2 X2 X1 transmitter A B C D V11 V12 V21 V22 channelreceiver

34. Postacademic Course on Telecommunications 20/4/00 p. 33 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (X) corresponds to two separate channels, each with input power, output power and additive noise total capacity is 2x SISO-capacity!

35. Postacademic Course on Telecommunications 20/4/00 p. 34 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (XI) Conclusion: in general, with P transmitters and P receivers, capacity can be increased with a factor up to P (!) But: have to be `lucky’ with the channel (cfr. the two `attempts/examples’) Example : V-BLAST (Lucent 1998) up to 40 bits/sec/Hz in a `rich scattering environment’ (reflectors, …)

36. Postacademic Course on Telecommunications 20/4/00 p. 35 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (XII) General I/O-model is : every H may be decomposed into this is called a `singular value decompostion’, and works for every matrix (check your MatLab manuals) diagonal matrix orthogonal matrix V’.V=I orthogonal matrix U’.U=I

37. Postacademic Course on Telecommunications 20/4/00 p. 36 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (XIII) With H=U.S.V’, V is used as transmitter pre-tranformation (preserves transmit energy) and U’ is used as a receiver transformation (preserves noise energy on every channel) S=diagonal matrix, represents resulting, effectively `decoupled’ (SISO) channels Overall capacity is sum of SISO-capacities Power allocation over SISO-channels (and as a function of frequency) : water pouring

38. Postacademic Course on Telecommunications 20/4/00 p. 37 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA MIMO Channel Capacity (XIV) Reference: G.G. Rayleigh & J.M. Cioffi `Spatio-temporal coding for wireless communications’ IEEE Trans. On Communications, March 1998

39. Postacademic Course on Telecommunications 20/4/00 p. 38 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Assignment 1 (I) 1. Self-study material Dig up your favorite (?) signal processing textbook & refresh your knowledge on -discrete-time & continuous time signals & systems -signal transforms (s- and z-transforms, Fourier) -convolution, correlation -digital filters...will need this in next lectures

40. Postacademic Course on Telecommunications 20/4/00 p. 39 Module-3 Transmission Marc Moonen Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA Assignment 1 (II) 2. Exercise (MIMO channel capacity) Investigate channel capacity for… -SIMO-system with 1 transmitter, Q receivers -MISO-system with P transmitters, 1 receiver -MIMO-system with P transmitters, Q receivers P=Q (see Lecture 2) P>Q P<Q