1. ADVANCED SIGNAL PROCESSING TECHNIQUES FOR WIRELESS COMMUNICATIONS Erdal Panayırcı Electronics Engineering Department IŞIK University

2. OUTLINE Introduction Knowledge Gaps in General The essential of EM algorithm The Sage algorithm Some Application Areas Sequential Monte Carlo Method (SMC) Knowledge Gaps in SMC

3. INTRODUCTION Future generation wireless commun. systems are confronted with new challenges mainly due to Hostile channel characteristics Limited bandwidth Very high data rates

4. Advanced Signal Proc. techniques such as The Expectation-Maximization algorithm The SAGE algorithm The Baum-Welch algorithm Sequential Monte Carlo Techniques Kalman filters and their extensions Hidden Markov modeling Stochastic approximation algorithms

5. In collaboration with Inexpensive and Rapid computational power provide powerful tools to overcome the limitations of current technologies.

6. Applications of advanced signal processing algorithms, include, but are not limited to Joint/Blind/Adaptive Sequence (data) detection Frequency, Phase,timing synchronization Equalization Channel Estimation techniques.

7. These techniques are employed in advanced wireless communication systems such as OFDM/OFDMA CDMA MIMO, Space-time-frequency Coding Multi-User detection

8. Especially, development of the suitable algorithms for wireless multiple access systems in Non-stationary Interference-rich environments presents major challenges to us.

9. Optimal solutions to these problems mostly can not be implemented in practice mainly due to high computational complexity

10. Advanced signal processing tools, I mentioned before, have provided a promising route for the design of low complexity algorithms with performances approaching the theoretical optimum for Fast, and Reliable communication in highly severe and dynamic wireless environment

11. Over the past decade, such methods have been successfully applied in several communication problems. But many technical challenges remain in emerging applications whose solutions will provide the bridge between the theoretical potential of such techniques and their practical utility.

12. The Key Knowledge Gaps Theoretical performance and convergence analysis of these Algorithms Some new efficient algorithms need to be worked out and developed for some of the problems mentioned above Computational complexity problems of these algorithms when applied to on-line implementations of some algorithms running in the digital receivers must be handled.

13. Implementation of these algorithms based on batch processing and sequential (adaptive) processing depending on how the data are processed and the inference is made has not been completely solved for some of the techniques mentioned above.

14. Some class of algorithms requires efficient generation of random samples from an arbitrary target probability distribution, known up to a normalizing constant. So far two basic types of algorithms, Metropolis algorithm and Gibbs sampler have been widely used in diverse fields. But it is known that they are substantially complex and difficult to apply for on-line applications like wireless communications. There are gaps for devising new types of more efficient algorithms that can be effectively employed in wireless applications.

15. THE EM ALGORITHM The EM algorithm was popularized in 1977 An iterative “algorithm” for obtaining ML parameter estimates Not really an algorithm, but a procedure Same problem has different EM formulations Based on definition of complete and incomplete data

16. L. E. Baum, T. Petrie, G. Soules and N. Weiss, A Maximization Technique in Statistical Estimation for Probabilistic Functions of Markov Chains, Annals of Mathematical Statistics, pp. 164-171, 970. A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum-Likelihood from Incomplete Data Via the EM Algorithm, Journal, Royal Statistical Society, Vol. 39, pp. 1-17, 1977. C. F. Wu, On the Convergence Properties of the EM Algorithm, Annals of Statistics, Vol. 11, pp. 95-103, 1983. Main References

17. The Essential EM Algorithm Consider estimating parameter vector s from data y (“incomplete” data): Parameters to be estimatedRandom parameters Then, the ML estimate of s is:

18. Thus, obtaining ML estimates may require: An Expectation Often analytically intractable A Maximization Computationally intensive

19. The EM Iteration Define the complete data x Many-to-one mapping having conditional density The EM iteration at the i-th step: E-step: M-step:

20. Convergence Properties At each iteration the likelihood-function is monotonically non-decreasing If the likelihood-function is bounded, then the algorithm converges Under some conditions, the limit point coincides with the ML estimate

21. EM Algorithm Extensions J. A. Fessler and A. O. Hero, Complete-data spaces and generalized EM algorithms, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-93), Vol. 4, pp. 1-4, 1993. J. A. Fessler and A. O. Hero, Space-alternating generalized EM algorithm, IEEE Transactions and Signal Processing, October 1994.

22. The SAGE Algorithm The SAGE algorithm is an extension of EM algorithm It provides much faster convergence than EM Algorithm alternates several hidden data spaces rather than just using one complete data space, and Updates only a subset of elements of the parameters in each itteration

23. Some Application Areas Positron-Emission-Tomography (PET) Genetics Neural Networks Radar Imaging Image / Speech processing Communications Channel Estimation / Equalization Multiuser detection Squence estimation Interference rejection

24. SEQUENTIAL MONTE CARLO TECHNIQUE (SMC) Emerged in the field of statistics, J. S. Liu and R. Chen, “Sequential Monte Carlo Methods for Dynamics Systems”, J. American Stat. Assoc., Vol. 93, pp. 1032- 1044, 1998.

25. Recently, SMC has been successfully applied to several problems in wireless communications, such as, Blind equalization Detection/decoding in fading channels It is basically based on approximating the expectation operation by means of sequentially generated Monte Carlo samples from either unknow state variables or system parameters.

26. Main Advantages SMC is self adaptive and no training/pilot symbols or decision feedback are needed Tracking of fading channels and the estimation of the data sequence are naturally in integrated Channel noise can be either Gaussian on Non-Gaussian It is suitable for MAP receiver design

27. If the system employs channel coding, the coded signal structure can be easily exploited to improve the accuracy of both channel and data estimation SMC is suitable for high-speed parallel implementation using VLSI Does not require iterations like in EM algorithm Updating with new data can be done more efficiently

28. SMC Method Let denote the parameter vector of interest Let denote the complete data so that is assumed to be simple is partially observed It can be partitioned as where denotes the observed part denotes in the incomplete data or unobservable or missing data.

29. Example 1. Fading channel Problem: Joinly estimate the data signal and the unknown channel parameters

30. 2. Joint Phase Offset and SNR Estimation is unknown phase offset is unknown noise variance is the data to be transmitted

31. Problem: Estimate based on complete datawhere observed part incomplete data

32. MAP SOLUTION USING SMC METHOD MAP solution of the unknown parameter vector  is Where p(  |Y t ) can be computed by means of incomplete data sequence as

33. Substituting this in the above, we have

34. To implement SMS, we need to draw m independent samples (Monte Carlo samples) from the conditional distribution of Usually, directly drawing samples from this distribution is difficult. But, drawing samples from some trial- distribution is easy.

35. In this case, we can use the idea of importance sampling as follows: Suppose a set of samples is drawn from the trial distributionBy associating weight to the sample

36. where, The pair is called a properly weighted sample. w. r. t. distribution We can now estimateas follows;

37. By properly choosing the trial distribution q(.), the weighted samples can be generated sequentially. That is, suppose a set of properly weighted samples Then SMC algorithm generates from this set, a new one at time t. at time t-1 is given.

38. 1. Draw samplesfrom the trial distribution q(.) and let 2.Compute the important weightfrom sequentially. 3.Compute the MAP estimate As a summary SMC algorithm is given as follows for j = 1, 2,..., m

39. KNOWLEDGE GAPS IN SMC Coosing the effective sample size m (empirically usually, 20 < m < 100 ). The sampling weights measures the “quality” of the corresponding drawn data sequence Small weights implies that these samples do not really represent the distribution from which they are drawn and have small contribution in the final estimation Resampling procedure was developed for it. It needs to be improved for differential applications

40. Delay Estimation Problem: Since the fading process is highly correlated, the future received signals contain information about current data and channel state. A delay estimate seems to be more efficient and promising than the present estimate summarized above.

41. In delay estimation: Instead of making inference on (S t,  ) with posterior density p( , S t |Y t ), we delay this inference to a later time (t+  ) with the distribution p( , S t |Y t+  )

42. Note: Such a delay estimation method does not increase computational cost but it requires some extra memory. Knowledge Gap: Develop computationally efficient delayed-sample estimation techniques which will find applications in channel with strong memory (ISI channel).

43. Turbo Coding Applications Because, SMC is soft-input and soft-output in nature, the resulting algorithms is capable of exchanging extrinsic information with the MAP outher channel decoder and sucessively improving the overall receiver performance. Therefore blind MAP decoder in turbo receivers can be worked out.