Efficient Statistical Pruning for Maximum Likelihood Decoding Radhika Gowaikar Babak Hassibi California Institute of Technology July 3, 2003

Outline Integer Least Squares Problem Probabilistic Setup, Complexity as Random Variable Sphere Decoder Modified Algorithm Statistical Pruning, Expected Complexity Results Analysis Conclusions and Future Work

Integer-Least Squares Problems Search space is discrete, perhaps infinite Given a “skewed” lattice Given a vector Find “closest” lattice point Known to be NP-hard

Applications in ML Decoding ML detection leads to integer least-squares problems Signal constellation is a subset of a lattice (PAM, QAM) Noise is AWG Eg. Multi-antenna systems

Approximate Solutions Zero forcing cancellation Nulling and canceling Nulling and canceling with optimal ordering computation BER comparison – ML vs. Approximate But Bit Error Rate suffers

Exact Methods Sphere Decoding : search in a hypersphere centered at (Fincke-Pohst ; Viterbo, Boutros; Vikalo, Hassibi) How do we find the points that are in the hypersphere?

To find points without exhaustive search When, this is an interval Use this to go from a -dimensional point to a (k+1) – dimensional point. Search over spheres of radius r and dimensions 1,2,…, N. Use to facilitate this Sphere Decoder

Sphere Decoder – How it Works Call

How it Works contd. depends only on

Solve these successively --- get a tree Complexity depends on the size of the tree Search Space and Tree

Reducing Complexity Not ML decoding any more

Results Complexity exponent and BER for N=20 with QPSK

Probability of Error Let e be the probability that the transmitted point s is not in the search space Can be shown that Need to keep small

Finding epsilon can be determined exactly in terms of s Choose s to make as small as desired Theorem:

Computational Complexity is the search region at dimension is the constellation Need to find

Finding s are independent. Hence Also, can be determined exactly Yet have to employ approximations…

Upper Bound For, it needs to satisfy conditions. For upper bound, just the -th condition. is the incomplete gamma function. Easy to compute

Approximations Can be shown that where and are functions of The complexity can now be determined by Monte Carlo simulations

Simulation Results Complexity exponent and BER for N=20 with QAM

Simulation Results Complexity Exponent and BER for N=50 with QAM

Conclusions and Future Work Significant reduction in Complexity BER can be made close to optimal Quantify trade-off between BER and Complexity Compare with other decoding algorithms Analyze for signaling schemes with coding Other applications for these techniques…?

How it Works contd. Solve these successively…