Graphical Models and Message Passing Receivers for Interference Limited Communication Systems Marcel Nassar PhD Defense Committee Members: Prof. Gustavo de Veciana Prof. Brian L. Evans (supervisor) Prof. Robert W. Heath Jr. Prof. Jonathan Pillow Prof. Haris Vikalo April 17, 2013

Outline Uncoordinated interference in communication systems Effect of interference on OFDM systems Prior work on receivers in uncoordinated interference Message-passing receiver design Learning interference model parameters: robust receivers Summary and future work Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Modern Communication Systems Single Carrier System: frequency selective fading Noise/Interference 𝑡 𝑇 𝑓 𝐻 Channel + equalizer 0110010 Implementation complexity Orthogonal Frequency Division Multiplexing (OFDM): 𝐻 𝑓 𝑡 𝑁𝑇 + Channel Noise/Interference 0110010 1 N sub-bands Simpler Equalization Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Interference in Communication Systems Wireless LAN in ISM band Powerline Communication Non-interoperable standards Co-Channel interference Platform Coexisting Protocols Non-Communication Sources Electromagnetic emissions Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Interference Management An active area of research … Orthogonal Access MAC Layer Access: Co-existence [Rao2002] [Andrews2009] Precoding Techniques: Inter-cell interference cancellation [Boudreau2009], network MIMO (CoMP) [Gesbert2007], Interference Alignment [Heath2013] Successive Interference Cancellation [Andrews2005] What about residual interference? What about non-communicating sources? What about non-cooperative sources? Complementary approach: statistically model and mitigate Uncoordinated Interference Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Thesis Contributions Contribution I: Uncoordinated Inference Modeling Contribution II: Receiver Design Contribution III: Robust Receiver Design Thesis Statement: Receivers can leverage interference models to enhance decoding and increase spectral efficiency in interference limited systems. Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Uncoordinated Interference Modeling

Statistical-Physical Modeling Wireless Systems Powerline Systems WiFi, Ad-hoc [Middleton77, Gulati10] Middleton Class-A 𝑝 𝑛 = 𝑖=0 ∞ 𝑒 −𝐴 𝐴 𝑖 𝑖! 𝑁 𝑛;0, 𝑖Ω 𝐴 A >0: Impulsive Index Ω>0: Mean Intensity Rural, Industrial, Apartments [Middleton77,Nassar11] WiFi Hotspots [Gulati10] Gaussian Mixture (GM) 𝐾∈ℕ : # of comp. 𝜋 𝑖 : comp. probability 𝛾 𝑖 : comp. variance Dense Urban, Commercial [Nassar11] Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Empirical Modeling – WiFi Platform [Data provided by Intel] Gaussian Mixture Model: Gaussian HMM Model: 1 2 𝑁 0, 𝜎 1 2 𝑁 0, 𝜎 2 2 Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Empirical Modeling – Powerline Systems Markov Chain Model [Zimmermann2002] 1 … 1 2 1 5 2 Impulsive Background Measurement Data [Katayama06] Proposed Model Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Receiver Design

OFDM Basics + System Diagram Noise Model Receiver Model 𝐻 𝑓 noise + interference 𝑓 𝐻 LDPC Coded Symbol Mapping Inverse DFT DFT Source + 0111 … 1+i 1-i -1-i 1+i … channel Noise Model Receiver Model After discarding the cyclic prefix: After applying DFT: Subchannels: total noise where background noise interference and GM or GHMM Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Effect of GM Interference on OFDM Single Carrier (SC) OFDM Single Carrier vs. OFDM Impulse energy high → OFDM sym. lost Impulse energy low → OFDM sym. recovered In theory, with joint MAP decoding OFDM ≫ Single Carrier [Haring2002] tens of dBs (symbol by symbol decoding) DFT Impulse energy concentrated Symbol lost with high probability Symbol errors independent Min. distance decoding is MAP-optimal Minimum distance decoding under GM: Impulse energy spread out Symbol lost ?? Symbol errors dependent Disjoint minimum distance is sub-optimal Disjoint minimum distance decoding: Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

pilots → linear channel estimation → symbol detection → decoding OFDM Symbol Structure Coding Data Tones Added redundancy protects against errors Symbols carry information Finite symbol constellation Adapt to channel conditions Pilot Tones Null Tones Edge tones (spectral masking) Guard and low SNR tones Ignored in decoding Known symbol (p) Used to estimate channel pilots → linear channel estimation → symbol detection → decoding But, there is unexploited information and dependencies Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Prior Work Category References Method Limitations Time-Domain preprocessing [Haring2001] Time-domain signal MMSE estimation ignore OFDM signal structure performance vs DFT rx degrades with increasing SNR and modulation order [Zhidkov2008,Tseng2012] Time-domain signal thresholding Sparse Signal Reconstruction [Caire2008,Lampe2011] Compressed sensing utilize only known tones don’t use interference models complexity [Lin2011] Sparse Bayesian Learning Iterative Receivers [Mengi2010,Yih2012] Iterative preprocessing & decoding Suffer from preprocessing limitations Ad-hoc design [Haring2004] Turbo-like receiver All don’t consider the non-linear channel estimation, and don’t use code structure Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

depends on linearly-mixed N noise samples and L channel taps Joint MAP-Decoding The MAP decoding rule of LDPC coded OFDM is: Can be computed as follows: depends on linearly-mixed N noise samples and L channel taps non iid & non-Gaussian LDPC code Very high dimensional integrals and summations !! Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Belief Propagation on Factor Graphs Graphical representation of pdf-factorization Two types of nodes: variable nodes denoted by circles factor nodes (squares): represent variable “dependence “ Consider the following pdf: Corresponding factor graph: Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Belief Propagation on Factor Graphs Approximates MAP inference by exchanging messages on graph Factor message = factor’s belief about a variable’s p.d.f. Variable message = variable’s belief about its own p.d.f. Variable operation = multiply messages to update p.d.f. Factor operation = merges beliefs about variable and forwards Complexity = number of messages = node degrees Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Coded OFDM Factor Graph Information bits Bit loading & modulation unknown channel taps Unknown interference samples Symbols Received Symbols Coding & Interleaving Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

BP over OFDM Factor Graph LDPC Decoding via BP [MacKay2003] Node degree=N+L!!! MC Decoding Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Generalized Approximate Message Passing [Donoho2007,Rangan2010] Decoupling via Graphs Estimation with Linear Mixing observations variables coupling Generally a hard problem due to coupling Regression, compressed sensing, … OFDM systems: If graph is sparse use standard BP If dense and ”large” → Central Limit Theorem At factors nodes treat as Normal Depend only on means and variances of incoming messages Non-Gaussian output → quad approx. Similarly for variable nodes Series of scalar MMSE estimation problems: 𝑂(𝑁+𝑀) messages Interference subgraph channel subgraph given given and and 3 types of output channels for each Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Proposed Message-Passing Receiver Schedule Turbo Iteration: Initially uniform coded bits to symbols symbols to Run channel GAMP Run noise “equalizer” to symbols Symbols to coded bits Run LDPC decoding GAMP LDPC Dec. GAMP Equalizer Iteration: Run noise GAMP MC Decoding Repeat Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Receiver Design & Complexity Design Freedom Not all samples required for sparse interference estimation Receiver can pick the subchannels: Information provided Complexity of MMSE estimation Selectively run subgraphs Monitor convergence (GAMP variances) Complexity and resources GAMP can be parallelized effectively Notation Operation Complexity per iteration MC Decoding 𝒪(𝑁) LDPC Decoding 𝒪( 𝑀 𝑐 +𝐶) GAMP 𝒪( min 𝑁 log 𝑁 , 𝑈 2 ) 𝑁: # tones 𝑀 𝑐 : # coded bits 𝐶: # check nodes 𝑈: set of used tones Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Simulation Settings Interference Model Receiver Parameters Definitions Two impulsive components: 7% of time/20dB above background 3% of time/30dB above background Two types of temporal dynamics: i.i.d. samples Hidden Markov Model Receiver Parameters Definitions 15 GAMP iterations 5 turbo iterations FFT Size 256 (PLC) FFT Size1024 (Wireless) 50 LDPC iterations SER: Symbol Error Rate BER: Bit Error Rate SNR: Signal to “noise + interference” power ratio Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Simulation - Uncoded Performance use LMMSE channel estimate performs well when interference dominates time-domain signal use only known tones, requires matrix inverse 2.5dB better than SBL 5 Taps GM noise 4-QAM N=256 15 pilots 80 nulls Settings within 1dB of MF Bound 15db better than DFT Matched Filter Bound: Send only one symbol at tone k Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Simulation – Modeling Gain correct marginal: 4dB gain temporal dependency: extra 4dB amplitude accuracy is not important using all tones gives 8dB gain amplitude accuracy gives 7db gain Settings flat ch. N=256 60 nulls Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Simulation – Tone Map Design Typical configuration performs significantly worse Tone Map Design How to allocate tones? Limited resources → select tones? Optimal solution not known……. Dictionary coherence For same node type Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Simulation - Coded Performance one turbo iteration gives 9db over DFT 10 Taps GM noise 16-QAM N=1024 150 pilots Rate ½ L=60k Settings 5 turbo iterations gives 13dB over DFT Integrating LDPC-BP into JCNED by passing back bit LLRs gives 1 dB improvement Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Robust Receiver Design

Recall - Coded OFDM Factor Graph contains parameters that might be unknown Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Learning the Interference Model Training-Based Robust Receiver parameter estimate quiet period model training corrupted transmission joint estimation & detection Θ Detection 01010011 Θ corrupted transmission Detection 01010011 Computationally simpler detection For slowly varying environments Suffer from model mismatch in rapidly varying environments More computation for parameter estimation (but not always) Adapts to rapidly varying environment Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Parameter Estimation via EM Algorithm Simplifies ML Estimation by: Marginalize over latent Maximize w.r.t parameters Marginalization easy for directly observed GM and GHMM samples EM for Robust Receivers Approximate marginalization using GAMP messages Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Sparse Bayesian Learning via GAMP Bayesian approach to compressive sensing Use data to fit 𝛾 via EM If e is sparse ⇒ lot of 𝛾 end up zero Requires big matrix inverse Use only null tones Linear channel estimation Prior: 𝑒|𝛾 ~ 𝐶𝛮 0,𝛤 ,𝛤≜diag 𝛾 SBL via GAMP Integrate into GAMP estimation functions Linear input estimator Can include all tones and code Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Simulation Result – SBL via GAMP SBL via EM using only null tones SBL via GAMP using all tones EM Parameter estimation 5 Taps GM noise 4-QAM N=256 15 pilots 80 nulls Settings Blind EM Parameter Estimation Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

FPGA Test System for G3 PLC Receiver Simplified message-passing receiver using only null tones In collaboration with Karl Nieman NI PXIe-7965R (Virtex 5) NI PXIe-1082 Real-time host Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

Summary Significant performance gains if receiver accounts for uncoordinated interference Proposed solution combines all available information to perform approximate-MAP inference Asymptotic complexity similar to conventional OFDM receiver Can be parallelized Highly flexible framework: performance vs. complexity tradeoff Robust for fast-varying interference environments

Future Work Temporal Modeling of Uncoordinated Interference Wireless Networks Powerline Networks Tractable Inference Pilot and null tone allocation in impulsive noise Coherence not optimal Trade-off between channel and noise estimation Extension to different interference and noise models Cyclostationary noise ARMA models for spectrally shaped noise Mitigation of narrowband interferers Sparse in frequency domain

Related Publications M. Nassar, K. Gulati, M. R. DeYoung, B. L. Evans and K. R. Tinsley, "Mitigating Near-Field Interference in Laptop Embedded Wireless Transceivers", Journal of Signal Processing Systems, Mar. 2009, invited paper. M. Nassar, X. E. Lin and B. L. Evans, "Stochastic Modeling of Microwave Oven Interference in WLANs", Proc. IEEE Int. Comm. Conf., Jun. 5-9, 2011, Kyoto, Japan. M. Nassar and B. L. Evans, "Low Complexity EM-based Decoding for OFDM Systems with Impulsive Noise", Proc. Asilomar Conf. on Signals, Systems and Computers, Nov. 6-9, 2011, Pacific Grove, CA USA. M. Nassar, K. Gulati, Y. Mortazavi, and B. L. Evans, "Statistical Modeling of Asynchronous Impulsive Noise in Powerline Communication Networks", Proc. IEEE Int. Global Comm. Conf.. Dec. 5-9, 2011, Houston, TX USA. J. Lin, M. Nassar and B. L. Evans, "Non-Parametric Impulsive Noise Mitigation in OFDM Systems Using Sparse Bayesian Learning", Proc. IEEE Int. Global Comm. Conf., Dec. 5-9, 2011, Houston, TX USA. M. Nassar, A. Dabak, I. H. Kim, T. Pande and B. L. Evans, "Cyclostationary Noise Modeling In Narrowband Powerline Communication For Smart Grid Applications“, Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, March 25-30, 2012, Kyoto, Japan. M. Nassar, J. Lin, Y. Mortazavi, A. Dabak, I. H. Kim and B. L. Evans, "Local Utility Powerline Communications in the 3-500 kHz Band: Channel Impairments, Noise, and Standards", IEEE Signal Processing Magazine, Sep. 2013 J. Lin, M. Nassar, and B. L. Evans, ``Impulsive Noise Mitigation in Powerline Communications using Sparse Bayesian Learning'', IEEE Journal on Selected Areas in Communications, vol. 31, no. 7, Jul. 2013, to appear. K. F. Nieman, J. Lin, M. Nassar, B. L. Evans, and K. Waheed, ``Cyclic Spectral Analysis of Power Line Noise in the 3-200 kHz Band'', Proc. IEEE Int. Symp. on Power Line Communications and Its Applications, Mar. 24-27, 2013, Johannesburg, South Africa. Won Best Paper Award. M. Nassar, P. Schniter and B. L. Evans, ``Message-Passing OFDM Receivers for Impulsive Noise Channels'', IEEE Transactions on Signal Processing, to be submitted.

Thank you

BACK UP SLIDES