1. Graphical Models and Message Passing Receivers for Interference Limited Communication Systems
Marcel Nassar
PhD Defense
Committee Members:
Prof. Gustavo de Veciana
Prof. Brian L. Evans (supervisor)
Prof. Robert W. Heath Jr.
Prof. Jonathan Pillow
Prof. Haris Vikalo
April 17, 2013

2. Outline Uncoordinated interference in communication systems
Effect of interference on OFDM systems
Prior work on receivers in uncoordinated interference
Message-passing receiver design
Learning interference model parameters: robust receivers
Summary and future work
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

3. Modern Communication Systems
Single Carrier System:
frequency selective fading
Noise/Interference 𝑡 𝑇 𝑓 𝐻
Channel +
equalizer
Implementation complexity
Orthogonal Frequency Division Multiplexing (OFDM): 𝐻 𝑓 𝑡 𝑁𝑇 +
Channel
Noise/Interference 1
N sub-bands
Simpler Equalization
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

4. Interference in Communication Systems
Wireless LAN in ISM band
Powerline Communication
Non-interoperable standards
Co-Channel interference
Platform
Coexisting Protocols
Non-Communication Sources
Electromagnetic emissions
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

5. Interference Management
An active area of research …
Orthogonal Access
MAC Layer Access: Co-existence [Rao2002] [Andrews2009]
Precoding Techniques: Inter-cell interference cancellation [Boudreau2009], network MIMO (CoMP) [Gesbert2007], Interference Alignment [Heath2013]
Successive Interference Cancellation [Andrews2005]
What about residual interference?
What about non-communicating sources?
What about non-cooperative sources?
Complementary approach: statistically model and mitigate
Uncoordinated Interference
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

6. Thesis Contributions Contribution I: Uncoordinated Inference Modeling
Contribution II: Receiver Design
Contribution III: Robust Receiver Design
Thesis Statement:
Receivers can leverage interference models to enhance decoding and increase spectral efficiency in interference limited systems.
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

7. Uncoordinated Interference Modeling

8. Statistical-Physical Modeling
Wireless Systems
Powerline Systems
WiFi, Ad-hoc
[Middleton77, Gulati10]
Middleton Class-A
𝑝 𝑛 = 𝑖=0 ∞ 𝑒 −𝐴 𝐴 𝑖 𝑖! 𝑁 𝑛;0, 𝑖Ω 𝐴
A >0: Impulsive Index
Ω>0: Mean Intensity
Rural, Industrial, Apartments [Middleton77,Nassar11]
WiFi Hotspots [Gulati10]
Gaussian Mixture (GM)
𝐾∈ℕ : # of comp.
𝜋 𝑖 : comp. probability
𝛾 𝑖 : comp. variance
Dense Urban, Commercial
[Nassar11]
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

9. Empirical Modeling – WiFi Platform
[Data provided by Intel]
Gaussian Mixture Model:
Gaussian HMM Model: 1 2
𝑁 0, 𝜎 1 2
𝑁 0, 𝜎 2 2
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

10. Empirical Modeling – Powerline Systems
Markov Chain Model [Zimmermann2002] 1 … 1 2 1 5 2
Impulsive
Background
Measurement Data
[Katayama06]
Proposed Model
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

11. Receiver Design

12. OFDM Basics + System Diagram Noise Model Receiver Model 𝐻 𝑓
noise + interference 𝑓 𝐻
LDPC Coded
Symbol Mapping
Inverse DFT
DFT
Source +
0111 …
1+i 1-i -1-i 1+i …
channel
Noise Model
Receiver Model
After discarding the cyclic prefix:
After applying DFT:
Subchannels:
total noise
where
background noise
interference
and
GM or GHMM
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

13. Effect of GM Interference on OFDM
Single Carrier (SC)
OFDM
Single Carrier vs. OFDM
Impulse energy high → OFDM sym. lost
Impulse energy low → OFDM sym. recovered
In theory, with joint MAP decoding OFDM ≫ Single Carrier
[Haring2002]
tens of dBs
(symbol by symbol decoding)
DFT
Impulse energy concentrated
Symbol lost with high probability
Symbol errors independent
Min. distance decoding is MAP-optimal
Minimum distance decoding under GM:
Impulse energy spread out
Symbol lost ??
Symbol errors dependent
Disjoint minimum distance is sub-optimal
Disjoint minimum distance decoding:
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

14. pilots → linear channel estimation → symbol detection → decoding
OFDM Symbol Structure
Coding
Data Tones
Added redundancy protects against errors
Symbols carry information
Finite symbol constellation
Adapt to channel conditions
Pilot Tones
Null Tones
Edge tones (spectral masking)
Guard and low SNR tones
Ignored in decoding
Known symbol (p)
Used to estimate channel
pilots → linear channel estimation → symbol detection → decoding
But, there is unexploited information and dependencies
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

15. Prior Work
Category
References
Method
Limitations
Time-Domain preprocessing
[Haring2001]
Time-domain signal MMSE estimation
ignore OFDM signal structure
performance vs DFT rx degrades with increasing SNR and modulation order
[Zhidkov2008,Tseng2012]
Time-domain signal thresholding
Sparse Signal Reconstruction
[Caire2008,Lampe2011]
Compressed sensing
utilize only known tones
don’t use interference models
complexity
[Lin2011]
Sparse Bayesian Learning
Iterative Receivers
[Mengi2010,Yih2012]
Iterative preprocessing & decoding
Suffer from preprocessing limitations
Ad-hoc design
[Haring2004]
Turbo-like receiver
All don’t consider the non-linear channel estimation, and don’t use code structure
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

16. depends on linearly-mixed N noise samples and L channel taps
Joint MAP-Decoding
The MAP decoding rule of LDPC coded OFDM is:
Can be computed as follows:
depends on linearly-mixed N noise samples and L channel taps
non iid & non-Gaussian
LDPC code
Very high dimensional integrals and summations !!
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

17. Belief Propagation on Factor Graphs
Graphical representation of pdf-factorization
Two types of nodes:
variable nodes denoted by circles
factor nodes (squares): represent variable “dependence “
Consider the following pdf:
Corresponding factor graph:
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

18. Belief Propagation on Factor Graphs
Approximates MAP inference by exchanging messages on graph
Factor message = factor’s belief about a variable’s p.d.f.
Variable message = variable’s belief about its own p.d.f.
Variable operation = multiply messages to update p.d.f.
Factor operation = merges beliefs about variable and forwards
Complexity = number of messages = node degrees
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

19. Coded OFDM Factor Graph
Information bits
Bit loading & modulation
unknown channel taps
Unknown interference samples
Symbols
Received Symbols
Coding & Interleaving
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

20. BP over OFDM Factor Graph
LDPC Decoding via BP [MacKay2003]
Node degree=N+L!!!
MC Decoding
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

21. Generalized Approximate Message Passing [Donoho2007,Rangan2010]
Decoupling via Graphs
Estimation with Linear Mixing
observations
variables
coupling
Generally a hard problem due to coupling
Regression, compressed sensing, …
OFDM systems:
If graph is sparse use standard BP
If dense and ”large” → Central Limit Theorem
At factors nodes treat as Normal
Depend only on means and variances of incoming messages
Non-Gaussian output → quad approx.
Similarly for variable nodes
Series of scalar MMSE estimation problems: 𝑂(𝑁+𝑀) messages
Interference subgraph
channel subgraph
given
given
and
and
3 types of output channels for each
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

22. Proposed Message-Passing Receiver
Schedule
Turbo Iteration:
Initially uniform
coded bits to symbols
symbols to
Run channel GAMP
Run noise “equalizer”
to symbols
Symbols to coded bits
Run LDPC decoding
GAMP
LDPC Dec.
GAMP
Equalizer Iteration:
Run noise GAMP
MC Decoding
Repeat
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

23. Receiver Design & Complexity
Design Freedom
Not all samples required for sparse interference estimation
Receiver can pick the subchannels:
Information provided
Complexity of MMSE estimation
Selectively run subgraphs
Monitor convergence (GAMP variances)
Complexity and resources
GAMP can be parallelized effectively
Notation
Operation
Complexity per iteration
MC Decoding
𝒪(𝑁)
LDPC Decoding
𝒪( 𝑀 𝑐 +𝐶)
GAMP
𝒪( min 𝑁 log 𝑁 , 𝑈 2 )
𝑁: # tones
𝑀 𝑐 : # coded bits
𝐶: # check nodes
𝑈: set of used tones
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

24. Simulation Settings Interference Model Receiver Parameters Definitions
Two impulsive components:
7% of time/20dB above background
3% of time/30dB above background
Two types of temporal dynamics:
i.i.d. samples
Hidden Markov Model
Receiver Parameters
Definitions
15 GAMP iterations
5 turbo iterations
FFT Size 256 (PLC) FFT Size1024 (Wireless)
50 LDPC iterations
SER: Symbol Error Rate
BER: Bit Error Rate
SNR: Signal to “noise + interference” power ratio
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

25. Simulation - Uncoded Performance
use LMMSE channel estimate
performs well when interference dominates time-domain signal
use only known tones, requires matrix inverse
2.5dB better than SBL
5 Taps
GM noise
4-QAM
N=256
15 pilots
80 nulls
Settings
within 1dB of MF Bound
15db better than DFT
Matched Filter Bound: Send only one symbol at tone k
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

26. Simulation – Modeling Gain
correct marginal: 4dB gain
temporal dependency: extra 4dB
amplitude accuracy is not important
using all tones gives 8dB gain
amplitude accuracy gives 7db gain
Settings
flat ch.
N=256
60 nulls
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

27. Simulation – Tone Map Design
Typical configuration performs significantly worse
Tone Map Design
How to allocate tones?
Limited resources → select tones?
Optimal solution not known…….
Dictionary coherence
For same node type
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

28. Simulation - Coded Performance
one turbo iteration gives 9db over DFT
10 Taps
GM noise
16-QAM
N=1024
150 pilots
Rate ½
L=60k
Settings
5 turbo iterations gives 13dB over DFT
Integrating LDPC-BP into JCNED by passing back bit LLRs gives 1 dB improvement
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

29. Robust Receiver Design

30. Recall - Coded OFDM Factor Graph
contains parameters that might be unknown
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

31. Learning the Interference Model
Training-Based
Robust Receiver
parameter estimate
quiet period
model training
corrupted transmission
joint estimation & detection Θ
Detection Θ
corrupted transmission
Detection
Computationally simpler detection
For slowly varying environments
Suffer from model mismatch in rapidly varying environments
More computation for parameter estimation (but not always)
Adapts to rapidly varying environment
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

32. Parameter Estimation via EM Algorithm
Simplifies ML Estimation by:
Marginalize over latent
Maximize w.r.t parameters
Marginalization easy for directly observed GM and GHMM samples
EM for Robust Receivers
Approximate marginalization using GAMP messages
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

33. Sparse Bayesian Learning via GAMP
Bayesian approach to compressive sensing
Use data to fit 𝛾 via EM
If e is sparse ⇒ lot of 𝛾 end up zero
Requires big matrix inverse
Use only null tones
Linear channel estimation
Prior: 𝑒|𝛾 ~ 𝐶𝛮 0,𝛤 ,𝛤≜diag 𝛾
SBL via GAMP
Integrate into GAMP estimation functions
Linear input estimator
Can include all tones and code
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

34. Simulation Result – SBL via GAMP
SBL via EM using only null tones
SBL via GAMP using all tones
EM Parameter estimation
5 Taps
GM noise
4-QAM
N=256
15 pilots
80 nulls
Settings
Blind EM Parameter Estimation
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

35. FPGA Test System for G3 PLC Receiver
Simplified message-passing receiver using only null tones
In collaboration with Karl Nieman
NI PXIe-7965R
(Virtex 5)
NI PXIe-1082
Real-time host
Introduction |Message Passing Receivers | Simulations | Robust Receivers| Summary

36. Summary
Significant performance gains if receiver accounts for uncoordinated interference
Proposed solution combines all available information to perform approximate-MAP inference
Asymptotic complexity similar to conventional OFDM receiver
Can be parallelized
Highly flexible framework: performance vs. complexity tradeoff
Robust for fast-varying interference environments

37. Future Work Temporal Modeling of Uncoordinated Interference
Wireless Networks
Powerline Networks
Tractable Inference
Pilot and null tone allocation in impulsive noise
Coherence not optimal
Trade-off between channel and noise estimation
Extension to different interference and noise models
Cyclostationary noise
ARMA models for spectrally shaped noise
Mitigation of narrowband interferers
Sparse in frequency domain

38. Related Publications
M. Nassar, K. Gulati, M. R. DeYoung, B. L. Evans and K. R. Tinsley, "Mitigating Near-Field Interference in Laptop Embedded Wireless Transceivers", Journal of Signal Processing Systems, Mar. 2009, invited paper.
M. Nassar, X. E. Lin and B. L. Evans, "Stochastic Modeling of Microwave Oven Interference in WLANs", Proc. IEEE Int. Comm. Conf., Jun. 5-9, 2011, Kyoto, Japan.
M. Nassar and B. L. Evans, "Low Complexity EM-based Decoding for OFDM Systems with Impulsive Noise", Proc. Asilomar Conf. on Signals, Systems and Computers, Nov. 6-9, 2011, Pacific Grove, CA USA.
M. Nassar, K. Gulati, Y. Mortazavi, and B. L. Evans, "Statistical Modeling of Asynchronous Impulsive Noise in Powerline Communication Networks", Proc. IEEE Int. Global Comm. Conf.. Dec. 5-9, 2011, Houston, TX USA.
J. Lin, M. Nassar and B. L. Evans, "Non-Parametric Impulsive Noise Mitigation in OFDM Systems Using Sparse Bayesian Learning", Proc. IEEE Int. Global Comm. Conf., Dec. 5-9, 2011, Houston, TX USA.
M. Nassar, A. Dabak, I. H. Kim, T. Pande and B. L. Evans, "Cyclostationary Noise Modeling In Narrowband Powerline Communication For Smart Grid Applications“, Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, March 25-30, 2012, Kyoto, Japan.
M. Nassar, J. Lin, Y. Mortazavi, A. Dabak, I. H. Kim and B. L. Evans, "Local Utility Powerline Communications in the kHz Band: Channel Impairments, Noise, and Standards", IEEE Signal Processing Magazine, Sep. 2013
J. Lin, M. Nassar, and B. L. Evans, ``Impulsive Noise Mitigation in Powerline Communications using Sparse Bayesian Learning'', IEEE Journal on Selected Areas in Communications, vol. 31, no. 7, Jul. 2013, to appear.
K. F. Nieman, J. Lin, M. Nassar, B. L. Evans, and K. Waheed, ``Cyclic Spectral Analysis of Power Line Noise in the kHz Band'', Proc. IEEE Int. Symp. on Power Line Communications and Its Applications, Mar , 2013, Johannesburg, South Africa. Won Best Paper Award.
M. Nassar, P. Schniter and B. L. Evans, ``Message-Passing OFDM Receivers for Impulsive Noise Channels'', IEEE Transactions on Signal Processing, to be submitted.

39. Thank you

40. BACK UP SLIDES