Using Feedback in MANETs: a Control Perspective Todd P. Coleman University of Illinois DARPA ITMANET TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A AAAA A A A

Current Uses of Feedback Theory Feedback modeled noiseless Point-to-point: capacity unchanged Significantly improved error exponents Reduction in complexity MANETs: Enlargement of capacity region

Current Uses of Feedback Practice Feedback is noisy, used primarily for Robustness to channel uncertainty Estimation of channel parameters ARQ-style communication w/ erasures

Current Uses of Feedback Practice Feedback is noisy, used primarily for Robustness to channel uncertainty Estimation of channel parameters ARQ-style communication w/ erasures But: Burnashev-style “forward error correction+ARQ” schemes are extremely fragile w/ noisy feedback (Kim, Lapidoth, Weissman 07)

Instantiate network feedback control algorithms for MANETs Develop iterative practical schemes for noisy feedback? Coding w/ feedback over statistically unknown channels? Develop fundamental limits of error exponents with feedback w/ fixed block length Applicability of Feedback in MANETs

Communication w/ Noiseless Feedback

Communication w/ Noiseless Feedback Given an encoder’s Tx strategy, decoding is almost trivial (Baye’s rule)

Communication w/ Noiseless Feedback Given an encoder’s Tx strategy, decoding is almost trivial (Baye’s rule) How do we select a (recursive) encoder strategy for an arbitrary memoryless channel?

A Control Interpretation of the Dynamics of the Posterior Coleman ’09: “A Stochastic Control Approach to ‘Posterior Matching’-style Feedback Communication Schemes”

A Control Interpretation of the Dynamics of the Posterior Coleman ’09: “A Stochastic Control Approach to ‘Posterior Matching’-style Feedback Communication Schemes”

F k-1 Controller Z -1 P(F k |F k-1, u k ) ukuk FkFk reference signal Fw*Fw* A Control Interpretation of the Dynamics of the Posterior Coleman ’09: “A Stochastic Control Viewpoint on ‘Posterior Matching’-style Feedback Communication Schemes”

XkXk FkFk F k+1 Fw*Fw* D(F w * ||F k+1 ) D(F w * ||F k ) Reward at any stage k is the reduction in “distance” to target Stochastic Control: Reward Coleman ’09

Maximum Long-Term Average Reward Coleman ’09

Maximum Long-Term Average Reward (1),(2) hold w/ equality if: a) Y’s all independent b) Each X i drawn according to P*(x) Coleman ’09

Maximum Long-Term Average Reward (1),(2) hold w/ equality if: a) Y’s all independent b) Each X i drawn according to P*(x) Horstein ’63 (BSC) Schalwijk-Kailath ’66 (AWGN) Shayevitz-Feder ‘07, ‘08 (DMC) Coleman ’09

The Posterior Matching Scheme: an Optimal Solution Next input indep of everything decoder has seen so far, with capacity-achieving marginal distribution No forward error correction. Adapt on the fly. Coleman ’09 Posterior matching scheme

The Posterior Matching Scheme: an Optimal Solution Coleman ’09 Next input indep of everything decoder has seen so far, with capacity-achieving marginal distribution No forward error correction. Adapt on the fly. Posterior matching scheme

Implications for Demonstrating Achievable Rates 01 1 Coleman ’09

Lyapunov Function Posterior matching scheme: Coleman ’09

Lyapunov Function (cont’d) Coleman ’09

Control Theory Information Theory Symbiotic Relationship Converse Thms Give Upper Bounds on Average Long-Term Rewards for Stochastic Control Problem Coleman ’09: “A Stochastic Control Viewpoint on ‘Posterior Matching’-style Feedback Communication Schemes”

Control Theory Information Theory Symbiotic Relationship Converse Thms Give Upper Bounds on Average Long-Term Rewards for Stochastic Control Problem KL Divergence Lyapunov functions guarantee all rates achievable Coleman ’09: “A Stochastic Control Viewpoint on ‘Posterior Matching’-style Feedback Communication Schemes”

Research Results with This Methodology Interpret feedback communication encoder design as stochastic control of posterior towards certainty Converse theorems specify fundamental performance bounds on a stochastic control problem related to controlling posterior. An optimal policy implies the existence of a Lyapunov function, which is in essence a KL divergence Lyapunov function directly implies achievability for all R < C Coleman ’09

Research Results with This Methodology Gorantla and Coleman ‘09: Encoders that achieve El Gamal 78: “Physically degraded broadcast channels w/ feedback“ capacity region in an iterative fashion w/ low complexity Interpret feedback communication encoder design as stochastic control of posterior towards certainty Converse theorems specify fundamental performance bounds on a stochastic control problem related to controlling posterior. An optimal policy implies the existence of a Lyapunov function, which is in essence a KL divergence Lyapunov function directly implies achievability for all R < C Coleman ’09

New Important Directions this Approach Enables Control Theory Information Theory Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process

New Important Directions this Approach Enables Control Theory Information Theory Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process Optimal coding w/ feedback over statistically unknown channels? Reinforcement learning from control literature

New Important Directions this Approach Enables Control Theory Information Theory Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process Optimal coding w/ feedback over statistically unknown channels? Reinforcement learning from control literature Develop fundamental limits of error exponents with feedback w/ fixed block length Lyapunov function enables a fundamental Martingale condition

New Important Directions this Approach Enables Control Theory Information Theory Develop iterative low-complexity encoders/decoders for noisy feedback? Partially Observed Markov Decision Process Optimal coding w/ feedback over statistically unknown channels? Reinforcement learning from control literature Develop fundamental limits of error exponents with feedback w/ fixed block length Lyapunov function enables a fundamental Martingale condition Also: stochastic control approach provides a rubric to check tightness of converses via structure of optimal solution