Distributed Network Utility Maximization in Multi-hop Wireless Networks: Noisy Feedback, Lossy Channel and Stability Junshan Zhang Department of Electrical Engineering Arizona State University Jan. 30, 2007

Outline Overview of our research on wireless ad hoc networks and sensor networks Distributed Network utility maximization in multi-hop wireless networks.

Wireless Ad-Hoc/Sensor Networks Potential applications: Battlefield wireless networks, Monitoring chemical/biological warfare agents, Homeland security. Basic network models: (1) Many-to-one networks, (2) Multi-hop wireless networks, (3) Sensory relay networks.

Network Models (2) (1) (3)

Unique Features Stochastic nature of ad-hoc networks: time-varying fading, co-channel interference, hostile jamming, … mobility, dynamic network topology, … Key features of sensor networks: node cooperation data correlation energy constraints QoS metrics: e.g., distortion, energy, event detection

Challenges and Opportunities Challenge: How to transmit data/information over ad-hoc/sensor networks efficiently and reliably? Network environments are not “friendly”; Heterogeneous wireless applications: wireless Internet access, battlefield multi-hop networks, sensor networks Diverse requirements: high-bandwidth video and data, low-bandwidth voice and data Every challenge, if taken properly, is an opportunity!

Why and How to do Cross-Layer Design Goals: reliable communication-on-the-move in dynamic environments, QoS provisioning, energy efficiency Why: There exists direct coupling between physical layer and upper protocol layers in wireless networks; Cross-layer design is perhaps single most promising approach to meet the fast-growing demands in many cases, e.g., CDMA/HDR, 802.11n, Flarion OFDM Misconceptions: Cross-layer design means to get rid of protocol layers. Cross-layer design requires to integrate all protocol layers.

Research Thrust I: Cross-Layer Optimization and Design Holistic approach to devise adaptive protocol modules to exploit interplay between layers For network models (1)-(3), we explore cross-layer design for joint optimization of transport layer, routing, MAC and physical layer.

Wireless Network Architecture Data Link/MAC Network Transport Session Presentation Application Physical A holistic approach to wireless networking Cross-layer Design: adaptive modules Application Driven our goal: To understand overriding principles for cross layer design

Projects on Cross-layer Design Self-similarity of multi-access interference in CDMA networks with bursty traffic Traffic-aided opportunistic scheduling MIMO ad-hoc Networks Joint clustering and routing in sensor networks Joint congestion control and MAC design in multi-hop wireless networks Distributed opportunistic scheduling for ad-hoc communications

Research Thrust II: Cooperative Relaying and Networking Capacity bounds of MIMO relay channel Power allocation in wireless relay networks Scaling laws of wideband sensory relay networks; Technical details can be found in our recent papers: B. Wang, JZ & Host-Madsen (IT 05); Host-Madsen & JZ (IT 05); B. Wang, JZ & L. Zheng (IT 06)

Distributed Network Utility Maximization in Multi-hop Wireless Networks Random access Bursty traffic Channel fading Distributed NUM in multi-hop wireless networks: Noisy feedback, Lossy channel, Stochastic stability

Outline of Distributed NUM I: Distributed NUM under noisy feedback Deterministic P-D algorithm via Lagrangian dual; Stochastic P-D algorithm under noisy feedback: Unbiased case: stochastic stability, rate of convergence; Biased case: contraction region, stochastic stability. II: Distributed NUM under lossy channel: Effective NUM over wireless networks. Impact of UDP. Acknowledgement: Part I: based on joint work with Dong Zheng & Prof. Mung Chiang; Part II: based on joint work with Qinghai Gao.

Basic NUM ² C o n s i d e r a w l t k m c g p h G = ( N ; E ) { L , b y - v f S u x . U ° ® M : ¥ · P j 8 1

Decentralized solution to NUM Distributed solutions are key in making the NUM framework attractive in practical networks. General form for flow control [Low-Srikant 04]: Key observation: Some level of message passing is always needed! Reverse engineering perspective: Above network control method can be viewed as distributed solution to optimization problems; implemented via (sub)-gradient algorithm. x s ( t + 1 ) = F ; q 8 2 S ¸ l G y L w h e r a n d o m - g i v f u c , p ° k .

Distributed NUM via Lagrange Dual Classification: Primal; dual; primal-dual Distributed NUM via Lagrangian dual decomposition ² L ( x ; ¸ ) = X s U + l c ¡ f a g r n e d u t i o : Q m · M h p b D - 1 [ ] 8

Example 1: Joint flow Control and MAC Design ² C o n s i d e r a w l t k m c g p h G = ( N ; E ) M u ± ° U x y f b . j 8 2 L I [ v , ' Goal: fair rate control through joint congestion control and MAC design in multi-hop random access networks. Approach: network utility maximization Treat rate control as a utility maximization problem Different layers function “cooperatively” to achieve the optimum point (equilibrium point).

Example 1: NUM for Cross-layer Rate Control ¥ : m a x P s 2 S U ( ) u b j e c t o i ; · p Q k N I 1 ¡ 8 = M w h r f ° d , v g n l y ¢ [ W ] I n g e r a l , P o b m ¥ i s - c v x d p .

Example 1: Change of Variables t ~ x s = l o g ( ) , w h a v P : m f 2 S U u b j c i ; p ¡ k N 1 · 8 M r . O b s e r v a t i o n : P l m c x d p f · ¸ 1 .

Example 1: Lagrange Dual Approach T h e L a g r n i f u c t o s ( ~ x ; p ¸ ) = 8 < : X U ¡ j l @ 2 S 1 A 9 + Y k N I P , d Q m · M b v y D

Example 1: Distributed Primal-dual Alg. ² T h e s o u r c a t p d b y ~ x ( n + 1 ) = 2 6 4 @ _ U ¡ X i ; j L ¸ P S A | { z } , 3 7 5 M : w l g k N I m f

Distributed NUM under Noisy Feedback Implementation of distributed algorithms hinges on message passing among network elements Unfortunately, in practical systems feedback is often obtained using error-prone measurement mechanisms and also suffers from other random errors in its transmissions. Feedback signals are stochastically noisy in nature. Examples: ECN for Internet congestion control: congestion price is obtained via probabilistic packet marking technique. Wireless networks with random access: flow rate is based on sample-path measurements and feedback signal is transmitted over lossy wireless channel. Congestion price is affected by stochastic UDP

An Example on Exponential Marking ² A s u m e x p o n t i a l r k g d f b c , w h v - y q = ³ P ( ; j ) 2 L ¡ ¸ S ´ T N K . ^ B ¯ ® ~ E [ ]

Stochastic Primal Dual Alg. ² I n t h e p r s c o f i y d b a k , g L x ( ¢ ) ¸ l . + 1 = ³ ^ ; ´ D 8 ¡ Fundamental open questions on impact of noisy feedback Would distributed NUM alg., in the presence of stochastic noisy feedback, converge to the optimal point of deterministic algorithms? If yes, under what conditions and how quickly does it converge? In case not, would the distributed NUM alg. be stable at all?

Stochastic Stability Feedback information is needed to estimate subgradients; Unfortunately, the feedback is noisy in practical systems! Stochastic stability is pertinent to following issues: number of users/queuing length remain finite; algorithms converge in some stochastic sense. Goal: obtain a clear understanding of impact of stochastic noisy feedback on stability.

Related Work Related work on stability Connection-level stability: [Bonald-Massoulie 01], and [Lin-Shroff-Srikant 06] [Eryilmaz-Srikant05] [Neely-Modiano-Li 05] [Stolyar 05] and many more ; Delay stability: [Low-Srikant 04] [Ranjan-La-Abed 06] … Joint flow control/routing/MAC/PHY design in wireless networks: [Lin-Shroff 05], [Wang-Kar 05], [Chiang 04] [Chen-Low-Chiang-Doyle 06] Deterministic feedback error: [Mehyar-Spanos-Low 04] Rate of convergence around the equilibrium points: [Kelly-Malloo-Tan 98], [Kelly-03], … ;

Structure of Stochastic Gradients ^ L x ( ¢ ; ) : O b v ¸ = + ® ³ w m , y E ¡ l

Structure of Stochastic Gradients (Cont’d) ^ L ¸ ( ; j ) ¢ : O b v x = + ¯ » w m f , y E ¡ l ®

Technical Assumptions 1 . W e a s u m t h i o r f g d n b c l y 2 C p z : ² > , ! P < 3 j ® ( ) ; 8 ¯ 4 E [ ³ ] »

Main Result 1: Stability for Unbiased Case Theorem 1: Under Conditions the iterates generated by the stochastic primal-dual algorithm, converge with probability one to the optimal solutions of Problem A 1 ¡ 4 , f ( x n ) ; ¸ = 1 2 : g , ¥ . 1 . G o d n e w s : T h t c a i l g r m v p u A { 4 2 C y k ² W b z , E x f q - N \ "

Remarks Biased case: Key ideas for Proof: Construct Lyapunov function based on saddle point properties for P-D alg.; Super-martingale method; stochastic analysis. Biased case: When the gradient estimator is biased, we cannot hope for almost sure convergence; Instead, we expect that the iterates return to a neighborhood of the optimal points under certain conditions. Indeed, we can show that if the biases are uniformly bounded, there exists a “contraction region” such that the iterates return to this region infinitely often w.p.1.

Main Result 2: Stability for Biased Case 7 . C o n d i t h e b a s r : T x - g v c f ® ¯ ( ; j ) u l m p · 8 D e ¯ n t h \ c o r a i g " A ´ s f l w : , ( x ; ¸ ) ® j L ~ p m · < 1 Theorem 2: U n d e r C o i t s A 1 ¡ 2 , 4 a 7 h f ( x ) ; ¸ = : g b y c p m l - u ´ ¯ w .

Remarks 1 . C o n d i t A 7 e s a l y r q u h b m p c , w k 3 g 2 T \ " ´ v f ± z - N

Rate of Convergence ² T h e r a t o f c n v g i s d w y m p b l z . A , u P - D ( x ¤ ; ¸ ) ² D e ¯ n U x ( ) , ¡ ¤ = p a d ¸ . C o s t r u c b h i w l f ; g + 2 [ 1 P U n + 2 U n ( t ) U n U n + 1 ² n ² n + 1 ² n + 2 t

Assumptions A 5 . L e t µ ( n ) , ~ x ; ¸ a d Á ³ » S u p o s f r y g v m l ½ > h ¯ c § = ¾ E [ T ¡ ] I j ¤ · ! 1 D H : 6 ² + 2 w z N b - B k 9

Main Result 3: Rate of Convergence (unbiased case) 1 3 ¡ 6 , ( ¢ c v g w k l y h u - S p b m µ x ¸ ¶ = H + I 2 ¾ z ; b ) I f ( x ¤ ; ¸ i s a n t e r o p h c , l m g U y G u d ® v w z § . c ) I f ( x ¤ ; ¸ i s o n t h e b u d a r y , l m g p U ° ® .

Engineering Implications 1 ) I n g e r a l , t h i m p o c s y ° d ® u G ; z f [ K 9 8 ] . 2 ) T h e l i m t p r o c s w u d b G a n y f ° . 3 ) I n t u i v e l y s p a k g , h r ° c o m w d f b z x . 4 ) T h e c o v a r i n m t x f l p s g u d q b , y \ " .

Numerical Studies C o n s i d e r l g a t h m u y f c w · = 1 .

Numerical Examples – Deterministic Case

Numerical Examples – Stochastic (unbiased) Case

Numerical Examples: Biased Case

Towards a Framework for Understanding Impact of Noisy Feedback on Distributed NUM Impact of noisy feedback on distributed Algorithm is not well understood; applicable to both wireline networks and wireless networks. We are developing a general theory on stochastic NUM. Single time scale algorithm (P-D Alg.) Two time scale algorithm (e.g., based on primal decomposition). Tradeoff between complexity and robustness. Asynchronous NUM algorithms. Correlated measurements: tradeoff between stability and subgradient estimation

Two Time-Scale Algorithms Among alternative decomposition methods, primal decomposition is a useful machinery for problems with coupled variables: Important issues: stability of multiple time-scale algorithms under noisy feedback. complexity and robustness of two time-scale algorithm, compared with single time-scale algorithm? Optimizing over selected variables first, then the remaining variables; yields multiple time-scale algorithms

NUM Problem C o n s i d e r t h f l w g N U M p b m : ¥ a x z P ( ) u 2 a x z · P ( ) u j c L ; 8 = H F L i n k c a p t e s f l g r u o ¯ M A C m , d w b ° H h . For ease of exposition, we assume convexity of the above NUM problem.

Deterministic Algorithm F a s t e r ( m l ) i c : I T h o u p d b y x n = g · M @ U ¡ X 2 L ¸ 1 A ; 8 w + 4 S 3 5 C " k ¤ # Cross-layer message passing is significantly reduced. The two time-scale algorithm has a lower complexity.

Stochastic Two Time-Scale Algorithm F a s t e r i m c l : I S A g o h f u p d n x ( ) = · M @ U ¡ 2 4 X L ¸ + 3 5 1 w - , b y . ; ± C k N

Geometric Interpretation Two possible ways to implement the above two time-scale Algorithm: One popular method is to let faster time-scale iteration converge first, and then execute slower time-scale loop. Alternatively, one can set and run algorithms at both time scales; Method 2 is more robust to noisy feedback. b n = o ( a )

Notation F D e ¯ n t , P d m ( ) s u c h · < . l o g y w f r p : M ¡ 1 i = d m ( ) s u c h · < + . b l o g y w f r p : M ; x X ¸ N

Distributed NUM Under Lossy Channel Motivation: Data transmission over wireless links suffers from lossy channels. Under-utilization with TCP over wireless channels. Existing solutions for flow control over wireless channels: distinguishing packet loss due to lossy channels from that due to congestion. Concerns: retransmissions incur time out; scalability

Problem Formulation Our approach: effective network utility maximization ² C h a n e l c p i t y s r d o m ; u f R · . T ® v P ( ) E w k : X U x Y 2 L 1 ¡ g b

Effective NUM Link Constraints: ENUM Problem: m a x P U ³ Q 1 ¡ p ´ : < s 2 S ( ) x g · ± ENUM Problem: m a x f s 2 I g P U ³ Q l L ( ) 1 ¡ p S ´ : t C < · ± ; 8

Lagrangian Dual Method ENUM with logarithm utility: m a x f s 2 I g P ! ³ l o ( ) ¡ L t S 1 ¹ ° ´ : · ~ c ; 8 w h e r i d n b y k u . Use Lagrangian Dual method: L ( x ; ¸ ) = X s @ ! l o g ¡ 2 P t S 1 ¹ ° A + ~ c D m a f I i n

ENUM Algorithm link congestion price depends on total flow rate x s ( n + 1 ) = " ! ¡ X l 2 Ã ¸ P t S ¹ ° # I 4 @ ~ c A 3 5 link congestion price depends on total flow rate Link error price depends on total flow rate and link SNR, along the path.

Numerical Example 2 links, 3 flows U(x) = log(x) SNR1=10dB, SNR2=20dB Each link has link constraint: log(1+SNR) Rayleigh fading x2 > x1 > x3

Injection Rate & Effective Rate

ENUM under UDP m a x f s 2 I g P U ³ e p ( ¡ l L ) t S + u 1 ¹ ° ´ : · ~ c ; 8 Decrease the data rate limit for elastic flows at each link. Increase the outage probability at each link.

ENUM under UDP

Conclusions and Ongoing work Distributed NUM over multi-hop wireless networks. Distributed NUM under noisy feedback Deterministic P-D alg. via Lagrangian dual; Stability of Stochastic P-D algorithm: Unbiased case: convergence, rate of convergence; Biased case: contraction region. Distributed NUM under lossy channel: Effective NUM over wireless networks. Impact of UDP. Combined impact of noisy feedback and lossy channel: Q) End-to-end approach vs. hop-by-hop approach?

Proof for the Main Result 1 : U s i n g h o c a L y u v b l T r m , w f d . ² L e t ( x ¤ ; ¸ ) b a s d l p o i n . D ¯ h y u v f c V ¢ w : V ( x ; ¸ ) , j ¡ ¤ 2 + m i n © ² D e ¯ n f o r a g i v ¹ > , h b d s t A x ; ¸ ) : V ( ·

Stochastic Lyapunov Function W e s h o w E n [ V ( x + 1 ) ; ¸ ] · 2 ² G O j ® ¯ i t u d r a g = ¡ ¤ T L m © . I t s u ± c e o h w a G ( x n ) ; ¸ < 2 A ¹ .

A Supermartingale Lemma f X n g b a R r - v l u d s o c h i p , V ( ¢ ) . S Y q m y P j < 1 w F ¾ ¡ ; · x A ½ E [ + ] ² = 2 ¯ T

Step II – Local Analysis : W s a b l i h , v \ o c n y " r u - d m g f w ( ~ x ) ; ¸ = 1 2 A 3 ¹ ¯ . L k ±

Proof for Main Result 2 N o w l e t ¹ ! , h a v A . 1 ) D e ¯ n A , [ ´ . I t c a b s h o w i m p A ´ , f ( x ; ¸ ) : ® s j L o r m e ¯ i · < 1 g 2 ) S i m l a r t o h e p f M n R s u 1 , c b w E [ V ( x + ; ¸ ] · ² ¡ ´ G O : 3 ) S i n c e ´ < 1 , u s g t h f a G ( x ; ¸ ¡ ± o r m p v w = 2 A l - ¯ y b Y 8 . N o w l e t ¹ ! , h a v A ´ .

Thank You!