1. 1 Backward Congestion Notification Version 2.0 Davide Bergamasco (davide@cisco.com)davide@cisco.com Rong Pan (ropan@cisco.com) Cisco Systems, Inc. IEEE 802.1 Interim Meeting Garden Grove, CA (USA) September 22, 2005

2. 222 Credits Valentina Alaria (Cisco) Andrea Baldini (Cisco) Flavio Bonomi (Cisco) Manoj K. Wadekar (Intel)

3. 333 BCN v2.0 Desire from Mick to see an analytical study of BCN stability BCN v2.0 improvements Linear control loop allows analysis of stability Simplified detection mechanism Reduced signaling rate Original BCN framework remains the same

4. 444 BCN Background

5. 555 Detection & Signaling

6. 666 Reaction

7. 777 Suggested BCN Message Format 0 15 31 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + DA = SA of sampled frame +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ SA = MAC Address of CP + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IEEE 802.1Q Tag or S-Tag | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | EtherType = BCN |Version| Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + CPID + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Qoff | Qdelta | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Timestamp | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | | | First N bytes of sampled frame starting from DA | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FCS | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

8. 888 Suggested RLT Tag Format 0 3 7 15 31 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + DA of rate-limited frame +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ SA of rate-limited frame + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IEEE 802.1Q Tag or S-Tag of rate-limited frame | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | EtherType = RLT |Version| Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | + CPID + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Timestamp |EtherType of rate limited frame| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | + Payload of rate-limited frame + | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FCS | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

9. 999 Simulation Environment (1) Congestion TCP Bulk UDP On/Off

10. 10 Simulation Environment (2) Short Range, High Speed DC Network Link Capacity = 10 Gbps Switch latency = 1 s Link Length = 100 m (0.5 s propagation delay) Control loop Delay ~ 3 s Parameters W = 2 Gi = 4 Gd = 1/64 Ru = 8 Mbps Workload ST1-ST4: 10 parallel TCP connections transferring 1 MB each continuously SU1-SU4: 64 KB bursts of UDP traffic starting at t = 10 ms

11. 11 BCNv1.0

12. 12 BCNv2.0 Higher Stability @ Steady State Faster Transient Response

13. 13 Simulation Environment (3) Long Range, High Speed DC Network Link Capacity = 10 Gbps Switch latency = 1 s Link Length = 20000 m (100 s propagation delay) Control loop Delay ~ 200 s Parameters W = 2 Gi = 4 Gd = 1/64 Ru = 8 Mbps Workload ST1-ST4: 10 parallel TCP connections transferring 1 MB each continuously SU1-SU4: 64 KB bursts of UDP traffic starting at t = 10 ms

14. 14 BCNv1.0

15. 15 BCNv2.0 Much higher stability @ steady state with larger loop delays

16. 16 Summary BCN v2 has a number of advantages … Can be studied analytically Better protection of TCP flows in mixed TCP and UDP traffic scenarios Detection algorithm independent of Switch implementation Better Performance Lower signaling frequency (from 10% to 1%) Better stability Increased tolerance to loop delays … and one disadvantage Slower convergence to fairness

17. 17 A Control-Theoretic Approach to BCN Design and Analysis

18. 18 Notation N: Number of Flows C: Link Capacity : Round Trip Delay w: Weight of the Derivitive P m : Sampling Probability G i : Additive Increase Gain G d : Multiplicative Decrease Gain

19. 19 Block Diagram of BCN Congestion Control + C _ qR Time Delay + + _ GiGi R PmPm GdGd + +

20. 20 Non-linear Differential Equations If Fb(t- ) > 0 If Fb(t- ) < 0 Link Control Source Control

21. 21 Linearization Around Operating Point Using feedback control to analyze local stability Operating point: R = C/N; q = q eq – q = 0; Linearization Difficulty: depending on sgn(Fb(t-d)), the system responses are different –Luckily, a piecewise-linear function Details are in the appendix

22. 22 Block Diagram of BCN Feedback Control + R _ + + q Fb lose 90 o margin add lead zero to compensate Multiplicative Decrease: Additive Increase:

23. 23 The Effect Of Zero From Time Domains Eyes R q zero:dq/dt

24. 24 Choosing Parameters – an example Network conditions (10G link) N = 50 = 200us Choose parameters such that the feedback loop is stable with a 35 o margin w = 4 G i = 2Mbps G d = 1/128 P m = 0.01

25. 25 Stability Result: lost 90 o margin 1.With N = 50, delay = 200us, the system is stable 2.Phase margin translates into allowing extreme network conditions of N -> 1000 flows or -> 1ms before oscillation

26. 26 Simulation Result Shows A Stable System for N = 50; Delay = 200us

27. 27 Simulation Result Shows System is stable, but on the verge of oscillation: N = 50, Delay = 1ms

28. 28 Change W = 4 -> 1 1.When w = 1, a system with N = 50, delay = 200us already runs out of margin, on the verge of oscillation 2.w = 1, diminishing zero effect. System cant cope with wide range of network conditions

29. 29 Indeed System is stable, but on the verge of oscillation even for N = 50, Delay = 200us when w = 1.0

30. 30 Requests to 802.1 Start a Task Force on Congestion Management Use BCN as a Baseline Proposal

31. 31 Appendix

32. 32 Linearizing…

33. 33 Linearizing Additive Increase Function

34. 34 Linearizing Additive Increase Function

35. 35 Linearizing Multiplicative Decrease Function

36. 36 Linearizing Multiplicative Decrease Function

37. 37 ----++++ Stop Generation of BCN Messages t Q Qeq Issue #1: Non-linearity ISSUE: Overshoots and undershoots accumulate over time SOLUTION: Signal only when Q > Q eq && dQ/dt > 0 Q < Q eq && dQ/dt < 0 Easy to implement in hardware: just an Up/Down counter Increment @ every enqueue Decrement @ every dequeue Reduces signaling rate by 50%!!

38. 38 Issue #2: Specific Detection Mechanism

39. 39