Network Elements based on Partial State A. L. Narasimha Reddy Dept. of Electrical Engineering Texas A & M University

Narasimha Reddy Texas A & M University 2 Acknowledgements Deying Tong (Cisco) Sai Gopalakrishnan (Cisco) Smitha (Intel) Phani Achanta (Graduating in Aug. 2002)

Narasimha Reddy Texas A & M University 3 Introduction Proposals for new network architectures –Full State (IntServ) Difficult to scale per-flow state with # of flows –No State (DiffServ) Flow isolation difficult

Narasimha Reddy Texas A & M University 4 Introduction What if we can build network elements with some fixed amount of state? –State is not enough for all the flows –What kind of services can we provide? Hypothesis: Only few flows need state, most flows can be aggregated.

Narasimha Reddy Texas A & M University 5 Motivation Typical Internet traffic consists of –Many short-lived flows (“mice”) pump below 20 packets (approximately 20KB) –Few large flows (“elephants”) Current resource management techniques do not distinguish the flows Dropping packets from short-lived flows may do little to ease congestion –Also, mice flows are latency sensitive

Narasimha Reddy Texas A & M University 6 Motivation (contd..) Congestion management “should” rely on controlling high bandwidth flows –Offer more control on traffic –Likely to be consuming disproportionate bandwidth –Likely to be “robust” (ftp for e.g..) May need mechanisms to control unresponsive applications –To improve fairness and to prevent congestion collapse

Narasimha Reddy Texas A & M University 7 Flow Classification Long-lived flows TCP flows (FTP Applications) UDP flows (Video Applications) Short-lived flows Telnet, HTTP transfers Responsive vs. Nonresponsive flows –ftp vs. some video transfers

Narasimha Reddy Texas A & M University 8 Basis for Partial State A Small fraction of flows contribute large fraction of bytes. If state can be allocated to these flows, resource management can be done efficiently without requiring full state.

Narasimha Reddy Texas A & M University 9 Basis for Partial State

Narasimha Reddy Texas A & M University 10 Basis for Partial State

Narasimha Reddy Texas A & M University 11 Basis for Partial State

Narasimha Reddy Texas A & M University 12 Partial State Approach Maintain Fixed amount of Partial State –State is not dependent on number of flows –State depends on engineering concerns Manage the state information to retain history of high-BW flows -- How? Adopt appropriate resource management based on the goals

Narasimha Reddy Texas A & M University 13 Partial State Approach Similar to how caches are employed in computer memory systems –Exploit locality Employ an engineering solution in an architecture-transparent fashion

Narasimha Reddy Texas A & M University 14 State Management Sampling is employed as a basic tool –High-BW flows more likely to be selected State organized as a Cache –Caches allow quick identification if flow is allocated state State Allocation can be –Policy Driven –Traffic Driven

Narasimha Reddy Texas A & M University 15 Policy Driven State Management An ISP could decide to monitor flows above 1Mbps –Will need state >= link capacity/1 Mbps Could monitor flows consuming more than 1% of link capacity –For security reasons –At most 100 flows with 1% BW consumption

Narasimha Reddy Texas A & M University 16 Traffic Driven State Management Monitor top 100 flows at any time –Don’t know the identity of these flows –Don’t know how much BW these may consume Employ LRU Cache management –Flows have to arrive at cache frequently to stay in cache –Maintains High-BW flows in a self-organizing way

Narasimha Reddy Texas A & M University 17 Traffic Driven State Management (contd…) Flows probabilistically admitted into cache, ‘p’. –Reduces the chance of short-term flows disturbing the cache state. Keep count of packet arrivals of cached flows –Declare a “high-BW” flow if count > Threshold

Narasimha Reddy Texas A & M University 18 The Algorithm New Packet In Cache? Cache size < ‘S’ Admit the flow into the cache with a probability ‘p’, count = 1 Update position and count Make a new entry, count=1 Yes No Yes No

Narasimha Reddy Texas A & M University 19 Why an LRU Cache? High bandwidth flows arrive often –Stay in the cache for longer periods Smooth flows stay in the cache longer compared to bursty flows –UDP flows (smooth) –TCP flows (bursty) Responsive flows reduce rate and get replaced –Nonresponsive flows remain in cache

Narasimha Reddy Texas A & M University 20 UDP Cache Occupancy

Narasimha Reddy Texas A & M University 21 TCP Cache Occupancy

Narasimha Reddy Texas A & M University 22 Resource Management Cached flows can be treated individually Noncached flows treated in an aggregate manner With larger state, finer control on traffic

Narasimha Reddy Texas A & M University 23 Resource Management Preferential Dropping (RED based) –Drop cached flows more often –Use Packet count as a scaling function Fair queuing –Cached flows, noncached flows in separate queues, employ WFQ –Possible to protect noncached flows from cached flows

Narasimha Reddy Texas A & M University 24 Resource Management

Narasimha Reddy Texas A & M University 25 Preferential Dropping drop prob Queue length drop prob for high bandwidth flows minthmaxth maxp 1 drop prob for other flows

Narasimha Reddy Texas A & M University 26 Preferential Dropping (contd..) As congestion builds up, above min_th, –if (flow->count >=‘threshold’) Pdrop = pred * flow->count / ‘threshold’ –else Pdrop = pred High-BW nonresponsive flows get higher drops Low-BW and responsive flows see lower drops

Narasimha Reddy Texas A & M University 27 Two Studies LRU-RED: Simulation based study –Provide lower drop rates for responsive and short-term flows –Approximately fair BW distribution LRU-FQ: Linux-based partial state router prototype –Contain DOS attacks –Provide shorter delays for short-term flows

Narasimha Reddy Texas A & M University 28 LRU-RED Simulations R1R2 40Mb 20Mb

Narasimha Reddy Texas A & M University 29 Topology 2 R1R2R3 40Mb30Mb 20Mb

Narasimha Reddy Texas A & M University 30 LRU-RED Results

Narasimha Reddy Texas A & M University 31 LRU-RED Results

Narasimha Reddy Texas A & M University 32 LRU-RED Results

Narasimha Reddy Texas A & M University 33 LRU-RED Results

Narasimha Reddy Texas A & M University 34 Varying Load

Narasimha Reddy Texas A & M University 35 RTT Bias -TCP flows

Narasimha Reddy Texas A & M University 36 Summary of LRU-RED LRU cache is effective in identifying high bandwidth, nonresponsive flows Combined the above with RED to propose a novel active queue management scheme Simulation results show the effectiveness of the scheme Sampling can further reduce the per-packet cost

Narasimha Reddy Texas A & M University 37 LRU-FQ Resource Management

Narasimha Reddy Texas A & M University 38 LRU-FQ Flow Chart – Enque Packet Arrival Is Flow in Cache? Yes No Does Cache Have space? Yes Admit flow with Probability ‘p’ No Is Flow Admitted? Record flow details Initialize ‘count’ to 0 Yes Increment ‘count’ Move flow to top of cache No Is ‘count’ >= ‘threshold’ No Yes Enqueue in Non-responsive Queue Enqueue in Responsive Queue

Narasimha Reddy Texas A & M University 39 LRU-FQ – Dequeue event  Dequeue event results in selection of a packet from either queues based on the Fair Queue algorithm used.  The weight assigned to the individual queues determine the proportion of bandwidth they are assigned.

Implementation Issues on Linux

Narasimha Reddy Texas A & M University 41 Linux IP Packet Forwarding Packet Arrival Check & Store Packet Enqueue pkt Request Scheduler To invoke bottom half Device Prepares packet Packet Departure Error checking Verify Destination Route to destination Update Packet Packet Enqueued Scheduler invokes Bottom half Scheduler runs Device driver Local packet Deliver to upper layers UPPER LAYERS IP LAYER LINK LAYER Design space

Narasimha Reddy Texas A & M University 42 Linux Kernel traffic control Filters are used to distinguish between different classes of flows. Each class of flows can be further categorized into sub-classes using filters. Queuing disciplines control how the packets are enqueued and dequeued

Narasimha Reddy Texas A & M University 43 LRU-FQ Implementation LRU component of the scheme is implemented as a filter. –All parameters: threshold, probability and cache size are passed as parameters to the filter Fair Queuing employed as a queuing discipline. –Scheduling based on queue’s weight. –Start-time Fair Queuing

LRU-FQ - Results

Narasimha Reddy Texas A & M University 45 Timing Results

Narasimha Reddy Texas A & M University 46 Long-Term flow differentiation Probability = 1/25Cache size= 11 threshold= 125 Normal TCP fraction = 0.07

Narasimha Reddy Texas A & M University 47 Long-term flow differentiation Probability = 1/25Cache size= 11 threshold= 125

Narasimha Reddy Texas A & M University 48 Protecting Web Mice

Narasimha Reddy Texas A & M University 49 Protecting Web mice 1:1LRU : Normal Queue 11LRU Cache Size 125Threshold 1/50Probability 20Web Clients 2 – 4LongTerm UDP Flows 20Long Term TCP Flows Experimental Setup

Narasimha Reddy Texas A & M University 50 Protecting Web Mice Bandwidth Results TCP Fraction TCP Tput # Web Requests UDP Tput UDP Flows TCP Fraction TCP Tput # Web Requests UDP Tput UDP Flows Normal Router LRU-FQ Router

Narasimha Reddy Texas A & M University 51 Protecting Web Mice Timing Results Normal Router LRU-FQ Router

Narasimha Reddy Texas A & M University 52 Summary of LRU-FQ Provides a good control of DOS attacks with limited number of flows Provides better delays for short-term flows Allows DDOS attack detection through wavelet signatures on miss traffic Automatically identifies resource hogs Partial state packet handling cost -not an issue at 100Mbps.

Narasimha Reddy Texas A & M University 53 References SACRED (Tong, Reddy ‘99) IWQOS SACRIO (Gopalakrishnan, Reddy ‘01): Partial state in Diff-serv Networks, NOSSDAV LRU-RED (Smitha, Reddy ‘01): Globecom 2001 LRU-FQ (Achanta, Reddy ‘02): In preparation WADeS (Ramanathan, Reddy ‘02): DDOS detection Please visit the following URLs for references – – m

Narasimha Reddy Texas A & M University 54 Applications of Partial State More intelligent control of network traffic Accounting and measurement of high bandwidth flows Denial of Service (DOS) attack prevention DDOS attack detection –Wavelet signatures of miss traffic give indications of attacks Tracing of high bandwidth flows QOS routing

Narasimha Reddy Texas A & M University 55 Related Work Route caching in LANs RED-PD[Mahajan,Floyd ’01]: RED drop history used to guide decisions Approximate Fairness through Differential Dropping [Pan, Breslau, Prabhakar, Shenker ’01]: Similar to RED-PD Traffic Measurement [Estan, Verghese ‘01]: Employ a lot more state, limited to measurement.

Narasimha Reddy Texas A & M University 56 Future Work Analyze impact of cache size and traffic behavior on QOS Provide mechanisms for “protection” of cached flows Implement on network processors to adapt to changing traffic conditions QOS routing of cached flows

Narasimha Reddy Texas A & M University 57 Thank you !! For more information, send e- mail to A.L. Narasimha Reddy at