1. Real-time Traffic monitoring and containment A. L. Narasimha Reddy Dept. of Electrical Engineering Texas A & M University reddy@ee.tamu.edu http://ee.tamu.edu/~reddy/

2. Narasimha Reddy Texas A & M University 2 Acknowledgements Deying Tong, Smitha, Phani Achanta Seong Soo Kim

3. Narasimha Reddy Texas A & M University 3 Outline Introduction & Motivation DOS attacks –Partial state routers DDOS attacks, worms –Aggregate Packet header data as signals –Signal/image based anomaly/attack detectors

4. Narasimha Reddy Texas A & M University 4 Introduction UDP-based multimedia traffic increasing UDP does not have congestion control Applications can be “selfish” –If everyone is selfish, network can break down Controlling “selfish” flows desired –Identify Resource hogs and control them

5. Narasimha Reddy Texas A & M University 5 Impact of UDP -- Unfairness When UDP and TCP compete, UDP wins by pushing TCP into congestion [Floyd&Fall 99]

6. Narasimha Reddy Texas A & M University 6 Unfairness - FIFO

7. Narasimha Reddy Texas A & M University 7 Unfairness - WRR

8. Narasimha Reddy Texas A & M University 8 Loss of goodput -FIFO Packets dropped later in network

9. Narasimha Reddy Texas A & M University 9 Loss of goodput -WRR

10. Narasimha Reddy Texas A & M University 10 UDP -- Summary Individual flows need to respond to congestion When end-hosts don’t respond to congestion –Need to identify and contain such flows –Need network mechanisms for such control

11. Narasimha Reddy Texas A & M University 11 Introduction (cont’d) Many Network attacks Exploit Application, Protocol, Network architecture vulnerabilities Denial of Service attacks –Consume all resources –Leave no resources for legitimate users

12. Narasimha Reddy Texas A & M University 12 TCP SYN Flooding (cont’d) The attack occurs by the attacker initiating a TCP connection to the server with a SYN. (using a legitimate or spoofed source address) The server replies with a SYN-ACK The client then doesn’t send back a ACK, causing the server to allocate memory for the pending connection and wait. (If the client spoofed the initial source address, it will never receive the SYN-ACK)

13. Narasimha Reddy Texas A & M University 13 TCP SYN Flooding: Results The half-open connections buffer on the victim server will eventually fill The system will be unable to accept any new incoming connections until the buffer is emptied out. There is a timeout associated with a pending connection, so the half-open connections will eventually expire. The attacking system can continue sending connection requesting new connections faster than the victim system can expire the pending connections.

14. Narasimha Reddy Texas A & M University 14 TCP Three-Way Handshake SYN Client wishes to establish connection SYN-ACK Server agrees to connection request ACK Client finishes handshake Client initiates request Connection is now half-open Client connection Established Server connection Established Client connecting to a TCP port

15. Narasimha Reddy Texas A & M University 15 SYN Flood Illustrated Client spoofs request half-open S S S Queue filled S S S Client SYN Flood I have ACKed these connections, but I have not received an ACK back!

16. Narasimha Reddy Texas A & M University 16 Smurf Example 1. Attacker sends ICMP packet with spoofed source IP Victim  10.1.2.255 2. Attacker sends ICMP packet with spoofed source IP Victim  192.168.1.255 3. Victim is flooded with ICMP echo responses 4. Victim hangs?

17. Narasimha Reddy Texas A & M University 17 Distributed Denial of Service Attacks (DDOS) Attacker logs into Master and signals slaves to launch an attack on a specific target address (victim). Slaves then respond by initiating TCP, UDP, ICMP or Smurf attack on victim.

18. Narasimha Reddy Texas A & M University 18 Network Attacks -- Summary Many vulnerabilities exist in Networks Malicious traffic increasing –For fun and profit Need mechansims to identify and control malicious traffic DOS and DDOS DOS, resource hog problem similar DDOS requires new approach

19. Narasimha Reddy Texas A & M University 19 Real-time traffic monitoring Attacks motivate us to monitor network traffic –Potential anomaly/attack detectors –Potentially contain/throttle them as they happen Line speeds are increasing –Need simple, effective mechanisms Attacks constantly changing –CodeRed yesterday, MyDoom today, what next

20. Narasimha Reddy Texas A & M University 20 Motivation Most current monitoring/policing tools are tailored to known attacks –Look for packets with port number 1434 (CodeRed) –Contain Kaaza traffic to 20% of the link Become ineffective when traffic patterns or attacks change –New threats are constantly emerging

21. Narasimha Reddy Texas A & M University 21 Motivation Can we design generic (and generalized) mechanisms for attack detection and containment? Can we make them simple enough to implement them at line speeds?

22. Narasimha Reddy Texas A & M University 22 Introduction Why look for Kaaza packets –They consume resources –Consume resources more than we want Not much different from DOS flood –Consumes resources to stage attacks Why not monitor resource usage? –Do not want to rely on attack specific info

23. Narasimha Reddy Texas A & M University 23 Attacks DOS attacks –Few sources = resource hogs DDOS attacks, worms –Many sources –Individual flows look normal –Look at the aggregate picture

24. Narasimha Reddy Texas A & M University 24 DOS attacks & Network Flows Too many flows to monitor each flow Maintain a fixed amount of state/memory –State not enough to monitor all flows (Partial state) –Manage the state to monitor high-bandwidth flows –How? Sample packets –High-BW flows more likely to be selected Use a cache and employ LRU type policy –Traffic driven –Cache retains frequently arriving flows

25. Narasimha Reddy Texas A & M University 25 Partial State Approach Similar to how caches are employed in computer memory systems –Exploit locality Employ an engineering solution in an architecture-transparent fashion

26. Narasimha Reddy Texas A & M University 26 Identifying resource hogs Lots of web flows –Tend to corrupt the cache quickly Apply probabilistic admission into cache –Flow has to arrive often to be included in cache –Most web flows not admitted Works well in identifying high-BW flows Can apply resource management techniques to contain cached/identified flows

27. Narasimha Reddy Texas A & M University 27 LRU with probabilistic admission Employ a modified LRU On a miss, flow admitted with probability p –When p is small, keeps smaller flows out –High-BW flows more likely admitted –Allows high-BW flows to be retained in cache Nonresponsive flows more likely to stay in cache

28. Narasimha Reddy Texas A & M University 28 Traffic Driven State Management Monitor top 100 flows at any time –Don’t know the identity of these flows –Don’t know how much BW these may consume

29. Narasimha Reddy Texas A & M University 29 Policy Driven State Management An ISP could decide to monitor flows above 1Mbps –Will need state >= link capacity/1 Mbps Could monitor flows consuming more than 1% of link capacity –For security reasons –At most 100 flows with 1% BW consumption

30. Narasimha Reddy Texas A & M University 30 Partial State –Trace-driven evaluation

31. Narasimha Reddy Texas A & M University 31 Partial State –Trace-driven Evaluation

32. Narasimha Reddy Texas A & M University 32 UDP Cache Occupancy

33. Narasimha Reddy Texas A & M University 33 TCP Cache Occupancy

34. Narasimha Reddy Texas A & M University 34 Resource Management

35. Narasimha Reddy Texas A & M University 35 Preferential Dropping drop prob Queue length drop prob for high bandwidth flows minthmaxth maxp 1 drop prob for other flows

36. Narasimha Reddy Texas A & M University 36 Multiple possibilities SACRED: Monitor flows above certain rate (policy driven), differential RED, (iwqos99) LRU-RED: Traffic driven state management, differential RED (Globecom01) –Approximately fair BW distribution LRU-FQ: Traffic driven state management, fair queuing (ICC 04) –Contain DOS attacks –Provide shorter delays for short-term flows

37. Narasimha Reddy Texas A & M University 37 SACRED Sampling And Caching RED Maintain flow rate as state for cached flows If flow rate > threshold, drop at higher rate –Drop rate keeps increasing if flow stays above threshold –Tends to punish nonresponsive flows, high-BW flows If flow rate < threshold, remove from cache –Make room for another flow

38. Narasimha Reddy Texas A & M University 38 SACRED results -10% state

39. Narasimha Reddy Texas A & M University 39 SACRED – cache associativity

40. Narasimha Reddy Texas A & M University 40 SACRED --Additive

41. Narasimha Reddy Texas A & M University 41 SACRED –TCP only

42. Narasimha Reddy Texas A & M University 42 LRU-FQ Resource Management

43. Narasimha Reddy Texas A & M University 43 LRU-FQ flow chart – enqueue event Packet Arrival Is Flow in Cache? Yes No Does Cache Have space? Yes Admit flow with Probability ‘p’ No Is Flow Admitted? Record flow details Initialize ‘count’ to 0 Yes Increment ‘count’ Move flow to top of cache No Is ‘count’ >= ‘threshold’ No Yes Enqueue in Partial state Queue Enqueue in Normal Queue

44. Narasimha Reddy Texas A & M University 44 Linux IP Packet Forwarding Packet Arrival Check & Store Packet Enqueue pkt Request Scheduler To invoke bottom half Device Prepares packet Packet Departure Error checking Verify Destination Route to destination Update Packet Packet Enqueued Scheduler invokes Bottom half Scheduler runs Device driver Local packet Deliver to upper layers UPPER LAYERS IP LAYER LINK LAYER Design space

45. Narasimha Reddy Texas A & M University 45 Linux Kernel traffic control Filters are used to distinguish between different classes of flows. Each class of flows can be further categorized into sub-classes using filters. Queuing disciplines control how the packets are enqueued and dequeued

46. Narasimha Reddy Texas A & M University 46 LRU-FQ Implementation LRU component of the scheme is implemented as a filter. –All parameters: threshold, probability and cache size are passed as parameters to the filter Fair Queuing employed as a queuing discipline. –Scheduling based on queue’s weight. –Start-time Fair Queuing

47. Narasimha Reddy Texas A & M University 47 Experimental Setup

48. Narasimha Reddy Texas A & M University 48 Long-Term flow differentiation Probability = 1/25Cache size= 11 threshold= 125 Normal TCP fraction = 0.07

49. Narasimha Reddy Texas A & M University 49 Long-term flow differentiation Probability = 1/25Cache size= 11 threshold= 125

50. Narasimha Reddy Texas A & M University 50 Protecting Web Mice

51. Narasimha Reddy Texas A & M University 51 Protecting Web mice 1:1LRU : Normal Queue 11LRU Cache Size 125Threshold 1/50Probability 20Web Clients 2 – 4LongTerm UDP Flows 20Long Term TCP Flows Experimental Setup

52. Narasimha Reddy Texas A & M University 52 Protecting Web Mice Bandwidth Results 0.0656.2192789.134 0.0585.55128489.803 0.0625.88131389.452 TCP Fraction TCP Tput # Web Requests UDP Tput UDP Flows 0.4944.511363246.244 0.4944.831382845.733 0.4944.921391545.732 TCP Fraction TCP Tput # Web Requests UDP Tput UDP Flows Normal Router LRU-FQ Router

53. Narasimha Reddy Texas A & M University 53 Protecting Web Mice Timing Results Normal Router LRU-FQ Router

54. Narasimha Reddy Texas A & M University 54 Summary of Partial-State Sampling and Caching allows simple identification of resource hogs Provides a good control of DOS attacks with limited number of flows Provides fairer distribution of link BW Partial state packet handling cost -not an issue at 100Mbps/1Gbps. –1Gbps implemented on Intel Network processor

55. Narasimha Reddy Texas A & M University 55 Applications of Partial State More intelligent control of network traffic Accounting and measurement of high bandwidth flows Denial of Service (DOS) attack prevention Tracing of high bandwidth flows QOS routing

56. Narasimha Reddy Texas A & M University 56 Aggregated packet analysis

57. Narasimha Reddy Texas A & M University 57 Approach Network Traffic Signal Generation & Data Filtering (Address correlation) Anomaly Detection (Thresholding) Detection Signal Statistical or Signal Analysis (Wavelets or DCT)

58. Narasimha Reddy Texas A & M University 58 Signal Generation Traffic volume (bytes or packets) –Analyzed before –May not be a great signal when links are always congested (typical campus access links) Lot more information in packet headers –Source address –Destination address –Protocol number –Port numbers

59. Narasimha Reddy Texas A & M University 59 Signal Generation Per packet cost is important driver Update a counter for each packet header field –Too much memory to put in SRAM Break the field into multiple 8-bit fields –32-bit address into four 8-bit fields –1024 locations instead of 2^32 locations –In general, 256* (k/8) instead of 2^k –k/8 counter updates instead of 1

60. Narasimha Reddy Texas A & M University 60 Signal Generation What kind of signals can we generate with addresses, port numbers and protocol numbers?

61. Narasimha Reddy Texas A & M University 61 Addresses are correlated Most of us have habits –Access same web sites Large web sites get significant part of traffic –Google.com, hp.com, yahoo.com Large downloads correlate over time –ftp, video On an aggregate, addresses are correlated

62. Narasimha Reddy Texas A & M University 62 Address Correlation –attacks? Address correlation changes when traffic patterns change abruptly –Denial of service attacks –Flash crowds –Worms Results in differences in correlation –High --single attack victim –Low – lots of addresses --worm

63. Narasimha Reddy Texas A & M University 63 Address correlation signals Address correlation: Simplified Address correlation:

64. Narasimha Reddy Texas A & M University 64 Address Correlation Signals

65. Narasimha Reddy Texas A & M University 65 Address Correlation Signals

66. Narasimha Reddy Texas A & M University 66 Signal Analysis Capture information over a sampling period –Of the order of a few seconds to minutes Analyze each sample to detect anomalies –Compare with historical norms Post-mortem/Real-time analysis –May use different amounts of data & analysis Detailed information of past few samples Less detailed information of older samples

67. Narasimha Reddy Texas A & M University 67 Signal Analysis Address correlation as a time series signal Employ known techniques to analyze time series signals Wavelets –one powerful technique –Allows analysis in both time and frequency domain Per-sample analysis has more flexibility –Not in forwarding path

68. Narasimha Reddy Texas A & M University 68 Does this work?

69. Narasimha Reddy Texas A & M University 69 Analysis of address signal

70. Narasimha Reddy Texas A & M University 70 Image based analysis Treat the traffic data as images Apply image processing based analysis Treat each sample as a frame in a video –Video compression techniques lead to data reduction –Scene change analysis leads to anomaly detection –Motion prediction leads to attack prediction

71. Narasimha Reddy Texas A & M University 71 Signal Generation

72. Narasimha Reddy Texas A & M University 72 Two dimensional images Horizontal/vertical lines indicate anomalies –Infected machine contacting multiple destinations (worm propagation) –Multiple source machines targeting a destination (DDOS)

73. Narasimha Reddy Texas A & M University 73 DCT analysis of addresses

74. Narasimha Reddy Texas A & M University 74 Semi-random attacks

75. Narasimha Reddy Texas A & M University 75 Random attacks

76. Narasimha Reddy Texas A & M University 76 Complex attacks

77. Narasimha Reddy Texas A & M University 77 Better than volume analysis

78. Narasimha Reddy Texas A & M University 78 Evaluation True Positive Rate False Alarm Rate or False Positive Rate True Negative Rate False Negative Rate LR = true positive rate/ false positive rate NLR = false negative rate/true –ve rate Ideally, LR = infinity, NLR = 0

79. Narasimha Reddy Texas A & M University 79 Comparison of Scalar signals

80. Narasimha Reddy Texas A & M University 80 Protocol Composition During attack, attack protocol volume will be higher –Observation of changes can lead to detection

81. Narasimha Reddy Texas A & M University 81 Protocol Composition

82. Narasimha Reddy Texas A & M University 82 Address based signals

83. Narasimha Reddy Texas A & M University 83 Port Number Domain

84. Narasimha Reddy Texas A & M University 84 Thresholds vs. Detection

85. Narasimha Reddy Texas A & M University 85 Motion prediction

86. Narasimha Reddy Texas A & M University 86 End host attacks Common solution to several kinds of attacks? Do something simple in the network layer –State maintenance and policing Our Key Idea: Per Resource regulation –Hierarchical regulation (per resource, per flow) also possible Move regulation away from server into the network (eg. At firewall)

87. Narasimha Reddy Texas A & M University 87 QOS Regulation to control network attacks

88. Narasimha Reddy Texas A & M University 88 End host – QOS regulation Limit consumption of each resource –At bastion Host Limit resource consumption to a traffic class so that other classes keep getting service

89. Narasimha Reddy Texas A & M University 89 End host protection Have a uniform picture of resources at the network layer –We do this at the QOS Regulator Resource Aggregates (resource principals) –Memory, Protocol State Buffers, mbuf / sk_buff Clusters, Network Bandwidth, CPU Cycles... Charge incoming traffic to one or more of these resource aggregates

90. Narasimha Reddy Texas A & M University 90 End host protection (cont’d) What does Rate Control achieve? –UDP food regulation –ICMP flood regulation –Interrupt / packet processing regulation –What about TCP SYN? CGI attack? –Consume Fixed number of resources What does Window Control achieve? –Regulates fixed number of resources –Need to keep track of resource usage –TCP SYN data structures, CGI processes, Memory –Sometimes action required to reset system state and free resources

91. Narasimha Reddy Texas A & M University 91 Experimental results

92. Narasimha Reddy Texas A & M University 92 Results – SYN attacks

93. Narasimha Reddy Texas A & M University 93 Advantages Not looking for specific known attacks Generic mechanism Works in real-time –Latencies of a few samples –Simple enough to be implemented inline

94. Narasimha Reddy Texas A & M University 94 Prototypes Linux-PC boxes On Intel Network processors –Can push to Gbps packet forwarding rates –Forwarding throughput not impacted –Sampling rates of a few ms possible

95. Narasimha Reddy Texas A & M University 95 Related Work Resource usage monitoring –Estan & Verghese –Bloom filters –Kodialam & Lakshman – Run detection –Mahajan et al – RED-PD –Duffield (AT & T) – Sampling –Others

96. Narasimha Reddy Texas A & M University 96 Related Work –Worms Payload monitoring –Singh, Savage & Verghese, Tang & Chen –Look for matches against constant length payloads Sampling, Rabin Signatures –Prototype implementation –Detects worms within 5-30 seconds –Effective with polymorphic worms

97. Narasimha Reddy Texas A & M University 97 Related Work -- Worms Look for TCP Reset signals –Weaver & Paxson –Random host scan at a specific ports –Not all hosts open attack port –Attacking worm will get many Resets –Too many Resets => Attacker –Effective for TCP based attacks –Can detect/contain in real-time

98. Narasimha Reddy Texas A & M University 98 Related Work -- Worms Quick spreading worms use randomly generated addresses –Normal users use names, DNS –Worms don’t have DNS activity –Lots of accesses without DNS requests => Worms –Many detectors within a campus Local DNS servers

99. Narasimha Reddy Texas A & M University 99 Related Work -- Worms Address honeypots –Arbor networks, Paxson, CrowCroft –Configure machines to accept packets for unassigned addresses –Only worms will contact these machines –Capture payloads to analyze –Quickly propagate signatures

100. Narasimha Reddy Texas A & M University 100 Related Work -- Worms IP Traceback – Savage et al –Address spoofing makes origin of attacks difficult to detect –Tracing, if universal, will limit attacks Fear of detection –Post-attack detection Not helpful in mitigating or detection –Most attack machines are innocent participants

101. Narasimha Reddy Texas A & M University 101 Related Work –host based Limit the number of new connections of individual hosts –TwyCross & Williamson (HP) –Reduces the speed at which a worm can spread –Can be used to detect worms Monitor application execution sequences –Profiling based indication of anomalous behavior => Detect and sandbox worms

102. Narasimha Reddy Texas A & M University 102 Conclusion Real-time resource accounting is feasible Real-time traffic monitoring is feasible –Simple enough to be implemented inline Can rely on many tools from signal/image processing area –More robust offline analysis possible –Concise for logging and playback

103. Narasimha Reddy Texas A & M University 103 Thank you !! For more information, http://ee.tamu.edu/~reddy reddy@ee.tamu.edu

104. Narasimha Reddy Texas A & M University 104 LRU-RED Results

105. Narasimha Reddy Texas A & M University 105 RTT Bias -TCP flows

106. Narasimha Reddy Texas A & M University 106 Impact of Cache size Effect of varying cache size –to study impact of cache size on performance of the scheme –probability= 1/55, threshold = 125 –number of TCP flows=20 –equal weights for both queues.

107. Narasimha Reddy Texas A & M University 107 Results – Cache size

108. Narasimha Reddy Texas A & M University 108 Normal Workloads Performance under normal workloads –working of scheme when non-responsive loads are absent or use their fair share of bandwidth –cache size = 9, threshold =125 –probability = 1/55

109. Narasimha Reddy Texas A & M University 109 Results – Normal workload

110. Narasimha Reddy Texas A & M University 110 Normal Mixed workload

111. Narasimha Reddy Texas A & M University 111 Interrupt processing overhead for server (incoming UDP traffic = 100Mbps) QoS Rate Limit on Regulator -> Received UDP Goodput (Kpkts/sec) ->