1. 60 Days of Basic Naughtiness
Probes and Attacks Endured by an Active Web Site
16 March 2001
This is part of a larger project – to analyze and track attack and probe methods and sources.
A holistic view of site probes and attacks.
To create an early warning and verification/validation system/site for others to use.
To track particularly popular source netblocks and assist the netblock owners with proper defense and mitigation.
Rob Thomas

2. 60 Days of Basic Naughtiness
Statistical analysis of log and IDS files.
Statistical analysis of a two-day DDoS attack.
Methods of mitigation.
Questions.
The analysis was performed using copious amounts of Perl, sed, awk, and coffee. 
The methods are still being honed – they are far from perfect.
Started with a series of simple questions raised by a manager :
“When did this attack start?”
“Did it slowly gain speed? Should we have seen it coming?”
“When will it end? Can we predict the end of the attack?”
Rob Thomas

3. Rob Thomas robt@cymru.com
About the Site
Production site for several (> 4) years.
Largely static content.
No e-commerce.
Layers of defense – more on that later!
Agreed to keep the site name and related data confidential.
I am always looking for more log files! Please feel free to share anything you have. Send it to my address.
Rob Thomas

4. Rob Thomas robt@cymru.com
About the Data
Data from router logs.
Data from IDS logs.
Snapshot taken from 60 days of combined data.
Data processed by several home-brew tools (mostly Perl and awk).
The router and IDS logs are not in sync, unfortunately.
This is the first significant effort with copious amounts (942MB) of log data.
Previous efforts utilized a significantly smaller sample size.
Rob Thomas

5. Definition of “Naughty”
Any traffic that is logged by a specific “deny” ACL.
Any traffic that presents a pattern detected by the IDS software.
The two log sources are not necessarily synchronized.
ACLs – deny and log RPC, bogon source addresses, etc.
IDS – detect SYN floods, Backorifice probes, etc.
The data files do not overlap perfectly. This would be ideal for a completely holistic view of site probes and attacks.
Remember – NTP is your friend!
Rob Thomas

6. Daily Probes and Attacks
TCP and UDP Probes and Attacks – ICMP not counted.
Average –
Standard deviation – !
60 Day Low – 83.00
60 Day High –
The standard deviation is higher than the mean! In other words, the ebb and flow of probes and attacks is of a greatly variable nature, and therefore difficult to model.
Note the 60 day low figure of This means that there is never a completely “safe” day!
Rob Thomas

7. Daily Probes and Attacks
Note that never a day goes by without at least 83 probes and/or attacks!
The miscreants hit the site 24 x 7 x 365.
Why so many? Because the miscreants do NOT share data! Elucidate this point (IRC wars).
Rob Thomas

8. Weekly Probes and Attacks
There is no steady-state.
Attacks come in waves, generally on the heels of a new exploit and scan.
Certain types of scans (e.g. Netbios) tend to run 24x7x365.
Proactive monitoring, based on underground and public alerts, will result in significant data capture.
There is no “steady-state,” and there is no “normal” week or day.
The vulnerability du jour (e.g. BIND hole, sendmail hole) tends to wane over a two to three month period.
When an alert comes out – particularly one that lists a port or ports – fire up an ACL and watch the log entries grow. 
Rob Thomas

9. Weekly Probes and Attacks Trend Analysis
Rob Thomas

10. Hourly Probes and Attacks
Myth: “Most attacks occur at night.”
An attacker’s evening may be a victim’s day – the nature of a global network.
Truth: Don’t plan based on the clock.
We are now part of a global network – a network that NEVER sleeps.
The average DDoS attack lasts between 24 and 72 hours – the zombies don’t need a break.
Take a guess – what hour of the day will be the most utilized by the miscreants to hit this site?
Rob Thomas

11. Hourly Probes and Attacks Trend Analysis
Rob Thomas

12. UDP Probes and Attacks Top Five Destination Ports
First – 137 NETBIOS
Second – 53 DNS
Third – 27960
Fourth – 500 ISAKMP
Fifth – (likely UNIX traceroute)
Rob Thomas

13. UDP Probes and Attacks Trend Analysis
Rob Thomas

14. TCP Probes and Attacks Top Five Destination Ports
First – 3663 (DDoS Attack)
Second – 0 Reserved (DDoS Attack)
Third – 6667 IRC (DDoS Attack)
Fourth – 81 (DDoS Attack)
Fifth – 21 FTP-control
With the exception of TCP 21, all other listed ports were part of a DDoS attack attempt.
Rob Thomas

15. TCP Probes and Attacks Trend Analysis
TCP port zero remains very popular with those who target this site.
TCP port 21 (FTP) probes have been supplanted by other probes, such as DNS.
Rob Thomas

16. Source Address of Probes and Attacks
Note that Multicast and experimental source address packets account for 53% of the naughty packets.
Thus 53% of the nasty packets are blocked at our border!
What about the other classes? Are the sources therein all legitimate?
Rob Thomas

17. Source Address of Probes and Attacks
Class A bogon source percentage – 48.08%!
Class B bogon source percentage – 20.80%.
Class C bogon source percentage – 11.87%.
Class D and E bogon source percentage – %.
Rob Thomas

18. Source Address of Probes and Attacks
Bogon source attacks still common.
Of all source addresses, 53.39% were in the Class D and Class E space.
Percentage of bogons, all classes – 66.85%!
This is good news – prefix-list, ACL defense, and uRPF will block 66.85% of these nasties!
Block bogons, both inbound and outbound, at your border!
Add anti-spoofing at your egress points so that only source addresses within your allocated netblocks may pass.
Three attack types:
Spoof the source using bogon addresses.
Spoof the source using legitimate addresses.
No spoofing.
If all networks performed these basic steps, how successful would spoof attacks be?
Rob Thomas

19. Source Region of the Naughty A dangerously misleading slide
The location of the miscreant is difficult to determine because of:
Spoof attacks.
Zombies.
Legacy netblock assignments.
Key point – the RIR or netblock does not necessarily indicate the location of the miscreant(s)!
Statistics can be misleading. Proper analysis requires a certain degree of “clue.”
Latest groups are from Brasil and Romania.
Rob Thomas

20. Intrusion (attempt) Detection
IDS is not foolproof!
Incorrect fingerprinting does occur.
You can not identify that which you can not see.
IDS is really a misnomer – it should be Intrusion Attempt Detection System, or IADS.
Intrusion Detection is easy – you will know your hull has been breached when your web page reads “Hackers love Ramen!” 
IDS can be overwhelmed. Stick is nothing new – we have likely all seen it before.
Keep your IDS database current!
Place your IDS tool where it will provide the best detection and analysis.
Rob Thomas

21. Top Five IDS Detected Probes
NetBus – TCP 12345
Backorifice – UDP 31337
TFTP – UDP 70
IDENT – TCP 113
Deep Throat – TCP/UDP 2140
Rob Thomas

22. Top Five Detected IDS Probes
Notice how the probes converge around day 13 and particularly days 26 and 27. This is not likely to be a coincidence!
Rob Thomas

23. Top Five IDS Detected Attacks
Rob Thomas

24. Top Five IDS Detected Sources
Azerbaijan – ISP
USA 01 – Consulting Company
South Korea – ISP
USA 02 – ISP dialup pool
Canada – Cable modem
Rob Thomas

25. Top Five IDS Detected Sources
Notice how netblock B (USA 01) trends sharply upward around days 24 through 25.
In the raw data spreadsheet, the sudden appearance of netblock B coincides with the sharp increase in probe activity as seen in the Top Five Probes slide.
Rob Thomas

26. Match a Source with a Scan
Rob Thomas

27. Rob Thomas robt@cymru.com
Two Days of DDoS
Attack that resulted in hits on day one and hits on day two.
Attack lasted 25 hours, 25 minutes, and 44 seconds.
Quasi-random UDP high ports (source and destination), small packets.
This was a relatively mild attack, and this makes for simple analysis. However, the analysis steps are the same regardless of attack intensity.
Rob Thomas

28. Rob Thomas robt@cymru.com
Two Days of DDoS
Perhaps as many as 2000 hosts used by the attackers.
23 unique organizations.
9 different nations located in the Americas, Europe, and Asia.
Source netblocks all legitimate.
The use of legitimate source addresses makes defense that much more difficult.
This was an extremely mild DDoS – perhaps a test of things to come.
Rob Thomas

29. Rob Thomas robt@cymru.com
Two Days of DDoS
As with all DDoS attacks, the ramp up time was quite fast – on the order of several seconds. The end came just as suddenly.
Rob Thomas

30. Rob Thomas robt@cymru.com
Two Days of DDoS
Note the rise and fall of myriad sources. This is not uncommon during even the most intense DDoS attacks, and can be due to several factors, such as:
Congestion at the attacker site(s)
Reactionary filtering (local and intermediate) during the attacks
Zombie control issues
Insert the IRC “out of control zombie story” here.
Rob Thomas

31. Site Defense and Attack Mitigation
While you can not prevent an attack, you can choose how to react to an attack.
Layers of defense that use multiple tools.
Layers of monitoring and alert mechanisms.
Know how to respond before the attack begins.
Each layer should protect the layer beyond.
Each layer should integrate with the layers on either side.
Each layer should be untrusted by the next site-facing layer.
Funnel the traffic, filtering at an ever more granular level of detail.
Rob Thomas

32. Site Defense and Attack Mitigation
Border router
Protocol shaping and filtering.
Anti-bogon and anti-spoofing defense (uRPF), ingress and egress filtering.
NetFlow.
IDS device(s)
Attack and probe signatures.
Alerts.
Block bogons and prevent spoofing!
The presence of an IDS device does not obviate the need to have the other filtering devices logging (ACL logs, rule base logs, NetFlow, etc.).
Compare log entries often – remember, NTP is your friend.
Rob Thomas

33. Site Defense and Attack Mitigation
Border firewall
Port filtering.
Logging.
Some IDS capability.
End systems
Tuned kernel.
TCP wrappers, disable services, etc.
Crunchy through and through!
Rob Thomas

34. Site Defense and Attack Mitigation
Don’t panic!
Collect data!
The good news - you can survive!
Rob Thomas

35. References and shameless self advertisements 
RFC
Secure IOS Template –
Secure BGP Template –
UNIX IP Stack Tuning Guide –
Rob Thomas

36. Rob Thomas robt@cymru.com
Any questions?
Rob Thomas

37. Rob Thomas robt@cymru.com
Thank you for your time!
Thanks to Jan, Luuk, and Jacques for inviting me to speak with you today.
Thanks to Surfnet/CERT-NL for picking up the travel.
Thanks for all of the coffee! 
Rob Thomas