1. Empirical Analysis of Denial of Service Attack Against SMTP Servers Boldizsár BENCSÁTH, Laboratory of Cryptography and System Security (CrySyS) Budapest University of Technology and Economics http://www.crysys.hu/ this is joint work with Miklós Aurél RÓNAI

2. www.crysys.hu Laboratory of Cryptography and System Security 2 Spam is a REAL problem today blue: SPAMgreen: Normal e-mails (ham) time As of 03/27/2007 ~10-fold increase in number of spam

3. www.crysys.hu Laboratory of Cryptography and System Security 3 Why to measure performance?  Many e-mail servers are overloaded  Sometimes they even stop to respond or just restart (DoS situation)  We don’t know the performance of the server  As more and more spam arrives, we should expect more problems (DoS, etc.)  Can we deploy a successful DoS attack against the server easily?  Can we protect our server against DoS attacks?  How does the content filtering (virus- and spam filtering) affects the server?

4. www.crysys.hu Laboratory of Cryptography and System Security 4 Related work  Some information is available on performance, but it is not enough (sometimes no data on content filtering, or no real comparison, etc.)  Microsoft has a more complex test to calculate performance need in their Exchange architecture, but it is too complex. (we don’t want to analyze the performance of e.g. the calendar and address book)  Some information is available on the spam traffic and also some on DoS situations in SMTP servers, but not very informative  We tried to make some standard measurements to support our other research activity, e.g. DoS protection

5. www.crysys.hu Laboratory of Cryptography and System Security 5 The performance testing of SMTP is complicated  It’s hard to send e-mails (complicated application, network connectivity can hang, resource consumption is high for random emails (needed for testing spam engine))  It’s hard to coordinate the sending (starting the same time)  It’s hard to measure successful transfers  SMTP delivery it a multi-step process (explained later)  Too much overload can cause server to hang  Different SMTP servers can work in very different ways

6. www.crysys.hu Laboratory of Cryptography and System Security 6 SMTP delivery  The SMTP server gets a new message  The server puts it into a temporary queue -Sometimes it delivers without the temporary queue  The server sometimes/always/immediately/later checks the temporary queue and finds the e-mail -If the target server (or e.g. content filtering) is not responding, retry and retry timeout can occour -If the server is overloaded, the delivery process can be delayed  The server tries to deliver the message (or start content filtering) -Sometimes content filtering results in a new email (dual smtp setup)

7. www.crysys.hu Laboratory of Cryptography and System Security 7 Phases of delivery during the test  Phase 1: SMTP server receives and delivers messages  Phase 2: SMTP server receives into temporary queue, no or low speed delivery  Phase 3: no SMTP mails received, just delivery from the queue  Our test server: 2800MHz, 1GB RAM

8. www.crysys.hu Laboratory of Cryptography and System Security 8 What was our approach  Load the server fully, but not to overload too much  Messages generated on multiple computers to avoid problems by resource problems on the tester computers  Coordination by IRC based ‘botnet’ architecture  Sending a large number of emails and measuring the delivery time (and ensuring that the server runs under full load by delivery parameters)  Tried to measure performance in the different phases of delivery  Standard SMTP servers with standard content filtering  QMAIL, Postfix, Sendmail, Exim, MS Exchange  Amavisd-new, ClamAV-daemon, Spamassassin

9. www.crysys.hu Laboratory of Cryptography and System Security 9 Botnet

10. www.crysys.hu Laboratory of Cryptography and System Security 10 First results, no content filtering

11. www.crysys.hu Laboratory of Cryptography and System Security 11 Some data on delivery process

12. www.crysys.hu Laboratory of Cryptography and System Security 12 Delivery rate, without content filtering

13. www.crysys.hu Laboratory of Cryptography and System Security 13 Exim and queue_only_load  Queue_only_load parameter stops delivering e-mails if the load average is too much  Without this paramter, delivery is continuous throughout the test  Results show that for the performance, this behaviour is not very important  Of course the parameter is important, e.g. to avoid DoS

14. www.crysys.hu Laboratory of Cryptography and System Security 14 Exim with content filtering Clamscan is not daemonized, Clamd is daemonized == always in memory Clamd is clearly faster Performance is down from 30 to 6.81/4.03 e-mails/sec

15. www.crysys.hu Laboratory of Cryptography and System Security 15 Exim, virus+spam filtering Performance is down from 30->6->1.58 messages/sec

16. www.crysys.hu Laboratory of Cryptography and System Security 16 DoS? DoS! Test messages: body size of 4kb, random text Exim, virus+spam filtering = 1.58 e-mails/sec 1.58e-mails/sec*4kb=6.32 kb/sec payload, ~50kbps, with overhead ~64kbps Using only 64kbps we can overload an SMTP server with content filtering!

17. www.crysys.hu Laboratory of Cryptography and System Security 17 Future work  Our tests can be easily extended -other SMTP servers (e.g. kerio, mailgate etc.) -other content filtering tools (mailscanner, milters, COTS tools and products) -other spam engines (dspam, commercial products) -different parameter settings (e.g. spamassassin tests) -test e-mail parameters (attachments, size) (now random text 4kb) -other test approach (testing under heavy/low load etc.) -testing ‘appliance’ solutions. The main goal is completed: some basic information is now available about the possibility of overloading (DoS) and the performance of the server

18. www.crysys.hu Laboratory of Cryptography and System Security 18 Thanks Thank You! http://www.crysys.hu boldi@crysys.hu