1. Reducing Disk I/O of Messaging Systems
Kristina Ninan

2. Email Protocols Simple Mail Transfer Protocol (SMTP)
Post Office Protocol 3 (POP3)
Internet Mail Access Protocol (IMAP)

3. SMTP Mail Transfer Agent (MTA)
Used between sender computer and sender mail server
Used between sender mail server and receiver mail server

4. POP3 Mail Access Agent (MAA)
Used between receiver mail server and receiver computer
Allows receiver to access messages through client by downloading messages to receiver computer

6. Role of Disk I/O in Email
Outgoing messages are saved temporarily to a disk queue until they can be sent
“Deliver” command from the messaging server reads a message file from here and then tries to send it
“SMTP Data” command from the messaging server receives the incoming message, creates a new file on disk, and writes message to this file
After successful delivery, message object is deleted off disk, and can be moved to a different section of memory (i.e. sent folder)

7. Issues
Deliver and SMTP Data commands take up significantly more time than all other commands
Inefficient lookup time if all files are saved in a single directory with subdirectories

8. Issues Most email systems store one message per file → high overhead
Causes fragmentation because files are stored in non-adjacent blocks, which increases seek time and decreases throughput
Deleting files after delivery also increases the disk I/O and therefore overhead and fragmentation

9. Issues Majority of emails are small
Separate files for each message cause a large number of small files
Causes large number of small file writes
Small files can’t make full use of bandwidth
Disk I/O = one of the primary performance bottlenecks

10. Potential Optimizations
Need to optimize storage management
Reduce seek time by storing files in a multiple hash directory - introduced by Qmail
Qmail also write files to the directory in a round robin manner

11. SmartQ
Proposed by a research group at the University of Electronic Science & Technology of China
Aims to decrease the overhead in file management - especially file creation and deletion
Main idea: Implement disk queue using pre-assigned continuous disk space
Adjacent messages stored in adjacent blocks
Policies: Lazy Gathering Write Policy and Sequential Grouping Prefetching

12. SmartQ - Lazy Gathering Write Policy
Accumulate multiple messages in a large buffer in one large read and write them all at once into the disk queue

13. SmartQ - Lazy Gathering Write Policy
Takes full advantage of disk bandwidth
Reduces disk seek time and rotational latency
Multiple adjacent messages are written to continuous blocks

14. SmartQ - Sequential Grouping Prefetching Policy
Multiple adjacent messages are fetched into buffer in one read

15. SmartQ - Sequential Grouping Prefetching Policy
Takes full advantage of bandwidth
Number of message objects read is determined by the buffer size and can also be determined from the MIN table in memory

16. SmartQ - Message Index Node (MIN)
To improve read performance
Store a MIN table in memory which indexes messages waiting to be delivered
Allows message to be quickly located by mapping from the message ID to its location
Deleting a message after successful delivery = deleting MIN and setting delete flag in memory, which is write of size 1 byte
Decreases disk I/O
Decreases fragmentation

17. IBM User Specified Conditions Disk Management
In 2012 IBM proposed an improvement based on user specified conditions
If enabled, user can specify disk usage conditions based on information associated with incoming mail
This information can include the sender’s address, the header, attachment type, etc
If maximum quota is met, or a condition is violated, incoming message is trashed without being downloaded

18. IBM User Specified Conditions Disk Management

20. IBM User Specified Conditions Disk Management
Use counters to keep track of the size of the mail saved to disk
Reduce number of reads when trying to determine if mailbox is full/can hold incoming message
Increases server overhead but decreases disk I/O

21. Sources
Wei, Qingsong & Lu, Xianliang & Ren, Liyong & Zhou, Xu. (2003). A novel disk queue to reduce disk I/O of messaging system. Operating Systems Review /
French, Stephen M. et. al. Managing Disk Usage Based on User Specified Condiditions. 18 Oct
D. Schuff, O. Turetken, J. D'Arcy and D. Croson, "Managing Overload: Solutions and Future Challenges," in Computer, vol. 40, no. 2, pp , Feb doi: /MC &isnumber=

22. Sources
McDonald, Alistair, et al. Linux Set up and Run a Small Office Server. Packt Publishing, 2005.
Curtis, Derek. “Derek Curtis.” Server, Group Chat, Help Desk and Web Analytics Software, SmarterTools Inc., 16 May 2013,
Curtis, Derek. “Derek Curtis.” Server, Group Chat, Help Desk and Web Analytics Software, SmarterTools Inc., 23 May 2013, overall-performance-for-your-mail-server-part-2.

23. Sources
Curtis, Derek. “Derek Curtis.” Server, Group Chat, Help Desk and Web Analytics Software, SmarterTools Inc., 30 May 2013,
Unknown. “Difference Between SMTP and POP3.” Tech Differences, TechDifferences, 7 July 2017, techdifferences.com/difference-between-smtp-and-pop3.html.
Celine. "Difference Between SMTP and POP." DifferenceBetween.net. June 29, 2017 < >.

24. Sources
Unknown. “How the SMTP Service Works.” How the SMTP Service Works, Wrox Press, 1998, msdn.microsoft.com/en- us/library/aa aspx.

25. Questions?