1. A new Mailing List infrastructure at CERN Ruben Gaspar Aparicio Michel Christaller & Ruben Leivas Ledo IT - Internet Services Group CERN

2. CERN Mail Service HEPIX – October 2004 Schedule What is Listbox? What is Listbox? A bit of history A bit of history New service architecture and design New service architecture and design Externals & External Lists Externals & External Lists Conclusions Conclusions

3. CERN Mail Service HEPIX – October 2004 What is Listbox Listbox is the Mailing list service at CERN Listbox is the Mailing list service at CERN –It includes the Mailing list editor (http://simba.cern.ch) and the mailing list archive web site http://simba.cern.ch Numbers Numbers –3,300 lists –Active lists: 350 per day, 800 per week –8 Gb of web archives, with access control –60 users per list (avg), up to 6,000 users per list (max) –15,000 CERN users –35,000 addresses-without-CERN-account –1,500 list owners –Traffic: 2,000 (in) / 45,000 (out) messages distributed in 24 hours (avg)

4. CERN Mail Service HEPIX – October 2004 List properties Name of the list (email address) Name of the list (email address) Owners Owners Members Members Description Description Subscription and unsubscription policies Subscription and unsubscription policies Posting restrictions Posting restrictions Moderation Moderation Message size limit Message size limit An Archive may exist + authorization settings An Archive may exist + authorization settings Delivery policy Delivery policy Alias Alias

5. CERN Mail Service HEPIX – October 2004 Listbox4.cern.ch SUN computer Old Service SMTP Gateways Simba ( Apache Web interface) Majordomo Sendmail Mhonarc (Web archive) List definitions (file system) AFS File System HR People database LDAP CCDB Accounts database External internet Mailbox Stores

6. CERN Mail Service HEPIX – October 2004 Motivations for renewal 10 years of home grown software 10 years of home grown software High maintenance cost High maintenance cost –Dedicated team and special skills necessary –Single computer, non scalable architecture Service sometimes overloaded, delivery delays Service sometimes overloaded, delivery delays –Flat file database, AFS and LDAP dependencies Difficult mirroring of flat files for majordomo and ldap repository for SIMBA Difficult mirroring of flat files for majordomo and ldap repository for SIMBA –Archiving problems –Little monitoring and alarms (requires human monitoring) Missing functionalities Missing functionalities –anti-spam, anti-virus, anti-flood, expiration, invalid recipients removal, web archive management by owner, …

7. CERN Mail Service HEPIX – October 2004 Weaknesses Old System Weakness Listbox Weakness Listbox –Info shared into Listbox4 and LDAP Problems of spurious synchronization. Problems of spurious synchronization. Membership 0. Membership 0. UNIX like UNIX like –All is in files. –SIMBA deals with LISTBOX4 machine and LDAP server. –Breaking Messages. –Sometimes a message with a strange MIME type or with some awful expression inside (Perl can't cope with) is breaking "mhonarc" generating a core file in the Listbox4 archives directory. –A lot of different perl scripts for guarantee the Majordomo features. –Resend and bounce messages. –Member addresses resolutions.

8. CERN Mail Service HEPIX – October 2004 New Service HR / CCDB databases List definitions (Active Directory) New Simba (ASP.NET Web interface) New Listbox Service (load balanced array of PCs) ____ Mail Distribution (Exchange 2003 Gateways Spam, AntiVirus, Flood Check) ____ Web Archives (Exchange 2003 Public Folders) External Lists

9. CERN Mail Service HEPIX – October 2004 Exchange Infrastructure New Service External Lists HR People database CCDB Accounts database List definitions (Active Directory) Mail Distribution Public Folders (Web archive) Simba (ASP.NET Web interface)

10. CERN Mail Service HEPIX – October 2004 New Service SMTP Gateways (Spam, AntiVirus, Flood Check) Exchange 2003 HR database List definitions Active Directory New Simba (Web interface) External internet New Listbox Service (load balanced array of PCs) Webmail MailBox & Public Folders Exchange 2000/2003 Externals Lists

11. CERN Mail Service HEPIX – October 2004 New Listbox infrastructure Infrastructure shared with Mail Service Infrastructure shared with Mail Service All machines Windows 2000/2003, Exchange 2000/2003, Dual Xeon 2.0Ghz, Hyperthreading on, 1 Gbit/s network card All machines Windows 2000/2003, Exchange 2000/2003, Dual Xeon 2.0Ghz, Hyperthreading on, 1 Gbit/s network card Public folders Public folders –2x 4U servers, 4GB memory. Mainly Lists Archives –Databases are replicated Front End Servers Front End Servers –4x 2U servers, 2GB memory. IMAP, POP, MAPI over HTTP and HTTP (webmail) gateway Store Servers Store Servers –12x 4U servers, 3-4GB memory, 2 SRCU32 Intel RAID controllers with each 1xRAID1 (2x70GB/SCSI), 1xRAID5 (3x120GB/SCSI). SMTP Gateways SMTP Gateways –6x 2U servers, 2GB memory. Windows Load Balancing, Symantec Antivirus for Exchange and CERN made C# Protocol Event Sink Spam Content filtering servers (CERN SpamKiller) Spam Content filtering servers (CERN SpamKiller) –4x 2U servers, 2GB memory.

12. CERN Mail Service HEPIX – October 2004 New Listbox design User Objects (CERN accounts or Externals) User Objects (CERN accounts or Externals) Global security groups Global security groups –Owners –Members List ~ Public Folder with email @ List ~ Public Folder with email @ –Message sent to the PF Restriction on who can send messages Restriction on who can send messages Restriction on message size Restriction on message size Forwards to the group of members, possible delivery in the folder Forwards to the group of members, possible delivery in the folder –Web archive through OWA –Archive Permissions Owners Editor Owners Editor Members Non Editing Authors Members Non Editing Authors Owners can have more freedom for managing the archive. Owners can have more freedom for managing the archive. –delete and modify possible

13. CERN Mail Service HEPIX – October 2004 Archive access from Outlook

14. CERN Mail Service HEPIX – October 2004 From users perspective Almost the same interface for managing lists Almost the same interface for managing lists No change in email addresses No change in email addresses Almost the same functionalities Almost the same functionalities What was changed : What was changed : –Bounced mail goes to sender instead of owners –No more Majordomo mail commands Every subscription must be authenticated –No more AFS archives / AFS lists WEBDAV access / External lists web service

15. CERN Mail Service HEPIX – October 2004 CERN External Accounts People without a CERN mail account can register into Listbox People without a CERN mail account can register into Listbox Implemented as special user accounts with a mail address Implemented as special user accounts with a mail address Access restricted to Access restricted to –SIMBA interface –Web archives (OWA) With a group policy which denies access from the network / logon locally on all CERN computers but a few servers Logon with the mail address (userPrincipalName property) Logon with the mail address (userPrincipalName property)

16. CERN Mail Service HEPIX – October 2004 CERN External Accounts Account is created disabled when address is inserted in a list Account is created disabled when address is inserted in a list Account must be validated with a mail response Account must be validated with a mail response Unused accounts are reclaimed Unused accounts are reclaimed –MemberOf property –altrecipientBL Web interface to manage account properties (validation, password.. ) http://cern.ch/externalsweb Web interface to manage account properties (validation, password.. ) http://cern.ch/externalswebhttp://cern.ch/externalsweb External Accounts can be reused by other CERN Services External Accounts can be reused by other CERN Services

17. CERN Mail Service HEPIX – October 2004 Externals notification

18. CERN Mail Service HEPIX – October 2004 External Lists Replaces AFS lists Replaces AFS lists Membership is provided through a web service as a file Membership is provided through a web service as a file Email addresses are extracted and matched to user objects Email addresses are extracted and matched to user objects Web interface shows the original file content (retains comments) Web interface shows the original file content (retains comments)

19. CERN Mail Service HEPIX – October 2004 DEMO http://simba.cern.ch/ http://simba.cern.ch/

20. CERN Mail Service HEPIX – October 2004 Simba web site

21. CERN Mail Service HEPIX – October 2004 Owners View

22. CERN Mail Service HEPIX – October 2004 Members View

23. CERN Mail Service HEPIX – October 2004 Web archive

24. CERN Mail Service HEPIX – October 2004 Conclusion New Listbox integrated with new Mail infrastructure deployed at CERN last year New Listbox integrated with new Mail infrastructure deployed at CERN last year Benefits from Benefits from –Infrastructure redundancy –Antivirus –SPAM fight –Flood control –Integrated Web archive –Integrated repository : Active Directory Smooth migration undergoing Smooth migration undergoing –New interface in production (combines search on the old-new system) –450 lists already in the new system –shorter time in delivery for list in the new system Owners will have more functionality (manage archives, add/delete in bulk) Owners will have more functionality (manage archives, add/delete in bulk) Less SPAM hassle for owners (bounces, subscription) Less SPAM hassle for owners (bounces, subscription)

25. CERN Mail Service HEPIX – October 2004 Questions