Resililent Scalable SMTP Architecture

Originally published on 2010-01-30

Some thoughts on a resilient SMTP architecture


                                    Email Architecture Proposed
                                     ===========================

    Incoming MX's                      Single spamd,clamd              0utgoing mail server clusters
    Distributed geographically         greylistd,instance              Distributed geographically
    Distributed across netblocks       consistent email                Distributed across netblocks
    Configuration manager (puppet?)   filtering/spam blocking          Configuration manager (puppet?)
    ensures consistent config                                          ensures consistent config across hosts
    across hosts

       ---------------                                                     --------------                                                 
       |  incoming   |                                                     | Outgoing   |
 ----> |    MX       |<----|             -------------               ----->| Mail       |--------> 
       ---------------     |             |           |               |     | Cluster    |
 Email                     |             |   spamd   |               |     -------------- 
       ---------------     | vpn         | greylistd |     vpn       |                     Email 
 ----> |  incoming   | <---|-----------> |   clamd   |<--------------|                    Outgoing
       |    MX       |     | links       |           |    links      |     --------------
       ---------------     |             -------------               |     | Outgoing   | 
 Incoming                  |                                         |     | Mail       |-------->
       ---------------     |                                         ----->| Cluster    | 
       |  incoming   |<----|          Email Consumer/Producer              --------------
 ----> |    MX       |                      Backends                             ^
       ---------------                   ----------------                        | 
              |                          | list server  |                        | 
              |                   ------>|   instances  |------>|                | 
              | Accepted Email    |      ----------------       |                | 
              |------------------>|                             |--------------->|  
                  Delivered       |      ---------------        |
                                  ------>|  Mailbox    |------->|  Mail producers send to dedicated 
                                         |  Servers    |           per service outgoing cluster 
                                         ---------------           (prevents one service contaminating 
                                                                    another service reputation)