Sendmail upgrade

It has been a long time coming, but my sendmail has been upgraded to 8.12.11.20060308 (Released 2004/01/18, patched 2006/03/08) from 8.11.6 (Released 2001/08/20), which I guess is progress. However, while this version supports MILTERs, many milters require Sendmail 8.13 or later. The latest stable version is 8.13.8 (Released 2006/09/08); I do not understand why Westhost have upgraded to an obsolete version.

Some of the milters I have looked at must be started before Sendmail and shutdown after it. I am not sure how we would achieve this as Sendmail is controlled at the Server level, rather than the VPS level where we have control

Also, some other users wanted SSL support. Issuing the command sendmail -d0.1 -bv tells us that Sendmail has been built with MILTER and SASL support, but not STARTTLS. This means that the username and password will be passed in the clear using Base64 Encoding rather than being encrypted. A pity.

Quoted from http://www.joreybump.com/code/howto/smtpauth.html

Make sure that sendmail was compiled with the necessary options:

sendmail -d0.1 -bv

STARTTLS and SASL must be present in the
resulting list.

Verify the location of sendmail.cf:

sendmail -d0.20 -bv | grep sendmail.cf

The output might look similar to this:

Conf file: /etc/mail/sendmail.cf (default for MTA)

Conf file: /etc/mail/sendmail.cf (selected)

Back up your configuration files:

cp /etc/mail/sendmail.cf /etc/mail/sendmail.cf~
cp /etc/mail/sendmail.mc /etc/mail/sendmail.mc~

Make your certificate (note that on Fedora Core 4 the location is now
/etc/pki/tls/certs):

cd /usr/share/ssl/certs
make sendmail.pem

Carefully edit the corresponding lines in

/etc/mail/sendmail.mc to match the following:

define(`confAUTH_OPTIONS', `A p y')dnl
TRUST_AUTH_MECH(`LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `LOGIN PLAIN')dnl
define(`confCACERT_PATH',`/usr/share/ssl/certs')
define(`confCACERT',`/usr/share/ssl/certs/ca-bundle.crt')
define(`confSERVER_CERT',`/usr/share/ssl/certs/sendmail.pem')
define(`confSERVER_KEY',`/usr/share/ssl/certs/sendmail.pem')

If you plan on using a DNSBL,
you should add this line:

FEATURE(delay_checks)dnl

The sendmail provided by Red Hat/Fedora Linux listens only on the loopback
interface (127.0.0.1) by default, so it will accept no outside connections. If
you haven’t already done so, comment out the related line. Change this:

DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl

To this:

dnl DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl

Generate sendmail.cf using the location you verified earlier, and restart
sendmail:

m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf

service sendmail restart

Configure the SASL authentication daemon to run when the server boots,
then start it:

chkconfig saslauthd on
service saslauthd restart

That’s it! You now have SMTP AUTH with encrypted logins!

Verbose mode

In order to provide encrypted logins, you must use a version of sendmail
that was compiled to use SASL and STARTTLS. The binaries provided with Red
Hat 9.0 and Fedora Core have been compiled with all of the necessary options.
In fact, Red Hat has done an excellent job in finally providing a version
of sendmail that is relatively easy to secure. Most of the of the options
mentioned are already present in the provided sendmail.mc, and merely need
to be uncommented.

Note: Users of Red Hat/Fedora Linux need to run the saslauthd
daemon. The following commands will start the daemon and ensure that it gets
launched at boot time:

chkconfig saslauthd on
service saslauthd restart

Now let’s go through each step:

Step 1: Back up important files

Red Hat has finally placed sendmail.cf in /etc/mail, where it belongs. To
verify the location of your configuration file, type this command:

sendmail -d0.20 -bv | grep sendmail.cf

The default installation outputs this:

Conf file: /etc/mail/sendmail.cf (default for MTA)
Conf file: /etc/mail/sendmail.cf (selected)

Be sure to use this path when generating your new sendmail.cf from sendmail.mc,
or no changes will take place. Back up your current sendmail.cf and the m4
file that generated it (probably /etc/mail/sendmail.mc):

cp /etc/mail/sendmail.cf /etc/mail/sendmail.cf~
cp /etc/mail/sendmail.mc /etc/mail/sendmail.mc~

Step 2: Make your certificate

We are also setting up STARTTLS, which allows sendmail to communicate
over an encrypted layer using TLS.
This is very important, as it allows us to use the LOGIN or PLAIN authentication
mechanisms without transferring the password in plain text. It also allows
the entire message to remain encrypted from the user’s machine to the mail
server. If sendmail relays the message to another server that offers STARTTLS,
the message will be encrypted again. But the most important advantage of
this approach is that we get to authenticate using regular system logins
and passwords, with no need to maintain a separate user database.

Red Hat’s openssl package includes a Makefile that makes it extremely easy
to generate a certificate (note that on Fedora Core 4 the location is now
/etc/pki/tls/certs):

cd /usr/share/ssl/certs
make sendmail.pem

Just follow the prompts and be sure to use the fully qualified domain name
of the mail server for the Common Name prompt. Users will still be warned
that the certificate is self-signed or not trusted, but you will prevent
a warning that the certificate doesn’t match the host offering it. This certificate
is suitable for testing, but you may want to investigate further about the
use of certificates before deploying it in a production environment, a topic
that is beyond the scope of this howto.

Step 3: Edit sendmail.mc

If you take a look at the sendmail.mc provided by Red Hat, you will notice
that the necessary directives are already present but have been commented
out (m4 doesn’t use the # symbol for comments, it starts a line with dnl,
which stands for "delete until new line"). Since we want the easiest
method possible, without sacrificing security, we need to edit these lines. Don’t cut & paste
from this web page, or you may introduce unwanted characters into your configuration
file that will prevent sendmail from starting.

The confAUTH_OPTIONS macro allows you to instruct sendmail not
to offer plain text authentication until after a secure mechanism such as
TLS is active (the p option). We are also prohibiting anonymous
logins (the y option). The A option is a workaround
for broken MTAs:

define(`confAUTH_OPTIONS', `A p y')dnl

Now we define which authentication mechanisms we will trust and use:

TRUST_AUTH_MECH(`LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `LOGIN PLAIN')dnl

Next, we tell sendmail where to find the certificates:

define(`confCACERT_PATH',`/usr/share/ssl/certs')
define(`confCACERT',`/usr/share/ssl/certs/ca-bundle.crt')
define(`confSERVER_CERT',`/usr/share/ssl/certs/sendmail.pem')
define(`confSERVER_KEY',`/usr/share/ssl/certs/sendmail.pem')

And finally, it may be useful to increase the log level for debugging purposes
(delete or comment out this line after everything is working properly):

define(`confLOG_LEVEL', `14')dnl

Use the m4 command to generate a new sendmail.cf:

m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf

Be sure to use the right location for sendmail.cf, as determined earlier.
Alternatively, you can use the following command in a stock Red Hat 9.0 or
Fedora Core installation:

make -C /etc/mail sendmail.cf

This uses the commands in /etc/mail/Makefile to generate the new sendmail.cf configuration file.

Step 4: Test the configuration

This is where things get really interesting. sendmail must be restarted
before it can use the new configuration file. Rather than simply restarting
sendmail with our fingers crossed, we can test it to verify that every thing
works properly. You can stop sendmail and then start it with command line
options that cause it to log to a specified file. There are various ways
to stop sendmail on a Red Hat/Fedora system:

service sendmail stop

or

cd /etc/mail
make stop

or

make -C /etc/mail stop

or

/etc/init.d/sendmail stop

We want to start sendmail with arguments to make it log the SMTP transaction
to a special file while we are testing it:

sendmail -bD -X /tmp/test.log

Now, try to send a message from an e-mail client on another computer that
does not have relay access, using your server as the outgoing mail server.
You should be denied relaying. Edit your preferences so that the client uses
authentication, with a login and password (not Secure Password Authentication,
or SPA, which is something completely different). You should still be denied
access. The last thing you need to do is to instruct the client to use SSL
or TLS with the outgoing mail server (there is no need to specify a special
port). After making this change, you should be able to send mail (you will
be prompted to accept the certificate, however, which you might want to install
to prevent further prompts). Now hit ctrl-c to stop sendmail.
Restart it normally:

service sendmail restart

Now it’s time to look at the log. After the first EHLO, sendmail
offers something like this:

30245 >>> 250-ENHANCEDSTATUSCODES
30245 >>> 250-PIPELINING
30245 >>> 250-8BITMIME
30245 >>> 250-SIZE

30245 >>> 250-DSN
30245 >>> 250-ETRN
30245 >>> 250-STARTTLS
30245 >>> 250-DELIVERBY
30245 >>> 250 HELP

The important thing is that AUTH is not offered here, because
the channel isn’t encrypted. If you see AUTH in the first exchange,
and it offers PLAIN or LOGIN, something is wrong.
Look at your logs, go over the previous steps, and make sure that you generated
a new sendmail.cf in the right location. The next entries in our log show
that TLS is activated:

30245 <<< STARTTLS
30245 >>> 220 2.0.0 Ready to start TLS

Another EHLO takes place, followed by something like this:

30245 >>> 250-ENHANCEDSTATUSCODES
30245 >>> 250-PIPELINING
30245 >>> 250-8BITMIME
30245 >>> 250-SIZE
30245 >>> 250-DSN

30245 >>> 250-ETRN
30245 >>> 250-AUTH LOGIN PLAIN
30245 >>> 250-DELIVERBY
30245 >>> 250 HELP

Now AUTH is offered with the allowed mechanisms (but not STARTTLS,
which isn’t needed here, as the channel is already encrypted). Authentication
takes place, and the message is relayed to its destination.

It’s interesting to note that the username and password is Base64 encoded
by the client, so it isn’t really sent as clear text:

30245 <<< AUTH PLAIN AHJvYmVydABzbHVncw==
30245 >>> 235 2.0.0 OK Authenticated

Nevertheless, it would be trivial to decode the string into the correct
username/login pair (robert/slugs, in this case). Therefore, it is best to
secure the transaction with TLS. If you want to verify that the transaction
is encrypted, open another terminal for root, and run tcpdump:

tcpdump -s 1500 -vvxX port 25

Send a mail with easy to identify strings. You shouldn’t see your login
or the message in tcpdump’s output. Note that the certificate will be exchanged
in plain text before TLS is enabled. If the mail is relayed to another server
that doesn’t offer STARTTLS, you will see the content of the
outgoing message in plain text.

Source Documentation

SMTP AUTH in sendmail
8.10-8.13
– The official documentation at sendmail.org.
Much of it concerns compiling cyrus-sasl and sendmail, steps that are not
necessary when using Red Hat/Fedora RPMs.

Configuring
Sendmail’s STARTTLS (SSL) and Relaying
– An excellent article by Jason
Heiss about STARTTLS and sendmail. He also explains how to allow relaying
based on certificates offered by clients.

My Experiences (So Far)
with STARTTLS and Sendmail
– Weldon Whipple shares his experiences
sorting out the intricacies of STARTTLS. Read this for a good introduction
to the way sendmail handles encryption and certificates.

RFC 2222: Simple Authentication
and Security Layer (SASL)

RFC 2487: SMTP Service Extension
for Secure SMTP over TLS

RFC 2554: SMTP Service Extension
for Authentication

Installing SHA1SUM

Westhost do not include SHA1SUM as part of their standard installation, so we cannot check the applications that we download have not been tampered with.

We therefore need to build our own copy of SHA1SUM. The source code for a cut down version can be obtained from gnupg.org

From the command line of your server

  • Type ftp ftp.gnupg.ogr to start a FTP session
  • Use username “anonymous” and password “anon@
  • Change to the required directory by issuing the command cd gnupg/binary
  • Download the source files using the commands get sha1sum.c and get sha1sum.c.sig
  • Close the FTP session using the command quit
  • If you installed GnuPG, verify the source file gpg –verify sha1sum.c.sig
  • Install the GCC Compiler Collection from your Site Manager if you have not already done so
  • Build the SHA1SUM executable by issuing the command cc -Wall -g -o sha1sum sha1sum.c
  • The SHA1SUM executable will now exist in the current directory. Move it to /usr/local/bin so that you can use it in the future. mv sha1sum /usr/local/bin

NOTE This version of SHA1SUM only calculates the SHA1 sum for the file specified; you have to visually check the sum against the supplied value for your download.

Procmail recipes

If you want to modify your procmail.rc rules file, then I suggest that you try your rule in isolation first before you break procmail and cause problems.
The first thing to do is to create the file with your test rule in it.

  • pico proctest.rc
    1. SHELL=/bin/sh
    2. TESTDIR=/home/rollingr/test
    3. MAILDIR=${TESTDIR}
    4. LOGFILE=${TESTDIR}/results.log
    5. LOG="--- Logging for ${LOGNAME}, "
    6.  
    7. #Troubleshooting:
    8. VERBOSE=yes
    9. LOGABSTRACT=all
    10.  
    11.  
    12. ##Insert your rule here instead of sample ##
    13. # Start of sample recipe to bounce blacklisted mail
    14. :0
    15. # Test whether required user is in any of these fields
    16. #Original-, Resent-, To, Cc, Bcc,.X-Envelope-To, Apparently-To or -Resent-To
    17. * ^TO_requireduser@mydomain.com
    18. # If match found then
    19. {
    20.   # Check if the sender is whitelisted
    21.   # Lock destination file for writing if match
    22.   :0:
    23.   # Extract headers From, Sender, Reply-To, Return-Path, To
    24.   # and compare to entries in white.list
    25.   # Entries should take form [email]me@my\.domain[/email]
    26.   # otherwise dot will match any character
    27.   # grep case insensitive, supress errors
    28.   # Click for a full explanation of [url=http://unixhelp.ed.ac.uk/CGI/man-cgi?egrep]egrep[/url]
    29.   * ? formail -x"From" -x"From:" -x"Sender:" \
    30.       -x"Reply-To:" -x"Return-Path:" -x"To:" \
    31.       | egrep -is -f white.lst
    32.   # If match found, deliver to /path/to/mailbox and stop
    33.   /path/to/mailbox
    34.  
    35.   # Bounce unwanted mail
    36.   # Start test - strip body from message
    37.      :0 h
    38.     # If not from mailing daemon
    39.     * ! ^FROM_DAEMON
    40.     # and not bounce of a bounce
    41.     * ! ^X-Loop: bounced@mydomain.com
    42.     # then pipe header with extra bounced header and error message in body to sendmail and stop
    43.     | ( formail -rt -A "X-Loop: bounced@mydomain.com"; \
    44.         echo "Your email address is not on our whitelist.  Please go to [url]http://www.mydomain.com/whitelist[/url] and subscribe"; \
    45.         ) | $SENDMAIL -t
    46.  
    47.   # If we got this far, then message is on blacklist and was previously bounced or is from the mailing daemon, so throw it away and stop
    48.   # Bin everything else sent to this user (such as bounces)
    49.   :0
    50.   /dev/null
    51. }
    52. # if not to required user then continue processing
    53.  
    54. ## End of sample Recipe ##
    55.  
    56.  
    57. # Catch anything that failed test Recipe here
    58. :0
    59. ${TESTDIR}failed.mail

    Now we need to create the supporting files

  • pico white.lst
    anyuser\@gmail\.com
  • Generate a valid message file which will trigger the rule
    1. From anyuser@gmail.com  Wed Nov 15 08:48:26 2006
    2. Return-Path: <anyuser@gmail.com>
    3. Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.170])
    4.         by mydomain.com (8.11.6/8.11.6) with ESMTP id kAFFmPd29195
    5.         for <me@mydomain.com>; Wed, 15 Nov 2006 08:48:25 -0700
    6. Received: by 10.66.252.9 with HTTP; Wed, 15 Nov 2006 07:48:23 -0800 (PST)
    7. Message-ID: <6757a1050611150748h6b0ada1fkf4dfe9aaefe721db@mail.gmail.com>
    8. Date: Wed, 15 Nov 2006 15:48:23 +0000
    9. From: "Sender" <anyuser@gmail.com>
    10. To: "Recipient" <me@mydomain.com>
    11. Subject: Testing Mail Header
    12. MIME-Version: 1.0
    13. Content-Type: text/plain; charset=ISO-8859-1; format=flowed
    14. Content-Transfer-Encoding: 7bit
    15. Content-Disposition: inline
    16.  
    17. This is a test for procmail
    18.  
    19. --
    20. Richard

    Next create a batch file to run the rule. Point TESTDIR to your directory.

  • pico procmail.bat
    1. #!/bin/sh
    2. #The executable file named "proctest"
    3. #
    4. # You need a test directory.
    5. TESTDIR=/home/yourlogin/test
    6. if [ ! -d ${TESTDIR} ] ; then
    7.   echo "Directory ${TESTDIR} does not exist; First create it"
    8.   exit 0
    9. fi
    10. #
    11. #Feed an email message to procmail. Apply proctest.rc recipes file.
    12. #First prepare a mail.msg email file which you wish to use for the
    13. #testing.
    14. procmail ${TESTDIR}/recipe.rc < mail.msg
    15. #
    16. #Show the results.
    17. less ${TESTDIR}/results.log
    18. #
    19. #Clean up.
    20. rm -i ${TESTDIR}/results.log
    21. rm -i ${TESTDIR}/failed.mail
  • And make it an executable
    chmod u+x procmail.bat
  • Run the test
    procmail.bat