Installing SHA1SUM

Westhost do not include SHA1SUM as part of their standard installation, so we cannot check the applications that we download have not been tampered with.

We therefore need to build our own copy of SHA1SUM. The source code for a cut down version can be obtained from gnupg.org

From the command line of your server

  • Type ftp ftp.gnupg.ogr to start a FTP session
  • Use username “anonymous” and password “anon@
  • Change to the required directory by issuing the command cd gnupg/binary
  • Download the source files using the commands get sha1sum.c and get sha1sum.c.sig
  • Close the FTP session using the command quit
  • If you installed GnuPG, verify the source file gpg –verify sha1sum.c.sig
  • Install the GCC Compiler Collection from your Site Manager if you have not already done so
  • Build the SHA1SUM executable by issuing the command cc -Wall -g -o sha1sum sha1sum.c
  • The SHA1SUM executable will now exist in the current directory. Move it to /usr/local/bin so that you can use it in the future. mv sha1sum /usr/local/bin

NOTE This version of SHA1SUM only calculates the SHA1 sum for the file specified; you have to visually check the sum against the supplied value for your download.

Configuring Sendmail

In order to modify Sendmail, you need to understand a little bit of m4. You can edit /etc/mail/sendmail.cf if you want to avoid the horrors of m4, but you will lose any changes if your host decides to modify one of the base files and rebuild your sendmail.cf

The components of sendmail.cf are

  • /etc/mail/sendmail.mc
    • /usr/share/sendmail-cf/m4/cf.m4
      • /usr/share/sendmail-cf/m4/cfhead.m4
        • /usr/share/sendmail-cf/sh/makeinfo.sh
        • /usr/share/sendmail-cf/ostype/$1.m4
        • /usr/share/sendmail-cf/mailer/$1.m4
        • /usr/share/sendmail-cf/domain/$1.m4
        • /usr/share/sendmail-cf/feature/$1.m4
        • /usr/share/sendmail-cf/hack/$1.m4
        • /usr/share/sendmail-cf/siteconfig/$1.m4
        • /usr/share/sendmail-cf/m4/proto.m4

$1 will be replaced by the m4 processor with the parameter passed using the keyword. For example OSTYPE(`linux’) causes the file /usr/share/sendmail-cf/ostype/linux.m4 to be included. Any changes we make will have to be made in /etc/mail/sendmail.mc as we do not have write access to any of the files in /usr/share on the Westhost VPS platform.

Structure of sendmail.cf

It is important that you maintain the order of your configuration file. The general rules are that the order should be:

VERSIONID
OSTYPE
DOMAIN
FEATURE
local macro definitions
MAILER
LOCAL_RULE_*
LOCAL_RULESETS

There are a few exceptions to this rule. Local macro definitions which influence a FEATURE() should be done before that feature. For example, a define(`PROCMAIL_MAILER_PATH’, …) should be done before FEATURE(`local_procmail’).

Beware: MAILER declarations should always be at the end of the configuration file, and MAILER(`smtp’) should always precede MAILER(`procmail’), and MAILER(`uucp’).

Adding Comments

You can include comments in your file, but it is imperative that you do not include keywords such as define or FEATURE in a comment as m4 will expand them. (A full list of m4 keywords can be found here). If you want the comment to appear in sendmail.cf, then start it with the # character. If you just want the comment to appear in sendmail.mc, then start it with dnl. For example
# This comment will appear in sendmail.cf
dnl but this one will not
# The first part of this comment will dnl and the rest will not
dnl # but none of this comment will appear in sendmail.cf

Diversions

m4 controls the generated output using diversions.What this means is that the order of the input files is not necessarily that of the output file. To quote from the m4 manual

When you start using m4, output is written to diversion zero, which is the standard output. When m4 reaches the end of the input, it writes out the contents of all the diversions in numerical order.

Use PUSHDIVERT ( new diversion ) and POPDIVERT to change the current diversion and restore it. For example:
PUSHDIVERT(9)
dnl Change logging level
undefine(`confLOG_LEVEL')dnl
define(`confLOG_LEVEL',`9')dnl
POPDIVERT

Making your changes

As I said earlier, the preferred place for changes at Westhost is in the /etc/mail/sendmail.mc file. Lets say that we wanted to add some headers to incoming email. We would add
dnl 15Nov06 Add extra mail headers for spam checking
dnl The Sender Address Relative to the recipient
HX-Envelope-From: $g
dnl Recipient's username - needed to check bcc
HX-Envelope-To: $u
dnl The incoming protocol used
HX-Protocol: $r

A full list of the available macros ($g, $u etc) are available here or on page 42 of the Sendmail Installation and Configuration guide

Compiling your changes

Once your changes are complete, you need to convert them into something that sendmail can understand. I recommend that you redirect the output to a new file so that you can check for changes before going live.

m4 sendmail.mc >sendmail.new
diff sendmail.cf sendmail.new

Installing Python

These are the commands you will have to issue from the command prompt on your server in order to download the necessary libraries and applications needed to build Python. I have piped all output from each stage into log.xxxx and err.xxxx files. Make sure that you read the err.xxxx files if they are not zero length and action any errors before proceeding to the next step. The information in log.xxxx will help you troubleshoot any errors.

As I do not have root access on Westhost, I have decided to create my own tree under /usr/mylocal for libraries and executables.

Download gdbm

  • ftp ftp.gnupg.org
  • login: anonymous password:anon@
  • cd gnu/gdbm
  • passive
  • dir gdbm-*
  • get gdbm-1.8.3.tar.gz
  • quit
  • gunzip -c gdbm-1.8.3.tar.gz |tar x
  • cd gdbm-1.8.3
  • ./configure –prefix=/usr/mylocal >log.config 2>err.config
  • make >log.make 2>err.make
  • make install >log.install 2>err.install
  • ls -l err.*
  • edit /etc/ld.so.conf to include /usr/mylocal/lib as first line
  • ldconfig
  • cd ..

Download ncurses

  • ftp ftp.gnupg.org
  • login: anonymous
  • cd /pub/gnu/ncurses
  • passive
  • dir ncurses*
  • get ncurses-5.5.tar.gz
  • get ncurses-5.5.tar.gz.sig
  • quit
  • gunzip -c ncurses-5.5.tar.gz |tar x
  • cd ncurses-5.5
  • ./configure –prefix=/usr/mylocal >log.config 2>err.config
  • make >log.make 2>err.make
  • make install >log.install 2>err.install
  • ls -l err.*
  • ldconfig
  • Run test/newdemo to check your installation. Press Q to stop.
  • cd ..
  • I was unable to link to ncurses/xxxxx.h files when building Python. As a workaround, you need to define these three symbolic links

  • ln -s /usr/mylocal/include/ncurses/ncurses.h /usr/mylocal/include/ncurses.h
  • ln -s /usr/mylocal/include/ncurses/curses.h /usr/mylocal/include/curses.h
  • ln -s /usr/mylocal/include/ncurses/panel.h /usr/mylocal/include/panel.h

Download Python

  • wget http://www.python.org/ftp/python/2.5/Python-2.5.tar.bz2
  • wget http://www.python.org/download/releases/2.5/Python-2.5.tar.bz2.asc
  • gpg –verify Python-2.5.tar.bz2.asc
  • bunzip2 -c Python-2.5.tar.bz2 |tar x
  • cd Python-2.5
  • ./configure –prefix=/usr/mylocal 1>log.config 2>err.config
  • make >log.make 2>err.make
  • make install >log.install 2>err.install
  • ls -l err.*
  • If /usr/mylocal/bin is not in your path, then you will need to create a symbolic link for it

  • ln -s /user/mylocal/bin/python /user/local/bin/python

GnuPG Cannot find Library libcurl

I have downloaded and installed GnuPG, but whenever I try to retrieve a public key from a keyserver, it logs the error
gpgkeys_hkp: error while loading shared libraries: libcurl.so.3: cannot open shared object file: No such file or directory

This is despite the fact that libcurl.so.3 is on my system
$ find / -name libcurl.so* 2>/dev/null
/usr/local/lib/libcurl.so
/usr/local/lib/libcurl.so.3
/usr/local/lib/libcurl.so.3.0.0
/usr/local/phplibs/lib/libcurl.so
/usr/local/phplibs/lib/libcurl.so.3
/usr/local/phplibs/lib/libcurl.so.3.0.0
/usr/home/mylogin/apps/bld/curl-7.16.0/lib/.libs/libcurl.so.4.0.0
/usr/home/mylogin/apps/bld/curl-7.16.0/lib/.libs/libcurl.so.4
/usr/home/mylogin/apps/bld/curl-7.16.0/lib/.libs/libcurl.so
/usr/mylocal/lib/libcurl.so.4.0.0
/usr/mylocal/lib/libcurl.so.4
/usr/mylocal/lib/libcurl.so
/ftp/usr/lib/libcurl.so
/ftp/usr/lib/libcurl.so.1
/ftp/usr/lib/libcurl.so.1.1.0
/ftp/usr/lib/libcurl.so.2

In fact it appears twice, once in /usr/local/lib and again in /usr/local/phplibs/lib.  So what is going on?  It would appear that while my system can locate the file in order to build it into the executable, it cannot locate the library at run time (when I run gpg); no errors were logged when I configured and made GnuPG. There is a useful article (here) by David Wheeler which explains how libraries work. It explains the difference between soname, realname and linker name and how the various links are created as part of the build process. It goes on to explain that when you install a new version of a library, you install it in one of a few special directories and then run the program ldconfig(8) in order to update the file /etc/ld.so.cache. These special directories are defined in /etc/ld.so.conf
When we look at ld.so.conf, it has two lines

[mylogin][~]$ cat /etc/ld.so.conf
/usr/local/lib
/lib

so the system should find the link in /usr/local/lib. (/lib is actually a symbolic link to /ftp/usr/lib) However, when I checked the cache, the file is missing

[mylogin][~]$ ldconfig -p | grep libcurl
libcurl.so.2 (libc6) => /lib/libcurl.so.2
libcurl.so.1 (libc6) => /lib/libcurl.so.1
libcurl.so (libc6) => /lib/libcurl.so

It appears that the script which installed php did not run the ldconfig utility to update the cache. This was resolved by running ldconfig manually.

[mylogin][~]$ ldconfig -v

My system has library files for libcurl in
/usr/local/lib
/usr/local/phplibs/lib
/ftp/usr/lib
and the version I built myself in /usr/home/mylogin/apps/bld/curl-7.16.0/lib/.libs and /usr/mylocal/lib

so the cache now has the following entries
[mylogin][~/apps/dl]$ ldconfig -p | grep libcurl
libcurl.so.4 (libc6) => /usr/mylocal/lib/libcurl.so.4
libcurl.so.3.0.0 (libc6) => /usr/local/lib/libcurl.so.3.0.0
libcurl.so.3 (libc6) => /usr/local/lib/libcurl.so.3
libcurl.so.2 (libc6) => /lib/libcurl.so.2
libcurl.so.1 (libc6) => /lib/libcurl.so.1
libcurl.so (libc6) => /lib/libcurl.so
libcurl.so (libc6) => /usr/mylocal/lib/libcurl.so
libcurl.so (libc6) => /usr/local/lib/libcurl.so

Note that the files from phplibs are not included directly (the libraries in /usr/local/bin are actually symlinks to phplibs) and that there are entries for /lib (which is itself a symlink to /ftp/usr/lib). I also edited /etc/ld.so.conf to include the directroy /usr/mylocal/lib.