Rolling Resource

I have just spent ages trying to resolve this problem, digging deep into openssl, certificates, PEM and the wonderful world of encryption, only to find that the answer was staring me in the face! Just goes to show how important it is to enter the right search terms into Google….

I have the following entry in my .fetchmailrc file
poll secure.server with proto POP3
user username@myisp.com there with password xxxx is localname here smtphost mydomain.com keep ssl sslcertck sslcertpath "/usr/local/ssl/certs"
Replace secure.server, username@myisp.com, xxxx, ,localname, mydomain.com as required and enter the correct path to your .pem files and hashes.

Everytime I tried to connect to the server, I got the following output

[mylogin][~]$ fetchmail -kv secure.server
fetchmail: 6.3.5 querying secure.server (protocol POP3) at Tue Oct 31 07:41:02 2006: poll started
fetchmail: getaddrinfo(“secure.server”,”pop3s”) error: Servname not supported for ai_socktype
fetchmail: Try adding the --service option (see also FAQ item R12).
POP3 connection to secure.server failed: No such file or directory
fetchmail: 6.3.5 querying secure.server (protocol POP3) at Tue Oct 31 07:41:12 2006: poll completed
fetchmail: Query status=2 (SOCKET)
fetchmail: normal termination, status 2

I thought that maybe the problem was with the certificate for secure.server (in my case secure.cnc.net) as it could not be validated – it uses Authority Information Access (AIA) to resolve its root certificate using OCSP – URI:http://ocsp.verisign.com. This was a red herring and threw me off on a wild goose chase. The real problem was this error getaddrinfo(“secure.server”,”pop3s”) error: Servname not supported for ai_socktype

This means that the service pop3s is not defined in /etc/services and that neither fetchmail nor openssl know which port to connect to. As soon as I added these lines to /etc/services, everything proceeds as expected (I still have to resolve the OCSP problem)

pop3s 995/tcp
imaps 993/tcp
ldaps 636/tcp

Here is the updated output from fetchmail

fetchmail -kv secure.server
fetchmail: 6.3.5 querying secure.server (protocol POP3) at Thu Nov 2 07:50:58 2006: poll started
Trying to connect to xxx.xxx.xxx.xxx/995…connected.
fetchmail: Issuer Organization: RSA Data Security, Inc.
fetchmail: Unknown Issuer CommonName
fetchmail: Server CommonName: secure.server
fetchmail: secure.server key fingerprint: 96:C6:81:FF:A7:22:2F:6D:F5:60:F8:8F:CE:2D:F0:5F
fetchmail: POP3< +OK POP3 Server Ready.
fetchmail: POP3> CAPA
fetchmail: POP3< -ERR Unknown command
fetchmail: Unknown command
fetchmail: Repoll immediately on username@myisp.com@server.myisp.com
Trying to connect to xxx.xxx.xxx.xxx/995…connected.
fetchmail: Issuer Organization: RSA Data Security, Inc.
fetchmail: Unknown Issuer CommonName
fetchmail: Server CommonName: secure.server
fetchmail: secure.server key fingerprint: 96:C6:81:FF:A7:22:2F:6D:F5:60:F8:8F:CE:2D:F0:5F
fetchmail: POP3< +OK POP3 Server Ready.
fetchmail: POP3> USER username@myisp.com
fetchmail: POP3< +OK Password required for username@myisp.com.
fetchmail: POP3> PASS *
fetchmail: POP3< +OK username has 0 message(s) (0 octets).
fetchmail: POP3> STAT
fetchmail: POP3< +OK 0 0
fetchmail: No mail for username@myisp.com at secure.server
fetchmail: POP3> QUIT
fetchmail: POP3< +OK Pop server at illustrious signing off.
fetchmail: 6.3.5 querying secure.server (protocol POP3) at Thu Nov 2 07:51:00 2006: poll completed
fetchmail: normal termination, status 1

Some files have been signed by their author to prove that nobody else has tampered with them. This is particularly true of source code or appllications you have downloaded off the web.

For example the antivirus package clamav can be downloaded from Sourceforge. There will be two files for you to download clamav-x.x.x.tar.gz and clamav-x.x.x.tar.gz.sig. The first file is the source code in a compressed format and the second is the signature for the format. In order to verify the signature, you will need GnuPG or PGP installed on the computer you are going to download the file to. I have given details of how to compile GnuPG elsewhere in this blog.

In order to verify the signature, we use the command gpg --verify clamav-x.x.x.tar.gz.sig

[mylogin][~]$ gpg --verify clamav-x.x.x.tar.gz.sig
gpg: keyring `/home/mylogin/.gnupg/secring.gpg’ created
gpg: keyring `/home/mylogin/.gnupg/pubring.gpg’ created
gpg: Signature made Mon Mar 13 17:44:03 2006 MST using DSA key ID 985A444B
gpg: Can’t check signature: public key not found

The trouble is that we do not have yet the public key for the person who signed the file. We can get this from a public keyserver such as keyserver.pgp.com or pgp.mit.edu. You will need the key ID printed in the penultimate (last but one) line above. In this case 985A444B.

We should be able to use the command gpg --keyserver pgp.mit.edu --recv-keys 0x985A444B to retrieve the key automatically, but get the error
[mylogin][~]$ gpg --keyserver pgp.mit.edu --recv-keys 0x985A444B
gpg: requesting key 985A444B from hkp server pgp.mit.edu
/usr/mylocal/libexec/gnupg/gpgkeys_hkp: error while loading shared libraries: libcurl.so.3: cannot open shared object file: No such file or directory
gpg: no handler for keyserver scheme `hkp’
gpg: keyserver receive failed: keyserver error

So we downloaded the public key from pgp.mit.edu instead and saved it to the file clamav.key. NOTE: In order to lookup the public key, you will need to add ’0x’ (zero x) to the front of the key ID, so 985A444B becomes 0x985A444B. The public key can be a large file, so copy and paste it rather than trying to type it.
[mylogin][~]$ gpg --import clamav.key
gpg: key 985A444B: public key “Tomasz Kojm ” imported
gpg: Total number processed: 1
gpg: imported: 1
gpg: no ultimately trusted keys found

Once we have added the key to our public keyring, we can verify the signature [mylogin][~]$ gpg --verify clamav-0.90RC1.1.tar.gz.sig
gpg: Signature made Mon Oct 16 02:56:15 2006 MDT using DSA key ID 985A444B
gpg: Good signature from “Tomasz Kojm “
gpg: aka “Tomasz Kojm “
gpg: aka “Tomasz Kojm “
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 0DCA 5A08 407D 5288 279D B434 5482 2DC8 985A 444B

Note that we still do not trust this key – we have just proven that the downloaded file has not been tampered with.

Step 1 – Download source

The first thing we need to do is download the source file from GnuPG.  I like to download the files directly to my server rather than downloading them to my PC and then uploading them to my server, so lets use a SSH session.  There are plenty of SSH Clients available, but PuTTY is available for free and does the job.  Instructions for configuring PuTTY for Westhost are available here.  Once you have logged in, create a directory to download your files to (for example, apps/dl) using the mkdir
[mylogin][~]$mkdir apps
[mylogin][~]$mkdir apps/dl

The files we need are only available via FTP from GnuPG.  If you clicked on the link to their site above, you will have found the latest version – at the time of writing this is 1.4.5.  Hover your mouse over the FTP link and you will find that the file we want to download is located at ftp://ftp.gnupg.org/gcrypt/gnupg/gnupg-1.4.5.tar.bz2, that is to say the file gnupg-1.4.5.tar.bz2 is located on the server called ftp.gnupg.org in the directory gcrypt/gnupg Follow the example below to download your own copy (user input is in bold).  It is good practice to download the signature file gnupg-1.4.5.tar.bz2.sig as well. 

If you prefer, you can download the gzipped file gnupg-1.4.5.tar.gz instead of gnupg-1.4.5.tar.bz.  You do not need both sets, although I download them in the example below.  GZip is just an older piece of software.  The command to extract the files would be gunzip instead of bunzip2

Step 2 – Checking the download

Now that we have the source files on our server, we need to check that they have not been tampered with.  The simplest way is to issue the command bunzip2 -t gnupg-1.4.5.tar.bz2

[mylogin][~/apps/dl]$ bunzip2 -tv gnupg-1.4.5.tar.bz2
gnupg-1.4.5.tar.bz2: ok

However, this only checks that the archive is not corrupt.  You really need to check the file’s digital signature, but we don’t have any means to do that yet.  We can verify the checksum but GnuPG only publish the SHA1 checksum for the code and we do not have sha1sum installed on our server.  However, I can reveal that the MD5sum is

811525965b4c0987e6418a7729a6444d gnupg-1.4.5.tar.bz2

If you save this text to the file gnupg-1.4.5.tar.bz2.md5, then enter md5sum -c gnupg-1.4.5.tar.bz2.md5 on the command line, you should get this response

[mylogin][~/apps/dl]$ md5sum -c gnupg-1.4.5.tar.bz2.md5
gnupg-1.4.5.tar.bz2: OK

If there was a problem with your download, you will get the following error message when testing the archive

bunzip2: gnupg-1.4.5.md5: data integrity (CRC) error in data

You can use the `bzip2recover’ program to attempt to recover
data from undamaged sections of corrupted files.

Step 3 – Other libraries

We need to download and install the libcurl, libiconv and gettext libraries before we can build GnuPG.  Also, since we do not have root access at Westhost, we need to tell all Linux packages to install to a different directory other than /usr/local.  We do this by specifying

./configure --prefix=/usr/mylocal
NOTE: Create the directory /usr/mylocal if it does not already exist

libiconv and gettext depend on each other, so we need to build libiconv then gettext and the rebuild libiconv.

Build libcurl

wget http://curl.haxx.se/download/curl-7.16.0.tar.bz2
wget http://curl.haxx.se/download/curl-7.16.0.tar.bz2.asc
bunzip2 -c curl-7.16.0.tar.bz2 | tar x
cd curl-7.16.0
./configure --prefix=/usr/mylocal
make
make install
cd ..


Build libiconv

wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.11.tar.gz
wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.11.tar.gz.sig
gunzip -c libiconv-1.11.tar.gz | tar x
cd libiconv-1.11
./configure --prefix=/usr/mylocal
make
make install
cd ..


Build gettext

ftp mirrors.usc.edu
 anonymous
 anon@
 cd pub/gnu/gettext
 mode image
 passive
 get gettext-0.16.tar.gz
 get gettext-0.16.tar.gz.sig
 quit
gunzip -c gettext-0.16.tar.gz |tar x
cd gettext-0.16
./configure --prefix=/usr/mylocal
make
make install


Rebuild libiconv

cd ../libiconv-1.11
make distclean
./configure --prefix=/usr/mylocal
make
make install
libtool --dry-run --finish /usr/mylocal/lib
libtool --finish /usr/mylocal/lib
cd ..


Step 4 – Creating the Executable

bunzip2 -c gnupg-1.4.5.tar.bz2 |tar x
cd gnupg-1.4.5
./configure --prefix=/usr/mylocal
make
make install


Once you have built GnuPG, you will need to create a few links as our directory is not in the PATH.

ln -s /usr/mylocal/bin/gpg /usr/local/bin/gpg

Alternatively, edit the PATH definition in /.bashrc to include /usr/mylocal/bin

You will also need to copy the configuration file
mkdir ~/.gnupg
cp -p /usr/mylocal/share/gnupg/options.skel ~/.gnupg/options

Boot

Processor

System

FileSystem

Network

Sometimes you cannot see the information you want because of all the error messages swamping the screen, or you want to manipulate the output in some way. So what to do? The answer is to redirect the output away from the screen to either the bit bucket or a file.

In Linux, there are three standard devices for input and output – Standard Input (STDIN), Standard Output (STDOUT), and Standard Error (STDERR). These can be referred to as 0, 1 and 2 respectively, so to redirect the output for STDERR to /dev/null (the bit bucket) you would type 2>/dev/null at the end of your command line.

For example find / -name fred 2>/dev/null results in any errors being discarded, leaving you with the list of files called fred displayed on the screen.

If you wanted to, you could redirect the output for STDERR to the same place as STDOUT by typing 2>&1. Note the & – if you don’t put that there, then the output for STDERR would be redirected to a file called 1.  By default, if you don’t name or number your redirection explicitly, then you’re talking about STDOUT, so find / -name fred > somefile.txt redirects the output for STDOUT to somefile.txt

The final alternative is to redirect the output streams to different places. For example find / -name fred >foundfiles.txt 2>/dev/null would send the list of files called fred to foundfiles.txt while discarding any errors reported by find.

You can, of course, redirect the standard output to the bit bucket and keep the errors if that’s what you really want to do!