SKS (Synchronising Key Server) Setup

The documentation wiki for SKS has been down for a while and building it on Ubuntu Dapper turned out to require a couple of minor tweaks, so this is my crib sheet for how to get it up and running. It's a compilation of wiki content extracted from Archive.org, the README, and my own notes.

Note: Thanks to Andy Ruddock there is now a guide to installing from the Ubuntu packages online at Installing the OpenSKS keyserver.

Useful References

Build SKS

apt-get install ocaml-dev

FIXME

wget tar zxf sks- cd sks-

FIXME

mv Makefile.local.au Makefile.local

Edit Makefile.local: change "libdb-4.1" to "libdb4.2";

change "BDB=-L/usr/lib" to "BDB=-L/usr" make dep
make all
sudo -s
make install
mkdir /sks
cd /sks
mkdir dump
cd dump

Pre-Populate Database

Rather than starting with an empty database and attempting to populate it by syncing with other keyservers (a bad idea because it loads up your peers with lots of traffic and will probably fail anyway with deadlocks in the conflict resolution system) we'll grab a static dump from an existing SKS server. There are a couple of sources out there, including:

ftp://ftp.pramberger.at/services/keyserver/keydump/
ftp://ftp.prato.linux.it/pub/keyring/
http://nynex.net/keydump/

Download Keydump

The keydump is about 3.3GB as of October 2007, so fetching it will take a long time. It's divided into a bunch of individual numbered files so you'll need to fetch all of them. Because I'm too lazy to spend 8 hours sitting there doing it manually I did it like this:

mkdir /sks/dump;
cd /sks/dump;
wget --recursive --timestamping --level=1 --cut-dirs=3 --no-host-directories ftp://ftp.pramberger.at/services/keyserver/keydump/

Many hours later, check that all the pieces downloaded correctly by comparing their checksums against the list published by the dump provider:

md5sum -c MD5SUMS

Build Local Database

There are two ways to do this: either a full build (which reads in the dump you just downloaded and leaves you with a complete, self-contained database) or a fastbuild (which just references the dump and requires it to be left in place after the fastbuild is complete). I started doing a full build, it looked like it was going to take forever so I aborted it and switched to a quickbuild. On the 4-processor machine I was using it still took in the order of 40 minutes to run so this might take a while.

You need to be in the basedir when running this and the dumps have to be in a sub-directory called "dump" (which they should be if you followed the steps above), so:

cd /sks
/usr/local/bin/sks_build.sh

If you edit the sks_build.sh script you'll discover it's just a shell script which calls SKS itself to do the heavy lifting. If you have trouble with lack of memory you may need to tweak the script a bit: in particular the "-n 10" flag used in the fastbuild call is a multiple of 15,000 keys to load at a time. The default setting therefore loads 150,000 keys at a time which could cause your machine to go into swap, and changing to something like "-n 2" will cause it to load only 30,000 at a time instead and possibly complete the job faster. The trick is to load as many as possible in each pass without hitting swap - if that happens, performance falls through the floor and you may as well abort it and start again (after deleting the KDB and PTree directories created by the aborted import).

If all goes smoothly you'll end up with KDB and PTree directories in /sks.

Create SKS Config File

/etc/sksconf is the main config file which is read on startup. This is my actual config file with my values, so change to suit your requirements:

# sks configuration for keys.keysigning.org
# Set the logfile to $basedir/log.[db|recon]
logfile: log

# debuglevel 4 is default (max debug level is 10)
debuglevel: 4

# Set the hostname of the server
hostname: keys.keysigning.org

# Set the email address we use to sync with other keyservers
from_addr: keysync@keys.keysigning.org

Set Up Peer Communications

It's important that all database changes made on individual SKS and PKS keyservers propogate not only to other servers but also across both networks. Keyservers (generally) should not operate in isolation: each one needs to be part of the mesh so it will receive all changes made on other servers and its own changes will be distributed as well.

SKS Gossip

Internally the SKS network uses a "gossip" protocol to propagate changes around, so as long as your keyserver is gossiping with other servers (your "peers") which in turn gossip with others etc etc then you'll be fine internally and all changes will eventually make it to and from your server. You don't need to be gossiping with every other SKS server, just a couple: as long as a chain can be traced from your keyserver through to any other keyserver, you're set. Changes can then "ripple" through the keyserver network until they touch every server.

You need to find other servers for your new server to gossip with, and put an entry for them in your /sks/membership file like this:

# List of other keyservers to peer with
sks.pkqs.net               11370
keys.keysigning.org        11370 # Jonathan Oxer <jon@oxer.com.au> 0x64011A8B
keyserver.gingerbear.net   11370
keyserver.oeg.com.au       11370
zimmermann.mayfirst.org    11370
# EOF

In the example snippet above you'll see that most of the lines have only a hostname and a port. That's the absolute minimum information you need to include for SKS to operate correctly. However, you'll see that one of the entries has a trailing comment that includes the name, email address, and key ID of the admin for that particular keyserver: that's the "best practice" format for a complete entry, and provides you a handy reference for how to get in touch with the operator of that keyserver if you ever have problems with peering.

Once again the snippet above is part of an actual "membership" file with real values in it, so it's only to provide an example of the format. It's up to you to find other keyserver operators to agree to peer with you so make sure you use their hostnames instead and don't just copy the example entries above.

The best approach for finding peers is to ask on the sks-devel mailing list with an email something along these lines:

Hi All,

I have a new keyserver running and would like to peer with other
servers. Please add me to your 'membership' file with the following
entry and provide your details in return so I can do the same:

keys.example.com  11370 # Joe Bloggs  0xDEADBEEF

Thankyou.

Of course you need to substitute your own details in the email.

Note: Once SKS is running it will automatically pick up changes to the membership file, so you don't need to restart the daemon if you add or remove gossip peers. By default SKS re-reads the membership file every 6 hours but you can override that by putting an entry in your sksconf file like this:

membership_reload_interval: 2

to change the interval to (for example) 2 hours. My understanding (possibly wrong!) is that SKS performs a DNS lookup to resolve hosts in the membership file only on each reload, so having it reload the membership file regularly is a good idea even if you aren't making frequent changes to it. That way SKS will be able to find any peers that have changed their IP address.

PKS Email Sync

The PKS network operates differently: it uses email notifications for internal updates, so to make sure all changes made on your server reach the PKS network you need to send out sync emails to a cooperative PKS server. SKS servers will only send sync emails for changes that were initiated locally (otherwise the PKS network would get a mail-storm of notifications for every change that propagates through the SKS network!) so having other SKS servers to gossip with is *not* enough to get changes out to the PKS network. You must also set up mailsync.

Put something like this in /sks/mailsync:

# Send an email notification to this server when a key is added so that
# all new keys propagate to the PKS network
#pgp-public-keys@keyserver.kjsl.com
#pgp-public-keys@dbma.keyserver.ca

You'll need to add the (uncommented) email addresses of your PKS peers, but don't just do this without their permission or they might get upset. You'll probably only need one PKS peer: I can't actually remember why I ended up with two.

Start DB Server

Finally time to start it up! Amazingly this is the point where I really came unstuck because nowhere in the README or on the archived wiki does it tell you what you need to do to start the actual server. I don't know if this is technically correct, but some poking around in the source led me to this and it seems to work so please let me know if this is not correct.

The main sks binary wraps a whole bunch of operations including the DB build stuff we just did. One of the operations is "db", which starts the main db server:

cd /sks
/usr/local/bin/sks db &

So that you can see what's going on, put a tail on the db logfile:

tail -f /sks/db.log

By default your new server will be running on port 11371 (the normal PGP/GPG server port) so edit your ~/.gnupg/options file on your workstation and select your keyserver with an entry like:

keyserver 1.2.3.4

where 1.2.3.4 is the IP address or hostname of your keyserver.

Then do something like:

gpg --recv-keys 64011A8B

and you should successfully pull down the key, as well as see the action appear in the DB log. Success! You now have a (partly) functioning keyserver.

Start Reconciliation Server

Once again this is just my interpretation so I may be wrong, but the next step seems to be starting the recon server that handles synchronising with other servers (ie: the "gossip" functionality mentioned previously).

cd /sks
/usr/local/bin/sks recon &

Check the operation with a tail on the recon logfile:

tail -f /sks/recon.log

Process Incoming Sync Email

Changes made on the PKS network also need to find their way back to the SKS network via email. However, it's less critical to handle incoming PKS email because as long as at least one keyserver on the SKS network receives a PKS update it will be propagated through the rest of the SKS network via the usual "gossip" mechanism.

There are two common methods for processing incoming PKS updates, both relying on a script called sks_add_mail which is distributed as part of SKS.

/etc/aliases file

Most Linux mail transport agents (MTAs) use the /etc/aliases file (or similar) as a system-level mapping of virtual email addresses to recipients. You can use the aliases file to redirect all incoming email for a specific address to the sks_add_mail script by adding an entry similar to this:

pgp-public-keys: "|/usr/local/bin/sks_add_mail /sks"

That entry will cause any email sent to the address "pgp-public-keys@yourdomain" to be redirected to the "sks_add_mail" script, which is also given the path to your SKS directory so that it can find the database to perform the update.

.procmailrc

If you already have a user on your server with the correct name and your MTA is configured to use Procmail, you can simply put a .procmailrc file in that user's home directory to have all mail with a subject starting with "incremental" (which PKS updates do by convention) redirected to the sks_add_mail script. For example:

PATH=/usr/local/bin
:0
* ^Subject: incremental
| sks_adde_mail /sks

Server Statistics

SKS generates database statistics internally and makes a report available with details of all peers and the internal stats. Because generating the report puts quite a heavy load on the server it's only done once each day.

You can access the stats for your server by pointing a browser at "/pks/lookup?op=stats" on port 11371, similar to this:

http://keys.keysigning.org:11371/pks/lookup?op=stats

Because the stats are generated daily you may find it annoying not to have them available for a while if you restart the DB daemon. You can get around this by forcing the daemon to do a full stats run immediately on startup with the -initial_stat option, like this:

cd /sks
/usr/local/bin/sks db -initial_stat &

Starting up will take longer, but you'll then have a stats report available right away.

Firewall Issues

If you run a firewall in front of your keyserver (or iptables rules directly on it) you need to make sure certain ports are open. The critical ports are:

TCP 11370 (for reconciliation with other keyservers)
TCP 11371 (for HTTP connections)

Port 11371 is a well-known registered port, but use of 11370 for reconciliation is only a convention and is configured in the membership file on a per-peer basis. I've never seen a peer use anything except 11370 for reconciliation, but keep in mind that it is possible and if so you'll need to open the appropriate port through your firewall.

SKS Network Status

The status of all known hosts in the SKS network is polled every 15 minutes and a report put online at:

www.pramberger.at/peter/services/keyserver/network/

New servers are discovered automatically if they are peering with others that are already known, so you don't need to request your new server to be added. Just set up peering and eventually your server will appear in the report.