Category Archives: Hosting

Trustworthy email: authentication using exim4, SPF, DKIM and DMARC

The email authentication technologies we’re about to implement are, according to the authentication authorities, all you need to be regarded as being from your domain when you send email, and someone else not being from your domain.  Effect: your emails can be considered trustworthy by email receivers who use these technologies. If they don’t use these technologies, they can’t tell.

At the very least, Google will be less likely to think your email is spam.

PTR record

A PTR record can be obtained from your host’s nameserver – it’s a reverse DNS record for your IP address. If the PTR record points at rather than (your domain), and you’re claiming to be sending mail from, what’s the email recipient meant to think?


will tell you what the host for that IP is. Lodge a ticket with your hosting provider and get that PTR record changed to This might take about a day.

SPF record

Create a Sender Policy Framework record on your nameserver:

TXT @ "v=spf1 a mx -all"

This says “for my domain, I will only send email from IP addresses listed on the nameserver”.  Nameserver changes take time to propagate.

After your nameserver changes have propagated, you can go to to check out if you got it right.


DomainKeys Identified Mail is where things get more involved.  We’re doing this on a Debian Linux, like Ubuntu for exim4. We’re making our signing key 2048 bits, which is long enough to make life slightly unpleasant for us. Fortunately for you I’ve written a bash script that outputs the TXT record we need to create on the nameserver – because some nameservers (I’m looking at you, Gandi) can’t hold “long” strings – it’s broken into “small” strings:

sudo apt install openssl
cd /etc/exim4
sudo openssl genrsa -out dkim.private 2048
sudo openssl rsa -in dkim.private -out dkim.public -pubout -outform PEM
echo $(echo $(date -u +%Y%m%d && echo '') | sed -e 's/[ ]//g' && echo $(echo ' TXT "v=DKIM1; p="' && echo $(grep 'PUBLIC KEY' -v dkim.public) | sed -e 's/[ ]//g' | fold -w200 | sed -e 's/\(.*\)/"\1"/g'))

which gives something like
20170419._domainkey TXT “v=DKIM1; p=” “MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvCNqU0Njd4YQ4e89T3FNc+uyOS2JwUqynGk7uwcSYHjIE2MGRuTxi56s4JgPKSnCVlBkJlUnXQHXFp2UGnLm8SADtjRMfWwpNxz6TmzXBpMnNZV1zvuoBBdcxh0Qg1TtCSACtWM6ehml0BmOHVA8Ippqj9iRlP2HMjuVMxZXewN9eJl”

That all goes into one nameserver TXT record, spaces and all.  The world will join up the ” ” and get one big string. Note the bold number up there? That’s the selector. That a number needs to get larger with each new key.  Periodically you’re going to have to reissue your key because security.  You know what gets larger as time goes by?  The date.  Use the date.  If you screw up, use tomorrow’s date, etc.

Once you’ve got our public key out to the public via our public nameserver, we need to get exim to sign the payloads:

sudo nano conf.d/main/01_exim4-config_listmacrosdefs

After the line CONFDIR = /etc/exim4, add:

#DKIM loading
DKIM_CANON = relaxed
DKIM_DOMAIN = ${sender_address_domain}
DKIM_SELECTOR = 20170419

and reload the mail server

sudo service exim4 restart

After an appropriate delay for nameserver propagation, use to check your work.
If that works out, from to ensure everything checks out:

echo -e "Test my DKIM plz\nMsg Body\n.\n\n" | mail -v


Domain-based Message Authentication, Reporting and Conformance is where the wheels can come off if you screwed anything up.  We’re going to set things up so that when you screw it up, computers scold you rather than putting your emails in the bin.

You will need to create two dmarc reporting accounts.  Servers will email you a (surprisingly detailed) report card on how you’re doing with your implementation. It’s best if these accounts are on the same domain, because technically you need to be or it’ll be ignored (Google will happily mail reports off-domain even if the other domain hasn’t said that’s okay).  Yours are and, according to the following nameserver entry: 1800 IN TXT "v=DMARC1;p=none;pct=100;;"

none is the consequence for screwing up. none is where we’ll start at, and see what the reporting records say to us.  After a while, you’ll be comfortable that everything is ticking along nicely, and you’ll up the consequent to quarantine (shove it in spam) or reject (burn it).

After your nameserver changes have propagated, you can go to to check out if you got it right.

As a human, to read the records sent to you, upload the files to

Making a captcha deamon for spamgourmet installations

For those of you following along at home, this is part of a cookbook style instruction set for getting spamgourmet going, but because of screwed up permission logic I can’t post this section there.

The captcha is for validating humanity when creating spamgourmet accounts. We’re going to limit what parts of the OS it can tromp over:

sudo useradd -c "captcha server for spamgourmet" -f -1 -M -r captcha
sudo /bin/mkdir -p /var/www-spamgourmet/captchasrv/
sudo chown -R captcha /usr/local/lib/spamgourmet/captchasrv/
sudo chown -R captcha /var/www-spamgourmet/captcha

Now we make our one-line shell script for running the daemon

sudo nano /etc/init.d/

# Provides:          captchasrv
# Required-Start:    $local_fs $network
# Required-Stop:     $local_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: captchasrv
# Description:       captcha daemon for spamgourmet
sudo -u captcha perl /usr/local/lib/spamgourmet/captchasrv/ &

Next we get it going

sudo chmod +x /etc/init.d/
sudo update-rc.d defaults

And check if that worked, there should be about four entries:

ps -aux | grep captc

Now the captcha server will start whenever the computer starts.

Installing a secure Apache webserver to run Perl

So, you want to run Perl on the web, because it’s the 90s all over again. You want HTTPS, because… no, there’s no because.  You want HTTPS.  Who wouldn’t?  Here’s what you do on a Debian Linux such as Ubuntu:
sudo apt-get install apache2 libapache2-mod-perl2
mod-perl is an Apache module that allows Perl programs to be executed from Apache.

Our goal is to get /var/www/html/ running at

print "Hello World"

Disable the default Apache virtual host:

sudo a2dissite 000-default.conf

Create an file in /etc/apache2/sites-available with your text editor, replacing instances of with your own domain name in both the configuration file and in the file name /etc/apache2/sites-available/

<VirtualHost *:80>
     ErrorLog ${APACHE_LOG_DIR}/error.log
     CustomLog ${APACHE_LOG_DIR}/access.log combined
     </Directory /var/www/>
              Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
              AllowOverride None
              AddHandler cgi-script .pl
              Require all granted

<IfModule mod_ssl.c>
<VirtualHost *:443>
     ErrorLog ${APACHE_LOG_DIR}/error.log
     CustomLog ${APACHE_LOG_DIR}/access.log combined
     </Directory /var/www/>
              Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
              AllowOverride None
              AddHandler cgi-script .pl
              Require all granted

If you have multiple sites, you’ll want to do things with DocumentRoot to isolate them from each other. But that’s for another post.

You might add in DirectoryIndex / to make execute your program.

The Directory section above allows you to isolate executable code from served code, which is good practice. For this example we’re dumping the executable in with everything else, which is questionable.

Repeat this process for any other domains you host.

sudo a2ensite
sudo service apache2 restart

Punch holes in your firewall for ports 80 and 443.  Navigate to to check all is okay. You ought to see Hello World displayed for your website.

Having security used to be a pain.  SSL certificates signed by a recognised CA cost money, and then you’d have to keep them up to date, and there wasn’t process automation, so you’d do all that stuff by hand.  LetsEncrypt address all these problems, handing out free certificates and scripted everything.

Now it’s time for the S part of HTTPS:
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install python-certbot-apache
sudo certbot --apache

certbot renew
If that works, we’ll automatically renew our 90-day certificates every month:
echo '@monthly root /usr/bin/certbot renew >> /var/log/letsencrypt/letsencrypt-auto-update.log' | sudo tee --append /etc/crontab

Done.  You’ll never have to worry about certificates again. Now alter your Apache sites-available file (look in /etc/apache2/sites-available/) to include the (optional) redirect HTTP to HTTPS and the mandatory location of the SSL certificates:

<VirtualHost *:80>
# Only allow HTTPS
RewriteEngine on
RewriteCond %{SERVER_NAME} =
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,QSA,R=permanent]

<IfModule mod_ssl.c>
<VirtualHost *:443>
SSLCertificateFile /etc/letsencrypt/live/
SSLCertificateKeyFile /etc/letsencrypt/live/
Include /etc/letsencrypt/options-ssl-apache.conf

Now make the secure version live, and in the process the insecure one… shy? When a request is made for a http page, like, the response will be “Here’s where what you asked for has moved to… forever!”:
sudo service apache2 restart
Now requesting ought to deliver you to

Install exim4 STARTTLS using a free LetsEncrypt certificate

Here we are on a Debian Linux, such as Ubuntu and we want to run a mail server. Exim4 is currently the most popular email server, but getting it up and working for free is a hassle – who wants to pay for a SSL certificate, on an ongoing basis? And then there’s the maintenance of the security of it – constant renewal, renouncing and re-installation of the certificates.

Wherever you see, swap in your Fully Qualified Domain Name. That may be
It’s assumed you’re not logged in as root, but user ubuntu
Wherever you see, swap in your machine’s local IP address, from
ifconfig | grep "inet addr" | grep -v ""

Security is all handled automatically by LetsEncrypt’s certbot. I’ll let you look that one up yourself. Run it up and get your certificate for

Once you’ve got that handled, punch a hole in your firewall so that port 25 can get through from the outside world to your machine. Be aware: the outside world is filled full of botnets trying to hack into your machine.  After installing exim, keep an eye on the logs in /var/log/exim4/ for a while.

Let’s install exim4:
sudo apt-get install exim4
sudo dpkg-reconfigure exim4-config

  • pick “Internet site”
  • system mail name is
  • IP address is (the one returned by ifconfig, not the externally accessable one)
  • Other destinations:
  • No relays
  • No smarthost
  • No Dial-on-Demand
  • mbox format (or whatever)
  • Split the files
  • ubuntu for postmaster mail

Check we’re now running a mail server:
sudo netstat -napt
should show
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0* LISTEN 25700/exim4

Now we have a mail server, the world needs to find it. Check your nameserver setting to ensure mail is destined this machine.  You probably want only one MX record.

Check the Internet can send mail to our server. After allowing for the appropriate propagation delay for your nameserver changes, use gmail or something to send an email to – you should be able to read it by typing

Now it’s time to enable MTA-MTA link encryption for secure transport of mail, by enabling STARTTLS on exim4 using our LetsEncrypt certificate
sudo nano /etc/exim4/conf.d/main/03_exim4-config_tlsoptions
Enable STARTTLS by adding/setting in the tlsoptions section:

before any of the IF shenanigans. Also add/replace pointers to the certificates:
tls_certificate = /etc/letsencrypt/live/
tls_privatekey = /etc/letsencrypt/live/

The MAIN_TLS_CERTKEY = no entry fixes an exim4 log message
2017-04-16 09:13:24 TLS error on connection from (IcePlanet) [] (cert/key setup: cert=/etc/exim4/exim.crt key=/etc/exim4/exim.key): Error while reading file.
You will see this when testing with swaks:
$ swaks -a -tls -q HELO -s -au test -ap '<>'
=== Trying
=== Connected to
< - 220 ESMTP Exim 4.86_2 Ubuntu Sun, 16 Apr 2017 09:13:24 +0000 -> EHLO IcePlanet
< - Hello []
< ** 454 TLS currently unavailable *** STARTTLS attempted but failed -> QUIT
< - 221 closing connection
=== Connection closed with remote host.

Allow exim (which when running runs as user Debian-exim) to get to the certificates:
sudo chown Debian-exim /etc/letsencrypt/live/
sudo chown Debian-exim /etc/letsencrypt/archive/
sudo chown Debian-exim /etc/letsencrypt/archive/
sudo chown Debian-exim /etc/letsencrypt/archive/

Changing these permissions doesn’t affect apache2’s ability to get them.

These permission changes prevent the following error message in your log file:
2008-06-03 08:27:35 TLS error on connection from ([]) [] (cert/key setup: cert=/etc/ssl/certs/server.pem key=/etc/ssl/private/server.key): Error while reading file.

Restart the service and the TLS settings ought to be working
sudo service exim4 restart
Test STARTTLS is working from another machine
swaks -a -tls -q HELO -s -au test -ap '<>'
There shouldn’t be any obvious complaining.


Easy ways to save bandwidth

After reading Jeff Atwood’s terrific post about saving bandwidth on web sites I’ve moved the Geekrant RSS feeds over to Feedburner, using Steve Smith’s mavellous WordPress Feedburner plugin, which works in WP 2.0x and 1.5x.

I also turned on HTTP compression, which in WordPress is as easy as clicking a checkbox. It not only saves you bandwidth, but users get your pages served quicker, since the bottleneck is bound to be their bandwidth, not their browser’s ability to decompress.

We’ll see how it goes. Bandwidth has been growing recently: January 2.8Gb; February 2.7Gb; March 3.4Gb. It’s not at ludicrous levels, but if it keeps climbing, I’ll end up paying more for the hosting. Hopefully this will help bring it back down.

Update 8:40pm. First thing I notice is that when reading the feed from within the Feedburner site, it doesn’t treat relative paths to images properly. I guess I’ll have to put absolute paths, ‘cos at the moment in the previous post it’s trying to load instead of I wonder how it treats relative links?

htaccess Generator

Daniel would love this htaccess Generator, not that he needs it, what with him being a htaccess-loving-geek and all

Outages and response times

Cam ponders web hosting SLAs and wonders what’s reasonable. For his hosting, they guarantee 99.99% uptime, which works out to 52 minutes per year. (His outage was about 9 hours, or about ten years’ worth).

Bad stuff happens. We all know that. Even if it’s the most reliable setup ever. But there’s some major factors in determining what’s acceptable:

Frequency — If it’s happening too regularly, then there’s a reliability problem. They need better hardware, better software, whatever it is, needs to be fixed. Cam reckons it’s the second time in a few months.

Response — Obviously, you want a quick response, and a quick (and reliable) solution. There’s also sorts of monitoring tools out there these days. Typically anything like a full outage should be known about within minutes. A reputable web host will have substitute hardware ready to switch-on and go just as soon as that nice recent backup is restored.

Communications — Any third party like this has to keep the customer informed. There’s no excuse for not doing so. SMS alarms, emails, phone calls, whatever. (I wrote about alarms recently for my work blog.)

BTW, Cam’s also having troubles with his iPod… or more accurately, Apple’s 90 day warranty on replacement units.

I reckon he’s jinxed, myself.

Seeing a new server before re-delegation

One of the weaknesses of WordPress and most other web-configured applications is that unless you want to go SQL or config-file-wrangling, it’s pretty much only configurable via the web, at least for tweaking, importing posts, setting up most of the options. This is a problem when, for instance, you’re migrating an existing site onto WP, and it’s on a new server, as you can’t get to the wp-admin screens.

The way to do it is to hack your hosts file. Once the new server is running and WP is setup on it, find your hosts file and add an entry to the new server. On Windows, this is the c:\windows\system32\drivers\etc\hosts file.

Chuck in a line that says contains your new server’s IP address, and the hostname. Something like:

(Whoopsie, real-world example with a fake IP. The new evision site is going live Real Soon Now.)

Save, then away you go. You can see the new site and tweak to your heart’s content, but nobody else will be able to see any of it until you re-delegate.

The catch? It probably won’t work from behind corporate networks, where your computer uses a proxy.

Hurricane Rita

I’ve been notified by my web ISP that Hurricane Rita is approaching Houston. Why does this matter? Because (and a number of other sites I run) are sitting on a server in a data centre in Houston. I’ve been encouraged to take backups of important content, which I’ll be doing. It’s a reminder that regular backups are an essential precaution.

If the site goes down in the next day or two, you’ll know why. Best wishes to those in the affected areas.

More from George

More goodness from George Skarbek’s column in The Age (19-Jul-2005).

A punter asks George about sending large files across the net. One suggestion is to set up a web host, and the reader is sent off to GoDaddy to find out about domain names and hosting fees, and even ponders if a web server should be set up on their own computer. Uhh, but these days but most ISPs provide a basic web hosting facility, good enough as a drop point for leaving big files… surely it’s better to look at this first? Not to mention the many online storage services, such as Yahoo Briefcase.

A question about whether one should turn on IE’s “Do not save encrypted pages to disk” option comes up with some gibberish about “static web pages and dynamic data”. Eh? The point of this option is explained in IE’s help: it avoids the browser saving the pages onto the hard disk where they might be snooped upon by other users on the same computer. Since an HTML page is plain text, and depending on the site used, user or session IDs or even passwords could be embedded in the HTML, in some environments it might be desirable to not save this in the cache.

(Don’t get me wrong; most of George’s answers are spot on. Just a few that haven’t quite lived up to expectations, and it’s been bugging me a bit…)

Pirates! Spammers! Gyroscopes! Bandwidth thieves!

This is officially getting ridiculous. Not only are my blogs getting a lot of comment spam, but my personal blog site is burning huge amounts of bandwidth, as particular (I assume zombie) hosts hit the site.

Below are the top ten bandwidth users of for June:

Top 10 of 15312 Total Sites By KBytes
# Hits Files KBytes Visits Hostname
1 14380 4.10% 3801 1.77% 111235 2.22% 159 0.24%
2 17558 5.01% 3191 1.48% 99441 1.98% 157 0.24%
3 3927 1.12% 3640 1.69% 75989 1.51% 3 0.00%
4 3062 0.87% 2797 1.30% 74881 1.49% 171 0.26%
5 3057 0.87% 2200 1.02% 62547 1.25% 392 0.60%
6 2691 0.77% 2248 1.04% 60684 1.21% 153 0.23%
7 2256 0.64% 2082 0.97% 56383 1.12% 124 0.19%
8 2146 0.61% 2033 0.94% 51665 1.03% 279 0.43%
9 2001 0.57% 1755 0.82% 47605 0.95% 23 0.04%
10 1686 0.48% 1571 0.73% 35979 0.72% 325 0.50% corporativos

It’s not like this site is hosting pr0n or something — there’s just no reason why any single host would need to grab 110Mb of traffic in a single month. In total traffic topped 4Gb for the month, which is ludicrous for a diary site with a few photos on it. 4Gb is actually my monthly limit — thankfully my web ISP isn’t too strict about charging extra for hitting that, but there’s always the risk if this is consistent that it’ll be costing me real money.

As a result I’ve started a list of bandwidth hogs’ IP addresses, which I’m putting in the .htaccess file. Anything with lots of hits and grabbing above about 5Mb per month is going onto the list, and the list is being duplicated (manually unfortunately) across to the other WordPress sites that I run.

Inspection of the access_log is particularly enlightening, with at present a staggering number of requests coming in with a referer at poker-related sites. Of the 6665 hits in the file for today (covering about 13 hours) there are 674 from (note the wonky spelling) and 1212 from All of these too are now being blocked with a 403 (forbidden) via .htaccess.

Sigh. I suppose it’s just too much to expect people to place nice?

.htaccess extract – Feel free to copy for your own site to block miscreants.
Continue reading

Green sites, dead pixels and Remote Desktop

Keep your web site green by hosting it in an environmentally sustainable data centre.

Unstick your dead pixels by flashing rapid colour changes through them. 60% success rate, apparently. What have you got to lose?

These guys claim to have got round the limitation of Windows XP Remote Desktop of only one user at a time, by replacing one of the Terminal Server DLLs with that from an older build of SP2.