Category Archives: Hosting

Securely run a low memory/low CPU Minecraft server

If you’ve got next to no memory and CPU available to run a Minecraft server, don’t fret. Cuberite is what you’re after. At them moment, Cuberite isn’t bug-free, nor indiscernible from a genuine Minecraft server, but it’s quite usable – and instead of needing 4GB+ of RAM, it needs less than 300MB. And it needs next to no processing power: some people run Cuberite on their Raspberry Pi and have plenty of CPU available.

I would at this point go on about how this is a significant point of difference between C++ and Java, but Java optimizes for something different to C++.  I got into an interesting discussion with Cathy about this after reading a question someone had about what Java was trying to be good at. I used to think that VB was the new COBOL, but clearly Java is the new COBOL; those Java programs are going nowhere, they’re verbose and easy to understand and maintain.

A point to note: The Minecraft protocols are bandwidth heavy, I found if I wanted to run a server at home I’d be able to have one, perhaps two players. Thus is Internet in Australia. Instead I’ve dropped this onto a free AWS VPS instance and bandwidth is no problem.

Still, it’s a random piece of software off the Internet, so we’re going to give it its own user account for our own safety. Let’s install the software:

curl -sSfL https://download.cuberite.org | sh
sudo mv Server /usr/local/cuberite
cd /usr/local/cuberite

Cuberite allows configuration through a web interface.  We now enable webadmin.ini
[User:admin]
; Please restart Cuberite to apply changes made in this file!
Password=yourstrongpassword
[WebAdmin]
Ports=8080
Enabled=1

Port 8080 is the alternative html port (http/https).  You could cd into webadmin and run GenerateSelfSignedHTTPSCertUsingOpenssl.sh and get https serving, but your browser will barf on the certificate. Instead, let’s use a LetsEncrypt certificate, one that we installed earlier. First we make our one-line shell script for running the daemon:

sudo useradd -c "Cuberite server" -f -1 -M -r cuberite
chown -R cuberite:`whoami` /usr/local/cuberite/
sudo nano /etc/init.d/cuberite.sh

#!/bin/sh
### BEGIN INIT INFO
# Provides: cuberite
# Required-Start: $local_fs $network
# Required-Stop: $local_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: cuberite
# Description: Cuberite server, a Minecraft server lookalike
### END INIT INFO
cd /usr/local/cuberite
sudo -u cuberite /usr/local/cuberite/Cuberite -d &

Next we set it going when the box starts up:

sudo chmod +x /etc/init.d/cuberite.sh
sudo update-rc.d cuberite.sh defaults

Before we can go to the website we need to allow user cuberite to get to the certificates:

sudo groupadd privkey_users
sudo usermod -aG privkey_users cuberite
sudo sudo chmod g+rx /etc/letsencrypt/live/
sudo sudo chmod g+rx /etc/letsencrypt/archive/
sudo chown root:privkey_users /etc/letsencrypt/archive/
sudo chown root:privkey_users /etc/letsencrypt/archive/example.com/
sudo chown root:privkey_users /etc/letsencrypt/archive/example.com/cert1.pem
sudo chown root:privkey_users /etc/letsencrypt/archive/example.com/chain1.pem
sudo chown root:privkey_users /etc/letsencrypt/archive/example.com/privkey1.pem
sudo chown root:privkey_users /etc/letsencrypt/archive/example.com/fullchain1.pem
sudo chown root:privkey_users /etc/letsencrypt/live/
sudo chown root:privkey_users /etc/letsencrypt/live/example.com/
sudo -u cuberite ln -s /etc/letsencrypt/live/example.com/cert.pem /usr/local/cuberite/webadmin/httpscert.crt
sudo -u cuberite ln -s /etc/letsencrypt/live/example.com/privkey.pem /usr/local/cuberite/webadmin/httpskey.pem

Changing these permissions doesn’t affect apache2’s ability to get them.
The reason we’ve used a group here is to allow both Cuberite and any other app (for example, exim) to access the private keys; just add any other user that needs to use the private keys to the privkey_users group.

Remember to punch a firewall hole for port 8080. Fire up Cuberite now:

sudo service cuberite restart

And check if that worked, there should be about one entry:

ps -aux | grep cuberitps -aux | grep cuberit

If not, you can check in the logs directory to see what’s wrong.

So now:

sudo lsof -i :8080
https://example.com:8080/

should be secure.  Note the https is mandatory, as your browser will use http if you fail to specify a protocol.

Trustworthy email: authentication using exim4, SPF, DKIM and DMARC

The email authentication technologies we’re about to implement are, according to the authentication authorities, all you need to be regarded as being from your domain when you send email, and someone else not being from your domain.  Effect: your emails can be considered trustworthy by email receivers who use these technologies. If they don’t use these technologies, they can’t tell.

At the very least, Google will be less likely to think your email is spam.

PTR record

A PTR record can be obtained from your host’s nameserver – it’s a reverse DNS record for your IP address. If the PTR record points at ec2-23-65-53-221.ap-southeast-2.compute.amazonaws.com rather than example.com (your domain), and you’re claiming to be sending mail from example.com, what’s the email recipient meant to think?

host 23.65.53.221

will tell you what the host for that IP is. Lodge a ticket with your hosting provider and get that PTR record changed to example.com. This might take about a day.

SPF record

Create a Sender Policy Framework record on your nameserver:

TXT @ "v=spf1 a mx -all"

This says “for my domain, I will only send email from IP addresses listed on the nameserver”.  Nameserver changes take time to propagate.

After your nameserver changes have propagated, you can go to https://dmarcian.com/spf-survey/ to check out if you got it right.

DKIM

DomainKeys Identified Mail is where things get more involved.  We’re doing this on a Debian Linux, like Ubuntu for exim4. We’re making our signing key 2048 bits, which is long enough to make life slightly unpleasant for us. Fortunately for you I’ve written a bash script that outputs the TXT record we need to create on the nameserver – because some nameservers (I’m looking at you, Gandi) can’t hold “long” strings – it’s broken into “small” strings:

sudo apt install openssl
cd /etc/exim4
sudo openssl genrsa -out dkim.private 2048
sudo openssl rsa -in dkim.private -out dkim.public -pubout -outform PEM
echo $(echo $(date -u +%Y%m%d && echo '.domainkey.example.com') | sed -e 's/[ ]//g' && echo $(echo ' TXT "v=DKIM1; p="' && echo $(grep 'PUBLIC KEY' -v dkim.public) | sed -e 's/[ ]//g' | fold -w200 | sed -e 's/\(.*\)/"\1"/g'))

which gives something like
20170419._domainkey TXT “v=DKIM1; p=” “MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvCNqU0Njd4YQ4e89T3FNc+uyOS2JwUqynGk7uwcSYHjIE2MGRuTxi56s4JgPKSnCVlBkJlUnXQHXFp2UGnLm8SADtjRMfWwpNxz6TmzXBpMnNZV1zvuoBBdcxh0Qg1TtCSACtWM6ehml0BmOHVA8Ippqj9iRlP2HMjuVMxZXewN9eJl”
“c6zsyOwQPvVKpJ+Rdvr+pPkDztAVTw7mNSeyy+TL6O/3L9sl7A19Yx8jLHKuGUh9LutVuv1VP16e7GwlnA3Zqn5C1jyY5Qvr2SEHZMcE3VzD7XKZtZWbpkGh+A5S15NrOH4k9tbVfNbjft6Y1jUJRTT+4DD0ZEVlr4zO+WQIDAQAB”

That all goes into one nameserver TXT record, spaces and all.  The world will join up the ” ” and get one big string. Note the bold number up there? That’s the selector. That a number needs to get larger with each new key.  Periodically you’re going to have to reissue your key because security.  You know what gets larger as time goes by?  The date.  Use the date.  If you screw up, use tomorrow’s date, etc.

Once you’ve got our public key out to the public via our public nameserver, we need to get exim to sign the payloads:

sudo nano conf.d/main/01_exim4-config_listmacrosdefs

After the line CONFDIR = /etc/exim4, add:

#DKIM loading
DKIM_CANON = relaxed
DKIM_DOMAIN = ${sender_address_domain}
DKIM_PRIVATE_KEY = CONFDIR/dkim.private
DKIM_SELECTOR = 20170419

and reload the mail server

sudo service exim4 restart

After an appropriate delay for nameserver propagation, use https://protodave.com/tools/dkim-key-checker/?selector=20170419&domain=example.com to check your work.
If that works out, mailto:check-auth@verifier.port25.com from example.com to ensure everything checks out:

echo -e "Test my DKIM plz\nMsg Body\n.\n\n" | mail -v check-auth@verifier.port25.com

DMARC

Domain-based Message Authentication, Reporting and Conformance is where the wheels can come off if you screwed anything up.  We’re going to set things up so that when you screw it up, computers scold you rather than putting your emails in the bin.

You will need to create two dmarc reporting accounts.  Servers will email you a (surprisingly detailed) report card on how you’re doing with your implementation. It’s best if these accounts are on the same domain, because technically you need to be or it’ll be ignored (Google will happily mail reports off-domain even if the other domain hasn’t said that’s okay).  Yours are dmarc_failures@example.com and dmarc_summary@example.com, according to the following nameserver entry:

_dmarc.example.com. 1800 IN TXT "v=DMARC1;p=none;pct=100;ruf=mailto:dmarc_failures@example.com;rua=mailto:dmarc_summary@example.com"

none is the consequence for screwing up. none is where we’ll start at, and see what the reporting records say to us.  After a while, you’ll be comfortable that everything is ticking along nicely, and you’ll up the consequent to quarantine (shove it in spam) or reject (burn it).

After your nameserver changes have propagated, you can go to https://dmarcian.com/dmarc-inspector/ to check out if you got it right.

As a human, to read the records sent to you, upload the files to https://dmarcian.com/dmarc-xml/

Making a captcha deamon for spamgourmet installations

For those of you following along at home, this is part of a cookbook style instruction set for getting spamgourmet going, but because of screwed up permission logic I can’t post this section there.

The captcha is for validating humanity when creating spamgourmet accounts. We’re going to limit what parts of the OS it can tromp over:

sudo useradd -c "captcha server for spamgourmet" -f -1 -M -r captcha
sudo /bin/mkdir -p /var/www-spamgourmet/captchasrv/
sudo chown -R captcha /usr/local/lib/spamgourmet/captchasrv/
sudo chown -R captcha /var/www-spamgourmet/captcha

Now we make our one-line shell script for running the daemon

sudo nano /etc/init.d/captcha.sh

#!/bin/sh
### BEGIN INIT INFO
# Provides:          captchasrv
# Required-Start:    $local_fs $network
# Required-Stop:     $local_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: captchasrv
# Description:       captcha daemon for spamgourmet
### END INIT INFO
sudo -u captcha perl /usr/local/lib/spamgourmet/captchasrv/captchasrv.pl &

Next we get it going

sudo chmod +x /etc/init.d/captcha.sh
sudo update-rc.d captcha.sh defaults

And check if that worked, there should be about four entries:

ps -aux | grep captc

Now the captcha server will start whenever the computer starts.

Installing a secure Apache webserver to run Perl

So, you want to run Perl on the web, because it’s the 90s all over again. You want HTTPS, because… no, there’s no because.  You want HTTPS.  Who wouldn’t?  Here’s what you do on a Debian Linux such as Ubuntu:
sudo apt-get install apache2 libapache2-mod-perl2
mod-perl is an Apache module that allows Perl programs to be executed from Apache.

Our goal is to get /var/www/html/index.pl running at http://www.example.com/index.pl:

#!/usr/bin/perl
print "Hello World"

Disable the default Apache virtual host:

sudo a2dissite 000-default.conf

Create an example.com.conf file in /etc/apache2/sites-available with your text editor, replacing instances of example.com with your own domain name in both the configuration file and in the file name /etc/apache2/sites-available/example.com.conf

<VirtualHost *:80>
     ServerName example.com
     ServerAlias www.example.com
     ErrorLog ${APACHE_LOG_DIR}/error.log
     CustomLog ${APACHE_LOG_DIR}/access.log combined
     <Directory /var/www/>
              Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
              AllowOverride None
              AddHandler cgi-script .pl
              Require all granted
     </Directory>
</VirtualHost>

<IfModule mod_ssl.c>
<VirtualHost *:443>
     ServerName example.com
     ServerAlias www.example.com
     ErrorLog ${APACHE_LOG_DIR}/error.log
     CustomLog ${APACHE_LOG_DIR}/access.log combined
     <Directory /var/www/>
              Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
              AllowOverride None
              AddHandler cgi-script .pl
              Require all granted
     </Directory>
</VirtualHost>
</IfModule>

If you have multiple sites, you’ll want to do things with DocumentRoot to isolate them from each other. But that’s for another post.

You might add in DirectoryIndex /index.pl to make http://www.example.com/ execute your program.

The Directory section above allows you to isolate executable code from served code, which is good practice. For this example we’re dumping the executable in with everything else, which is questionable.

Repeat this process for any other domains you host.

sudo a2ensite example.com.conf
sudo ln -r -s /etc/apache2/sites-available/example.com.conf /etc/apache2/sites-enabled/example.com.conf
sudo service apache2 restart

Punch holes in your firewall for ports 80 and 443.  Navigate to http://www.example.com/index.pl to check all is okay. You ought to see Hello World displayed for your website.

Having security used to be a pain.  SSL certificates signed by a recognised CA cost money, and then you’d have to keep them up to date, and there wasn’t process automation, so you’d do all that stuff by hand.  LetsEncrypt address all these problems, handing out free certificates and scripted everything.

Now it’s time for the S part of HTTPS:
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install python-certbot-apache
sudo certbot --apache

certbot renew
If that works, we’ll automatically renew our 90-day certificates every month:
echo '@monthly root /usr/bin/certbot renew >> /var/log/letsencrypt/letsencrypt-auto-update.log' | sudo tee --append /etc/crontab

Done.  You’ll never have to worry about certificates again. Now alter your Apache sites-available file (look in /etc/apache2/sites-available/) to include the (optional) redirect HTTP to HTTPS and the mandatory location of the SSL certificates:

<VirtualHost *:80>
....
# Only allow HTTPS
RewriteEngine on
RewriteCond %{SERVER_NAME} = example.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,QSA,R=permanent]
</VirtualHost>

<IfModule mod_ssl.c>
<VirtualHost *:443>
...
SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

Now make the secure version live, and in the process the insecure one… shy? When a request is made for a http page, like http://example.com/index.html, the response will be “Here’s https://example.com/index.html where what you asked for has moved to… forever!”:
sudo service apache2 restart
Now requesting http://www.example.com/index.pl ought to deliver you to https://www.example.com/index.pl

Install exim4 STARTTLS using a free LetsEncrypt certificate

Here we are on a Debian Linux, such as Ubuntu and we want to run a mail server. Exim4 is currently the most popular email server, but getting it up and working for free is a hassle – who wants to pay for a SSL certificate, on an ongoing basis? And then there’s the maintenance of the security of it – constant renewal, renouncing and re-installation of the certificates.

Wherever you see example.com, swap in your Fully Qualified Domain Name. That may be mail.example.com
It’s assumed you’re not logged in as root, but user ubuntu
Wherever you see 1.2.3.4, swap in your machine’s local IP address, from
ifconfig | grep "inet addr" | grep -v "127.0.0.1"

Security is all handled automatically by LetsEncrypt’s certbot. I’ll let you look that one up yourself. Run it up and get your certificate for example.com

Once you’ve got that handled, punch a hole in your firewall so that port 25 can get through from the outside world to your machine. Be aware: the outside world is filled full of botnets trying to hack into your machine.  After installing exim, keep an eye on the logs in /var/log/exim4/ for a while.

Let’s install exim4:
sudo apt-get install exim4
sudo dpkg-reconfigure exim4-config

  • pick “Internet site”
  • system mail name is example.com
  • IP address is 1.2.3.4 (the one returned by ifconfig, not the externally accessable one)
  • Other destinations: example.com
  • No relays
  • No smarthost
  • No Dial-on-Demand
  • mbox format (or whatever)
  • Split the files
  • ubuntu for postmaster mail

Check we’re now running a mail server:
sudo netstat -napt
should show
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 1.2.3.4:25 0.0.0.0:* LISTEN 25700/exim4

Now we have a mail server, the world needs to find it. Check your nameserver setting to ensure mail is destined this machine.  You probably want only one MX record.

Check the Internet can send mail to our server. After allowing for the appropriate propagation delay for your nameserver changes, use gmail or something to send an email to ubuntu@example.com – you should be able to read it by typing
mail

Now it’s time to enable MTA-MTA link encryption for secure transport of mail, by enabling STARTTLS on exim4 using our LetsEncrypt certificate
sudo nano /etc/exim4/conf.d/main/03_exim4-config_tlsoptions
Enable STARTTLS by adding/setting in the tlsoptions section:
MAIN_TLS_ENABLE = yes
MAIN_TLS_CERTKEY = no

before any of the IF shenanigans. Also add/replace pointers to the certificates:
tls_certificate = /etc/letsencrypt/live/example.com/fullchain.pem
tls_privatekey = /etc/letsencrypt/live/example.com/privkey.pem

The MAIN_TLS_CERTKEY = no entry fixes an exim4 log message
2017-04-16 09:13:24 TLS error on connection from your.home.ip.com (IcePlanet) [5.6.7.8] (cert/key setup: cert=/etc/exim4/exim.crt key=/etc/exim4/exim.key): Error while reading file.
You will see this when testing with swaks:
$ swaks -a -tls -q HELO -s example.com -au test -ap '<>'
=== Trying example.com:25...
=== Connected to example.com.
< - 220 your.vps.host.com ESMTP Exim 4.86_2 Ubuntu Sun, 16 Apr 2017 09:13:24 +0000 -> EHLO IcePlanet
< - 250-your.vps.host.com Hello your.home.ip.com [5.6.7.8]
STARTTLS
< ** 454 TLS currently unavailable *** STARTTLS attempted but failed -> QUIT
< - 221 your.vps.host.com closing connection
=== Connection closed with remote host.

Allow exim (which when running runs as user Debian-exim) to get to the certificates:

sudo groupadd privkey_users
sudo usermod -aG privkey_users Debian-exim
sudo sudo chmod g+rx /etc/letsencrypt/live/
sudo sudo chmod g+rx /etc/letsencrypt/archive/
sudo chown root:privkey_users /etc/letsencrypt/archive/
sudo chown root:privkey_users /etc/letsencrypt/archive/example.com/
sudo chown root:privkey_users /etc/letsencrypt/archive/example.com/cert1.pem
sudo chown root:privkey_users /etc/letsencrypt/archive/example.com/chain1.pem
sudo chown root:privkey_users /etc/letsencrypt/archive/example.com/privkey1.pem
sudo chown root:privkey_users /etc/letsencrypt/archive/example.com/fullchain1.pem
sudo chown root:privkey_users /etc/letsencrypt/live/
sudo chown root:privkey_users /etc/letsencrypt/live/example.com/

Changing these permissions doesn’t affect apache2’s ability to get them.
The reason we’ve used a group here is to allow both exim and any other app (for example, a secondary service that wants to use 8080 to serve up a configuration page) to access the private keys; just add any other user that needs to use the private keys to the privkey_users group.

These permission changes prevent the following error message in your log file:
2008-06-03 08:27:35 TLS error on connection from me.at.home.com ([1.2.3.4]) [5.6.7.8] (cert/key setup: cert=/etc/ssl/certs/server.pem key=/etc/ssl/private/server.key): Error while reading file.

Restart the service and the TLS settings ought to be working
sudo service exim4 restart
Test STARTTLS is working from another machine
swaks -a -tls -q HELO -s example.com -au test -ap '<>'
There shouldn’t be any obvious complaining.

Done!

Easy ways to save bandwidth

After reading Jeff Atwood’s terrific post about saving bandwidth on web sites I’ve moved the Geekrant RSS feeds over to Feedburner, using Steve Smith’s mavellous WordPress Feedburner plugin, which works in WP 2.0x and 1.5x.

I also turned on HTTP compression, which in WordPress is as easy as clicking a checkbox. It not only saves you bandwidth, but users get your pages served quicker, since the bottleneck is bound to be their bandwidth, not their browser’s ability to decompress.

We’ll see how it goes. Bandwidth has been growing recently: January 2.8Gb; February 2.7Gb; March 3.4Gb. It’s not at ludicrous levels, but if it keeps climbing, I’ll end up paying more for the hosting. Hopefully this will help bring it back down.

Update 8:40pm. First thing I notice is that when reading the feed from within the Feedburner site, it doesn’t treat relative paths to images properly. I guess I’ll have to put absolute paths, ‘cos at the moment in the previous post it’s trying to load http://feeds.feedburner.com/files/2007/mediagate-mg35.jpg instead of http://www.geekrant.org/files/2007/mediagate-mg35.jpg. I wonder how it treats relative links?

htaccess Generator

Daniel would love this htaccess Generator, not that he needs it, what with him being a htaccess-loving-geek and all

Outages and response times

Cam ponders web hosting SLAs and wonders what’s reasonable. For his hosting, they guarantee 99.99% uptime, which works out to 52 minutes per year. (His outage was about 9 hours, or about ten years’ worth).

Bad stuff happens. We all know that. Even if it’s the most reliable setup ever. But there’s some major factors in determining what’s acceptable:

Frequency — If it’s happening too regularly, then there’s a reliability problem. They need better hardware, better software, whatever it is, needs to be fixed. Cam reckons it’s the second time in a few months.

Response — Obviously, you want a quick response, and a quick (and reliable) solution. There’s also sorts of monitoring tools out there these days. Typically anything like a full outage should be known about within minutes. A reputable web host will have substitute hardware ready to switch-on and go just as soon as that nice recent backup is restored.

Communications — Any third party like this has to keep the customer informed. There’s no excuse for not doing so. SMS alarms, emails, phone calls, whatever. (I wrote about alarms recently for my work blog.)

BTW, Cam’s also having troubles with his iPod… or more accurately, Apple’s 90 day warranty on replacement units.

I reckon he’s jinxed, myself.

Seeing a new server before re-delegation

One of the weaknesses of WordPress and most other web-configured applications is that unless you want to go SQL or config-file-wrangling, it’s pretty much only configurable via the web, at least for tweaking, importing posts, setting up most of the options. This is a problem when, for instance, you’re migrating an existing site onto WP, and it’s on a new server, as you can’t get to the wp-admin screens.

The way to do it is to hack your hosts file. Once the new server is running and WP is setup on it, find your hosts file and add an entry to the new server. On Windows, this is the c:\windows\system32\drivers\etc\hosts file.

Chuck in a line that says contains your new server’s IP address, and the hostname. Something like:

192.168.0.1 www.evision.com.au

(Whoopsie, real-world example with a fake IP. The new evision site is going live Real Soon Now.)

Save, then away you go. You can see the new site and tweak to your heart’s content, but nobody else will be able to see any of it until you re-delegate.

The catch? It probably won’t work from behind corporate networks, where your computer uses a proxy.

Hurricane Rita

I’ve been notified by my web ISP that Hurricane Rita is approaching Houston. Why does this matter? Because geekrant.org (and a number of other sites I run) are sitting on a server in a data centre in Houston. I’ve been encouraged to take backups of important content, which I’ll be doing. It’s a reminder that regular backups are an essential precaution.

If the site goes down in the next day or two, you’ll know why. Best wishes to those in the affected areas.

More from George

More goodness from George Skarbek’s column in The Age (19-Jul-2005).

A punter asks George about sending large files across the net. One suggestion is to set up a web host, and the reader is sent off to GoDaddy to find out about domain names and hosting fees, and even ponders if a web server should be set up on their own computer. Uhh, but these days but most ISPs provide a basic web hosting facility, good enough as a drop point for leaving big files… surely it’s better to look at this first? Not to mention the many online storage services, such as Yahoo Briefcase.

A question about whether one should turn on IE’s “Do not save encrypted pages to disk” option comes up with some gibberish about “static web pages and dynamic data”. Eh? The point of this option is explained in IE’s help: it avoids the browser saving the pages onto the hard disk where they might be snooped upon by other users on the same computer. Since an HTML page is plain text, and depending on the site used, user or session IDs or even passwords could be embedded in the HTML, in some environments it might be desirable to not save this in the cache.

(Don’t get me wrong; most of George’s answers are spot on. Just a few that haven’t quite lived up to expectations, and it’s been bugging me a bit…)

Pirates! Spammers! Gyroscopes! Bandwidth thieves!

This is officially getting ridiculous. Not only are my blogs getting a lot of comment spam, but my personal blog site is burning huge amounts of bandwidth, as particular (I assume zombie) hosts hit the site.

Below are the top ten bandwidth users of danielbowen.com for June:

Top 10 of 15312 Total Sites By KBytes
# Hits Files KBytes Visits Hostname
1 14380 4.10% 3801 1.77% 111235 2.22% 159 0.24% host-148-244-150-58.block.alestra.net.mx
2 17558 5.01% 3191 1.48% 99441 1.98% 157 0.24% host-207-248-240-119.block.alestra.net.mx
3 3927 1.12% 3640 1.69% 75989 1.51% 3 0.00% csr010.goo.ne.jp
4 3062 0.87% 2797 1.30% 74881 1.49% 171 0.26% rrcs-24-97-174-130.nys.biz.rr.com
5 3057 0.87% 2200 1.02% 62547 1.25% 392 0.60% msnbot.msn.com
6 2691 0.77% 2248 1.04% 60684 1.21% 153 0.23% 64.124.85.78.become.com
7 2256 0.64% 2082 0.97% 56383 1.12% 124 0.19% 98-101-196-200.linkexpress.com.br
8 2146 0.61% 2033 0.94% 51665 1.03% 279 0.43% dsl-250-198.monet.no
9 2001 0.57% 1755 0.82% 47605 0.95% 23 0.04% host133.sprintnetops.net
10 1686 0.48% 1571 0.73% 35979 0.72% 325 0.50% corporativos

It’s not like this site is hosting pr0n or something — there’s just no reason why any single host would need to grab 110Mb of traffic in a single month. In total traffic topped 4Gb for the month, which is ludicrous for a diary site with a few photos on it. 4Gb is actually my monthly limit — thankfully my web ISP isn’t too strict about charging extra for hitting that, but there’s always the risk if this is consistent that it’ll be costing me real money.

As a result I’ve started a list of bandwidth hogs’ IP addresses, which I’m putting in the .htaccess file. Anything with lots of hits and grabbing above about 5Mb per month is going onto the list, and the list is being duplicated (manually unfortunately) across to the other WordPress sites that I run.

Inspection of the access_log is particularly enlightening, with at present a staggering number of requests coming in with a referer at poker-related sites. Of the 6665 hits in the file for today (covering about 13 hours) there are 674 from texasholdemcenteral.com (note the wonky spelling) and 1212 from sportscribe.com. All of these too are now being blocked with a 403 (forbidden) via .htaccess.

Sigh. I suppose it’s just too much to expect people to place nice?

.htaccess extract – Feel free to copy for your own site to block miscreants.
Continue reading