The World’s Greatest Blog

Until we started running our own servers, and generally having many more pages (over 250,000 at the last count), it wasn’t really that important if our little old blog or website was hit by a few extra hits from rogues and bad bots.

Now it is.

Today, on one sub-domain alone at one stage we had over 400 hits from bots sent all from the same IP range. What they were doing, I do not know, whatever it was, they weren’t our old friend the Googlebot, or Slurpy (or whatever Yahoo calls their search spider) and they were putting some severe strain on our pretty hefty server with their persistent requests (opening a page about every 1.5 seconds - remember that there were over 400 on at once!) - that’s quite a load.

The fix?

It’s actually relatively simple (for we apache webmasters), using our httpd.conf file, and adding the following (I have removed the IP addresses for now, as I’m not sure they were up to anything naughty, but I will investigate more later):

<Directory /home/default/*/user/htdocs>
Order allow,deny
Allow from all
Deny from ROGUE IP RANGE
Deny from ROGUE IP RANGE
Deny from ROGUE IP RANGE
Deny from ROGUE IP RANGE
</Directory>

Obviously, where I just wrote ROGUE IP RANGE, you will want to put the IP address to be blocked (from your log files, or whatever tracking method has shown up the larger than life usage!). The ‘*’ is a wildcard for all of the different domains under this particular server - it basically covers all of them.

Blocking a range

I tried various methods which were suggested, though the following seemed to be the only which worked:

IP range to be blocked = 111.222.333.000 - 111.222.333.444 (a huge range I know, but that’s equivalent to what one of them had) - I simplyentered the following:

Deny from 111.222.333

Notice I just left it at the common start. It is POSSIBLE that this will block some bone fide users, but that is preferable to the alternative of not blocking the old scoundrels.

Please feel free to improve my clumsily cobbled httpd.conf method in the comments box! I didn’t even know what httpd.conf meant until early 2008 - imagine picking up the Apache manual for the first ever time. That’s not a learning curve: that’s vertical.

Enough coding chat, just thought I’d try to help someone in a similar position - I couldn’t find this until taking hours of effort!