Ben Jao Ming

In case you need a quick way of getting rid of spammers on your apache server, use this method as a temporary solution until you find a better one.

Please take not that it will evaluate EVERY SINGLE REQUEST through a Deny policy in Apache’s mod_access – it does not halt your performance much if you’re just running a site with few requests. But I only suggest this solution as a temporary one until you have a better integration. I am currently working on a Django middleware that will only check the client’s IP address in case it is a POST request directed at a certain URL.

Since I’ve been told that Apache doesn’t use /etc/hosts.deny, you should probably use this method instead, unless you have a webserver that actually uses hosts.deny. Another method would be to add the IP addresses to iptables, if you’re running that — this is probably more efficient than using Apache configuration policies.

This method creates a file that you can include in httpd.conf (to cover ALL your virtual hosts). The overall goal is to avoid .htaccess files since they are evaluated at runtime. Thus, you won’t have to configure each site, and you save a but of CPU time.

1. Copy the script below to a file on your server, eg. /usr/sbin/stopforumspam.py. You might want to read it quickly as it will pretty much explain itself…
2. Make the file executable, eg. chmod +x /usr/sbin/stopforumspam.py
3. Add it to your crontab for automatic execution each night, eg. crontab -e and then insert the line 0 0 * * * /usr/sbin/stopforumspam.py && /etc/init.d/apache2 reload
4. By default, we will ban an entire class C subnet if more than 5 IP addresses are in this space. You can configure this behavior by giving stopforumspam.py a single argument, ie. /usr/sbin/stopforumspam.py 10 would mean that at least 10 IPs have to be within the class C subnet to qualify it for a ban.
5. By default stopforumspam.py creates /etc/apache2/stopforumspam.conf – you can change this by editing the script.
6. Add Include /etc/apache2/stopforumspam.conf to /etc/apache2/httpd.conf (Debian/Ubuntu).
7. Caution! Make sure that you DO NOT change the Order Deny,Allow option in VirtualHost directives or .htaccess files as this unblock the deny policies.

#!/usr/bin/python
 
import re
import urllib
import zipfile
import sys
 
if len(sys.argv) > 1:
    SUBNET_THRESHOLD = int(sys.argv[1])
else:
    SUBNET_THRESHOLD = 5
 
DOWNLOAD_ZIP = "http://www.stopforumspam.com/downloads/listed_ip_7.zip"
ZIP_FILENAME = "listed_ip_7.txt"
 
HTTPD_CONFIG_INCLUDE = "/etc/apache2/stopforumspam.conf"
 
# For security purposes we test that each line is actually an IP address
IP_MATCH = re.compile(r"^(\d+)\.(\d+)\.(\d+)\.(\d+)$")
 
filename, headers = urllib.urlretrieve(DOWNLOAD_ZIP)
 
z = zipfile.ZipFile(filename)
ips = z.read(ZIP_FILENAME)
 
ips = ips.split("\n")
 
# Remove non-ip members
ips = filter(lambda ip: IP_MATCH.match(ip), ips)
 
def get_ip_segments(ip):
    segments_match = IP_MATCH.search(ip)
    return [int(segments_match.group(i)) for i in range(1,5)]
 
def convert_ip_to_number(ip):
    numeric_value = 0
    ip_segs = get_ip_segments(ip)
    for i in range(4):
      numeric_value += ip_segs[i]*(255**(4-i))
    return numeric_value
 
# Sort everything first
ips.sort(key=convert_ip_to_number)
 
subnets = {}
 
for ip in ips:
    ip_segs = get_ip_segments(ip)
    key = (ip_segs[0], ip_segs[1], ip_segs[2])
    if not key in subnets.keys():
        subnets[key] = [ip]
    else:
        subnets[key].append(ip)
 
final_list = []
 
for subnet, subnet_ips in subnets.items():
    if len(subnet_ips) > SUBNET_THRESHOLD:
        # Ban the whole subnet
        final_list.append(".".join(map(str, subnet)) + ".0/24")
    else:
        final_list = final_list + subnet_ips
 
print ""
print "Lengh of original list: %d" % len(ips)
print "Lengh of final list: %d" % len(final_list)
 
apache_conf_file = file(HTTPD_CONFIG_INCLUDE, "w")
apache_conf_file.write("\n")
apache_conf_file.write("    Order Deny,Allow\n")
 
for entry in final_list:
    apache_conf_file.write("    Deny from %s\n" % entry)
 
apache_conf_file.write("\n")
 
apache_conf_file.close()