Help | Site Map
| Text size: - +
(Answer) (Category) SpamCop FAQ : (Category) SpamCop Blocking List information :
Is it possible to download the entire blocklist?

Note: This method is only for customers willing to pay for direct access to the blocklist. If you want information on using the blocklist in the normal, casual way see the How do I configure my mailserver FAQ.

The SCBL is a constantly changing real-time list. Therefore, downloading the whole list (doing a "zone transfer") is not as effective at blocking spam unless it is done every minute. Downloading it every minute would be very inefficient.

For this reason, we provide a more efficient option of running a mirrored server through transfer using rsync and ssh. You have the option of the mirror server being public or private as described below. rsync access is provided free of charge to sites willing to host a public mirror of the BL.

Private Mirror BL Server

The bad news is that we charge $1000 per year per server for this service. To pay for private mirror access, make a $1000 payment through PayPal:
Make a payment with PayPal.
Be sure to include the email address you will be contacting us from in your PayPal payment to assist us in matching the payment to you.

First, create a new user and an ssh key:
useradd -G nobody bl
mkdir /home/bl
chown bl.nobody /home/bl
su - bl
ssh-keygen -t dsa -N "" -f ~/.ssh/id_dsa
cat ~/.ssh/id_dsa.pub

Now, send us us the key. Include your Paypal receipt number and username so we can match the payment on our end.

Once you have been authorized to retrieve the blocklist, retrieve it once manually, accepting the host key from SpamCop (if it matches):
rsync -e ssh -i -L bl@blrsync.spamcop.net:bl.data ~/bl.data

Or if you prefer to fetch the data in rbldnsd format: rsync -e ssh -L bl@blrsync.spamcop.net:rbldnsdbl.data ~/rbldnsdbl.data

You should see:
DSA key fingerprint is cb:d6:6f:03:39:5c:8b:f5:b0:56:a9:22:bd:f3:57:65.
or
RSA key fingerprint is 13:04:f8:93:08:f2:a0:dd:9d:1f:23:c9:10:28:a8:c3.

If you do, answer yes. If you do not (very unlikely), abort!

You should now have an initial bl.data file in the user's home directory.

Now configure a cron to rsync this file every minute:

As bl user:
crontab -e

Add to file:
* * * * * rsync -e ssh -L bl@blrsync.spamcop.net:bl.data ~/bl.data

Now, you will have a constantly-updated mirror of the SpamCop blocklist file You can do with it what you like (except share it with others). Ask to be added to the mailing list for mirror operators. Critical updates will be sent to this list when access information changes. Also monitor the cron output for errors (redirect bl's email to an address you read). You may get a few failure errors when Internet connectivity is congested or down. This is normal (the whole point of running a mirror).

One possible use is to republish it internally using the same software SpamCop uses, rbldns.

Here are some very brief instructions on setting up rbldns. For more detailed information, please see D.J. Bernstein's documentation.

# install rbldns and set up a cron which runs as the "bl" user:
# first daemontools from: http://cr.yp.to/daemontools/install.html
# (as root)
mkdir /package
chmod 1755 /package
cd /package
wget http://cr.yp.to/daemontools/daemontools-0.76.tar.gz
tar -xvzf daemontools-0.76.tar.gz
cd admin/daemontools-0.76/
package/install
package/run.rclocal
# tail /etc/rc.local for startup command instead of rebooting.
# next, djbdns from http://cr.yp.to/djbdns/install.html
cd /usr/src
wget http://cr.yp.to/djbdns/djbdns-1.05.tar.gz
tar -xvzf djbdns-1.05.tar.gz
cd djbdns-1.05
make
make setup check
rbldns-conf bl bl /var/rbldns YOURIPADDR spamcopbl.YOURDOMAIN.TLD
chown -R bl.nobody /var/rbldns/root/
rm /var/rbldns/root/data ln -s ~bl/bl.data /var/rbldns/root/data
ln -s /var/rbldns /service

Now, edit your cron, add "; make -C /var/rbldns/root > /dev/null" to the every-minute cron, so that it will rebuild the rbldns data file after every sync. Make sure data.cdb remains up-to-date. You should now have a working DNS mirror of the SpamCop data under spamcopbl.YOURDOMAIN.TLD. You will need to set up NS records pointing to YOURIPADDR for spamcopbl.YOURDOMAIN.TLD.

Public Mirror BL Server

Free mirror service is provided to sites who are willing to host a public mirror (serving data to other free users).

To host a public mirror we require:

If you are interested, please contact us with a brief description of your network and the server. We will configure and maintain it with the software needed to provide the SpamCop blocklist service. Cable modem or xDSL connections are not sufficient.


[Append to This Answer]