This description is subject to change and may be out of date.
The description that follows is complex. It is an attempt to explain accurately and in detail the SpamCop Blocking List (SCBL), specifically the SCBL rules and how the SCBL decides to list an IP address. SpamCop provides this description so that email senders and recipients will understand better how and why email is refused, blocked or filtered. We intentionally omit the description of certain processes in order to make it more difficult for senders of spam to evade or "game" the SCBL.
- The SCBL is an aggressive spam-fighting tool. By using this list, you can block a lot of spam, but you also may block or filter wanted email. Because of this limitation, one should strongly consider using the SCBL as part of a scoring system and explicitly whitelist wanted email senders (e.g., mailing lists and other IPs from which you want to receive email).
- With any spam filtering system, you should consider keeping suspected spam so that it can be retrieved. Doing so will prevent bounces from your system hitting innocent third parties.
- New users of the SCBL should read the description below and all other documentation carefully before deciding to use the SCBL.
The SCBL is a list of IP addresses which have transmitted reported email to SpamCop users, which in turn is used to block and filter unwanted email. The SCBL is a fast and automatic list of sites sending reported mail, with a number of report sources, including automated reports and SpamCop user submissions. The SCBL also quickly and automatically delists these sites when reports stop.
The SCBL aims to block spam with minimal blocking or misidentification of wanted email. Given the power of the SCBL, SpamCop encourages users to also actively maintain a whitelist of wanted senders of email. SpamCop also encourages SCBL users to tag and divert email, rather than block it outright. In the end, most SCBL users find that the amount of unwanted email successfully filtered makes the risks and additional efforts worthwhile.
- Reported Email. Mail reported as spam by SpamCop users will be referred to as "reported email" or "reports" throughout this document. The SpamCop reporting tool cannot determine if email reported by users is or is not spam; it can only parse and report email which users give it. SpamCop users can and do make mistakes.
- Spamtrap Reports. SpamCop reports generated as a result of mail sent to non-existent email addresses ("spamtraps") set up by SpamCop.
- Spamtraps. Non-existent email addresses set up by SpamCop to definitively identify spam. As SpamCop never used these email addresses to signup for a mailing list or purchase an item, for example, SpamCop knows spammers harvested the emails for their mailing lists.
- Reputation Points. Part of a scoring system SpamCop uses to weight reported email. A mail sender receives a reputation point for each SCBL query that is not reported as spam.
- Open Proxy. Systems that accept connections from any network address, acting as a blind intermediary to virtually any other network addresses. A growing source of spam, as the anonymous nature of the transaction makes it difficult to track the source of email.
- Open Relays. Typically refers to an e-mail server (SMTP server) that is configured to deliver any incoming mail to another mail server. In the past, open relays (open relay servers) were common, but today, most e-mail servers block all e-mail that does not originate with the customers of the service or employees of the company.
- Whitelist. A list of mailservers from which one expects, wants or needs to receive email. Marking these email senders in one's whitelist exempts these IP addresses from blocking and/or filtering.
The SCBL is a list of IP addresses which have transmitted reported email to SpamCop users. The sending system can be a direct email source (such as a site's primary mail server) or an indirect source (such as an open proxy or open relay that has been abused to send spam). The SCBL weights the number of reports referencing an IP against a sample of the total amount of email sent by that IP. This method is not perfect. For example, some IPs which send a significant amount of reported mail may rarely or never be listed in the SCBL because those IPs also send a lot of non-reported mail.
SpamCop uses a number of report sources, including SpamCop users, spamtraps and websites that use the SCBL. Spamtraps are email addresses that spammers have harvested or created, but the owner of these email addresses never used them to receive wanted email or to subscribe intentionally to mailing lists. SpamCop also monitors queries from a sample of sites that use the SCBL. SCBL users query the SCBL servers during every SMTP transaction. We count the total number of queries for each IP address and whether or not that IP address appears on the SCBL, to generate an estimate of how much email is transmitted by each IP. When a sampled site queries the SCBL about an IP address sending mail which is not reported mail, that host is given a reputation point.
Most of the sites SpamCop monitors send either mostly reported email or mostly non-reported email. The difficult part is deciding what to do with ones in the middle. These few systems account for the most email.
Some blocking lists block mail from misconfigured or insecure servers (such as open proxies or open relays), or from certain classes of machines (such as machines with dynamically-assigned IP addresses). The SCBL does not consider these characteristics. Instead, the SCBL lists only IP addresses of machines that are sending reported email. As a result, IP addresses which do not host a misconfigured or insecure server, but do send reported mail, may be listed. An insecure machine that has never been abused would not be listed.
Timeliness is key to the SCBL's value. The automated queries results in fast listing of spam, which increases the accuracy of the SCBL. Also, without any additional reports, a reported address stays on the SCBL for only 24 hours. This limits the amount of damage if users make a mistake and report legitimate mail using SpamCop.
The system currently operates based on these rules:
- SCBL lists IP addresses with a large number of reports relative to reputation points. The SpamCop team manually balances the threshold in an effort to make the list as accurate as possible.
- The SCBL weights reports depending on how recently the mail was
received (or "freshness"):
- The SCBL counts the most recently received reports 4:1.
- The SCBL counts reports for email 48 hours and older 1:1, with a linear sliding scale between the most recent and 48 hours past.
- The SCBL ignores reports for email received more than one week ago.
- The SCBL uses Spamtrap reports to weight total reports. For
spamtrap scores less than 6, the SCBL multiplies by 5 the quantity of
spamtrap reports and adds this to the report score. For larger
spamtrap scores, the SCBL squares the quantity. Examples:
- If an IP address has 2 spamtrap reports and 3 SpamCop user-reported reports, its weighted score is 13: (2 * 5) + 3 = 13.
- If a host has 7 spamtrap reports and 3 manual reports, its weighted score is 52: (7 * 7) + 3 = 52.
- The SCBL does not count reports regarding URLs or addresses in the body of the email. Therefore, the SCBL does not list websites or email addresses used to receive replies in reported email, unless that IP is also used to send the mail.
- The SCBL will not list an IP address with only one report filed.
- With only two reports against an IP address, the SCBL will list the IP address for a maximum of 12 hours after the most recent reported mail was sent.
- The SCBL will not list an IP address if there are no reports against it within 24 hours.
- If a server sends bounces to an SCBL spamtrap in sufficient quantity to meet the listing criteria, the SCBL will list that server. This situation results as some mailservers do not reject mail during the SMTP transaction, but rather accept the mail and then send a bounce message later. (These servers usually run qmail or postfix). Viruses and spam often contain a forged From: line. If email is rejected or blocked during the SMTP transaction, the bounce will go to the connecting IP. If the bounce comes after the mail is accepted for delivery, then the bounce will go to the address in the From: field. Viruses and spam often use addresses from the list of recipients to populate the From: field. Sometimes, these addresses are spamtraps.
[Append to This Answer]