Greylite effectiveness

A quantitative measurement of the effectiveness of greylite is provided in the home page. This page explains how the graph has been derived.

The experiment assessed three scenarios:

1. a domain that is not protected by any form of greylisting
2. a domain that is protected by greylite's standard greylisting (every first message from a server is temporarily rejected once)
3. a domain that is protected by greylisting customized with a set of suspicion rules (see suspicion)

Without protection

This represents the amount of SPAM per day received by a mailserver that is not protected in any manner.

It means that greylite does not help anyhow in filtering SPAM. Filtering, if any, is only performed by the MDA itself. In the case of the specific experiment, the MDA is only applying SPF.

This is accomplished by using

0 e r:@domain.com$

With plain greylisting only

This represents the amount of SPAM per day received by a mailserver protected with a standard greylisting tool. The standard, unconfigured installation of greylite gives similar results.

It means that standard greylisting is applied: every message is rejected the first time with a temporary error, then accepted.

This is emulated again with a suspicion rule, of this form:

1 e r:@domain.com$

With customized suspicion ruleset

This represents the amount of SPAM per day received by a mailserver protected by greylite configured with suspicion.

It means that every delivery attempt is looked up against a set of rules configured by the administator. When a rule matches, the client is said suspicious and the rule decides how many attempts to require to the client. This avoids the workaround used by spammers to issue delivery twice or thrice consecutively.

The suspicion ruleset used for the experiment (as used for all the other domains on the mailserver):

# 15 times for clients tricked by greetdelay or
# repeating the deliveries immediately
15 b greetdelay retryinterval
# 6 times for clients blacklisted. The dnsblenv module
# polls zen.spamhaus.org and dnsbl.sorbs.net
6 v BLACKLISTED
# 2 or 3 times for servers whose PTR domain is suspicious (contains "dynamic"
# or "dialup" parts)
3 r (^|[^a-z])(a?dsl|dyn(amic)?(ip)?|dial(in|up)?|ppp|customer|user|host|home)([^a-z]|\.?$)
2 r (([0-9]{1,3}[-.]){3})[0-9]{1,3}
# 4 times for servers outside the business of this server (Europe and US)
3 g ! AT BE CH DE ES EU FI FR GB IT MC NO SM VA US

Mind that the first matching rule wins.

These are some statistics on the effectiveness and probability to match of each rule in this ruleset (you can get your values with the scan.sh script):

5 rules, 61211 total connections, 48762 found suspicious
       rule#      matches   %-of-conns %-of-matches
           1         4143     6.760000     8.490000
           2        42733    69.810000    87.630000
           3         1037     1.690000     2.120000
           4          279     0.450000     0.570000
           5          510     0.830000     1.040000

Results show that:

• the blacklisting rule is by far the most effective
• the client behaviour analysis brings a good contribute
• the remaining hostname analysis is almost negligible
• the GeoIP has a visible, yet small contribute

Mind that this doesn't mean that PTR rules are ineffective at all. Rules in higher position win, thus if the client behavior and the blacklist rules earlier were missing the PTR would have had most likely a much higher effectiveness.

Graphics

Results are summarized by the following graphs. The red line represents the without protection scenario, the blue line represents the with plain greylisting scenario, the green line represents the with customized ruleset scenario. The first graph is linear to give the impression. The second graph is logarithmic to give the exact values.