Hi,
I'm one of the moderators of the fedora-mingw mailing list. We've set up our mailing list with this policy: Only subscribed people can directly send mail to the list All mail sent by non-subscribed people needs to be approved by a moderator first
In the last couple of weeks we've noticed a large increase in the amount of spam which gets sent to the moderation queue.
I've create an overview of the amount of messages which have ended up in our moderation queue in the last couple of weeks:
{{{ 2013-05-18: 3 2013-05-19: 12 2013-05-20: 12 2013-05-21: 16 2013-05-22: 14 2013-05-23: 16 2013-05-24: 11 2013-05-25: 17 2013-05-26: 11 2013-05-27: 13 2013-05-28: 15 2013-05-29: 23 2013-05-30: 6 2013-05-31: 8 2013-06-01: 8 2013-06-02: 14 2013-06-03: 16 2013-06-04: 12 2013-06-05: 18 2013-06-06: 14 2013-06-07: 11 2013-06-08: 18 2013-06-09: 5 2013-06-10: 12 2013-06-11: 21 2013-06-12: 20 2013-06-13: 36 2013-06-14: 36 2013-06-15: 42 2013-06-16: 29 2013-06-17: 36 2013-06-18: 44 2013-06-19: 38 2013-06-20: 34 2013-06-21: 38 2013-06-22: 34 2013-06-23: 37 2013-06-24: 47 2013-06-25: 32 2013-06-26: 43 2013-06-27: 49 2013-06-28: 32 }}}
These numbers make it hard to properly moderate a list. From time to time also legit mail ends up in our moderation queue (like from upstream developers) so we have to be careful not to accidentally drop these legit mails.
Would it be possible to add automated spam filtering to mailman so that mail which is (very likely to be) spam will automatically get rejected/dropped? All mails of which it isn't certain whether it is spam or legit can then be sent to the moderation queue so that list moderators can manually approve/reject these.
Perhaps something like http://www.jamesh.id.au/articles/mailman-spamassassin/ could be implemented in the Fedora mailman infra
We have a similar problem (although in lower proportions) on the fedorareview mailing list. We often have more than 10 spams per day (that's the limit mailman notifies us that the spam will be discarded), I don't think we reach to 30 or 40 though.
They all have asiatic fonts, I did not check the headers to see if there is some common recurring origin.
Yeah, most of the spam we see these days on fedora-mingw is using asiatic fonts as well. All these mails have unique 'From:' addresses which make it impossible for us to properly filter them using the regular mailman interface
Couldn't we have amavisd-new + SpamAssassin in front of the Mailman setups? We IIRC anyway have Postfix, so this would fit. Amavisd-new could be configured to reject spam e-mails in a socalled pre-queue-content-filter setup. And one or the other DNSBL would also add some benefits with less CPU/RAM costs.
We could go the system wide route, but then there wouldn't be a chance to have per list settings in case some lists are wanting to be more lenient than others. If we keep the discard level very high that might be ok though.
Is anyone noticing any particular networks/blocks that are to blame for the increase?
Replying to [comment:4 kevin]:
We could check the fedora-review list, there are ~30 spam email waiting for approval (was busy this week-end and I have cleaned the list yet). I can keep it up until we process it :)
I have to second this request. The amount of spam we receive to sssd-devel is roughly one mail per hour. There is a real danger that when purging the moderation queue, we'd also flush a genuine e-mail by a person who just didn't subscribe.
Is there any ETA on fixing this?
I don't have a specific ETA. It's on my list, but I need to get stuff higher up on that list done before I can get to it.
So, after some thinking today, I put in place something that may help. ;)
I added a postfix content-filter to our list server to pass every email through spamassassin.
spamassassin will add 3 headers to the emails:
{{{ X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on collab03.fedoraproject.org X-Spam-Level: * X-Spam-Status: No, score=1.3 required=10.0 tests=HTML_MESSAGE,RDNS_NONE autolearn=no version=3.3.1 }}}
So, you should be able to add spam filters to your list to suit your needs. Either a filter that looks for "X-Spam-Status: Yes" or a number of *'s in X-Spam-Level and either moderates or discards.
Can you all test this out and see if it meets you needs?
It's nice to see that SpamAssassin is in place again. Thanks Kevin!
SpamAssassin
I added filters to a few of the lists I administered when they were still hosted at redhat.com. For anyone that wants to use this, it might be worth noting that Mailman 2.1 doesn't match "'''{{{^}}}'''" the way one might expect^1^.
What I've used successfully in the past:
{{{ \nx-spam-flag:\s+yes }}}
This is placed in a rule under Privacy options -> Spam filters. I set the action to Discard.
Discard
Similarly, to use the X-Spam-Level header, something like this should work:
X-Spam-Level
{{{ \nx-spam-level:\s+*** }}}
The regex matches are done case-insensitively, as well.
^1^ At least, it didn't the last time I checked -- if I'm mistaken and that's now been changed/fixed, I'm happy to be corrected. I know the issue was known upstream.
Thanks for implementing this. I've configured the mingw mailing list to match content based on the first regex which was mentioned above by tmz. I'll let you know in a couple of days whether this gives the desired effect for us
ok, this seems to be working to me.
Please feel free to open new ticket(s) if there's any changes needed.
Login to comment on this ticket.