Spam in Discussion Forums

We’ve recently had a problem with spam in the Mozilla discussion forums, coming in from Google Groups. Here’s a document which outlines why this isn’t entirely easy to solve, and suggests a way to deal with the problem. Comments welcomed.

11 thoughts on “Spam in Discussion Forums

  1. If MailMan automoderates the list to let in anything that passes the filters, why is the global whitelist necessary?

    Assuming that it is necessary, isn’t the cost of adding new NNTP posters ongoing, not just a one-time operation?

    I’d suggest doing it with two test groups first to be sure that there aren’t weird or unintended side-effects when things are posted to multiple groups by either the same media (news & news, mail & mail) or different media (news & mail). Failure modes to look out for include bounced messages and/or broken threading.

    One thing to like about this proposal is that even if SpamAssassin isn’t good enough, there are other anti-spam things we could try as well (eg Postini).

  2. Definitely agreeing with dmose on the testing first :) Even more agreeing on stopping the spam, though! One or two messages per day per group/list kind of adds up… Around 30 groups I’m subscribed to with spam today.

    I’m also curious just how much moderation is necessary – if the posters could approve their own messages via some sort of web interface, would it still work? Obviously this would be some sort of stage 2 thing that shouldn’t block the solution being implemented right now, of course.

    Not actually relevant: gmane also does mail/news gatewaying (I’m using it for a sourceforge mailing list right now), and IIRC requires you to reply to an email to verify it before you’re allowed to post. Not relevant because, as far as I can tell, most of the spam (both this round and in previous instances) are coming from google groups, but it may be useful in the future if it shifts.

  3. Isn’t this whole arrangements superovercomplicated? Why not just having (as everyone else) an email list (on your own mailman, if you wish) and then just use gmane (as everyone else) to have both web and NNTP interface?

  4. The proposed solution sounds pretty good. I’d suggest to try it on a middle-frequently used newsgroup to see if there are any weird situations where posts get unintentionally blocked. Giving it a try shouldn’t be harmful if you don’t try it on newsgroups like m.f.support first.

    Who is responsible for the newsgroups? ATM I have some free time and could help implement and testing.

    Sincerely,

    Michael Kohler

  5. Dan: Some very good points.

    Matěj: In the past, we’ve rejected that suggestion because of perceived deficiencies in gmane’s web interface/searching. Also, if we stopped running our own news server (which, I guess, that plan doesn’t in fact require) then getting everyone to reconfigure their news clients would be somewhat disruptive.

    Michael: I’m responsible for the groups if anyone is; various members of the IT team like jlazaro, shyam and justdave do the technical heavy lifting.

    Gerv

  6. We already have a few groups set up this way that have been for a couple months, so if you want to consider that having it tested already…

    mozilla.dev.security and mozilla.dev.security.policy

  7. I realize now that I hadn’t totally understood your proposal when I initially replied. As far as ongoing cost of moderation, in tb-planning, at least, that cost is not very high, and it’s been working fine in large part just by having moderators who are able to respond (relatively) quickly, such as within a day. I’m fairly happy with the system as it stands, though it wouldn’t scale indefinitely.

    Matej’s suggestion has merit as well, IMO, particularly since people could continue to use the Google Groups for the web interface and we could just take advantage of the NNTP-portion of gmane, assuming the GMane folks were up for that.

    Is there anything I need to do to be notified of subsequent comments to this blog post? I was expecting emails, but haven’t seen any.

  8. What does the Report Spam button do in Google Groups? I click it whenever I can on spammy-posts but I’ve never really checked back to see if anything happened to them.

  9. If, as dveditz explains, mozilla.dev.security and mozilla.dev.security.policy are both running this setup already then lets consider that a successful implementation and switch the others over asap.