Junk management for newsgroups in Thunderbird 3

July 21, 2010 – 8:14 am

Thunderbird since version 3 has had experimental support for junk filtering in newsgroups. The feature basically works fine, but the user interface mostly fights against your attempts to use it. I’d like to give brief instructions here for anyone who wants to try it.

You’ll need to install my addon JunQuilla to enable one critical piece of user interface. JunQuilla supports a folder property that lets you selectively enable or disable junk processing for a tree of folders. So after you’ve installed JunQuilla, enable processing of junk for a newsgroup:

This will run future posts sent to the newsgroup m.d.a.thunderbird through the bayes junk filter in Thunderbird. After this is enabled, some of the junk management controls on the folder should be enabled. Try “Run Junk Controls on Folder” to process existing posts for junk.

But nothing will appear to happen when you do, because there is no functionality to delete or hide the junk messages for newsgroups built in. Still, you can see the results using JunQuilla’s “junk percent” and “junk status +” columns:

The next step is to remove those junk messages from your view. The easiest thing to do is to create a virtual folder that shows the messages in the newsgroup without the junk messages. This is where the user interface fights against you. But you can trick it in the following manner.

Create a virtual folder on a message folder with search criteria “Junk Percent < 80″. Save the folder as a subfolder of a local mail folder. Now open the virtual folder and change the folders that it scans, removing the original folder and replacing it with the newsgroup folder:

Now the spam posts are hidden from this virtual folder:

You may need to specifically train the junk filter using a few junk and good messages in news. But the good news is that newsgroup spammers are not really optimizing against bayes filters, so it seems to be a lot easier to detect newsgroup spam than mail spam.

One Response to “Junk management for newsgroups in Thunderbird 3”

  1. JoeS says:

    I gave this a try using a rather “long-in-the-tooth” system.(P3-S 1.4 gig)
    Running junk controls on folder seemed to run those controls on other folders as well as the selected newsgroup (probably just some settings) but I stopped the process after an hour or so.
    Problem 1
    Running the Junk analysis seemed to re-download every header in the newsgroup. For the newsgroup that I tried (mozilla.general)that amounted to 25,747 headers.

    The killer was, that after running the junk controls, just opening that newsgroup takes about 15-20 seconds at 100% cpu utilization. I assume this is because of the overhead in reading the msf for the junk score.

    I didn’t try setting up the virtual folder.
    Also the %junk numbers seemed wrong, although maybe that was because I didn’t let the analysis run to completion.
    For instance, some of my posts showed as 50 while the real junk showed as 0 or 1

    This is just an FYI as I’m thinking that I just don’t have the CPU power to really utilize this.

Leave a Reply