Forum

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
Current Issues
January 25, 2009
6:00 pm
Admin
Moderators
Forum Posts: 423
Member Since:
July 12, 2008
sp_UserOfflineSmall Offline
  • Mitra has reported some issues in using JunQuilla with a Mac-based theme. I'm currently investigating.
  • With my personal junk mail, I'm finding that the number of personal messages that are appearing in the Uncertain folder are still too small. I've lowered the limit of what I see there down to 5% or higher, and still the vast majority of my messages there are junk. I'm also currently running a limit of 75% rather than the default of 90%. In spite of the fact that personal emails with a junk score of 5% or higher are rare, I still occasionally get a real email that score 50% - 70%, so I am reluctant to lower my limits any further. That may be because the vast majority of my real emails are white-listed, which automatically bypasses the Bayes processing.
  • Bug 471682 has plagued me for a long time, removing the metadata from my junk messages which JunQuilla relies on. But I now have a patch for it, and hopefully it will be added to the tree in time to make Thunderbird beta 2.
February 2, 2010
8:46 am
Member
Members
Forum Posts: 8
Member Since:
February 2, 2010
sp_UserOfflineSmall Offline

Hi, this is my first post. I would love to participate in the discussions, but my prime reason for posting is an (apparent) bug in Junquilla. I am using Thunderbird 3.0.1 on Mac OSX1 10.6.2 with an Intel Mac, Junquilla 1.0.0.

Junquilla appears to be working and solving all my previous problems of interaction between filters and junk processing....  But I have a bug!

Regularly the folder properties of some folders change, seemingly randomly. The most important folder is the one that needs junk processing, and its properties change (all by themselves) from Analyze Junk "Enabled" to "Inherit". To make sure that the folder's status is being saved, as a frequent test I will quit Thunderbird, relaunch it and inspect the folder's properties. It comes up fresh as "Enabled" but after an hour or so of email activity, it goes to "inherit" and then junk processing does not happen with this folder. 

I can then manually run junk processing on the folder and reenable the "Enabled" checkbox and things will run successfully for a while, but eventually the folder reverts to "inherit". I've seen other unstable properties checkboxes on other folders in this one's vicinity, mostly changing to "inherit" when I had wanted both checkboxes to be unchecked.

Is there a log I can inspect that would help us debug this?

Here is a brief description of my folder structure:

Using the global inbox, all mail comes in. A "local folders" set of filters moves a lot of the mail to various prefiltered inboxes. All of those destination inboxes have (or are supposed to have) their "Enable Junk Processing" turned off. Then there is a "catch-all" box where the remainder that didn't get filtered gets sent and this "catch-all" box is supposed to have its junk processing Enabled. It is this catch-all box whose junquilla property seems to change all by itself.  

Hope this helps!

February 2, 2010
10:19 am
Admin
Moderators
Forum Posts: 423
Member Since:
July 12, 2008
sp_UserOfflineSmall Offline

I have not heard of this issue before. I strongly suspect that this is an issue of corruption of the folder database by the core program, rather than JunQuilla itself resetting that value. I tend to be involved in issues like this in the core as well, so I'm interested in any case.

I don't know of any logs that could be used for this. But let me ask you this - do you see an evidence of database corruption in the folder that is losing that information? Specifically, if you look at the junk percent column in that folder, are any of the old values for emails lost?

Also, do you see any notices in the Error Console that might be relevant?

February 2, 2010
10:49 am
Member
Members
Forum Posts: 8
Member Since:
February 2, 2010
sp_UserOfflineSmall Offline

Thanks for replying, Mr. Kent. By the way, the folder in question has been "stable" now for about 4 hours, so sometimes it keeps its enabled status for a long while. As for Junquilla's junk percent column in the suspect folder, which I barely know how to interpret,older emails are either at 100% or no entry. The newest 10-20 emails in that folder are all no entry (no value in there).

As for errors, I'm beginning to think they are related to compaction. I have my compaction set to ask the user and so I see frequent warnings when this folder is manually checked for junk and it wants to compact the folder when these messages are moved ("deleted") from the folder. At 10:20 this morning there were a whole slew of error messages, all with the same theme. I'm wondering if the header in question is the "Junk Enabled" attribute?

2010-02-02 10:20:40    gloda.index_msg    WARN    Observed header that claims to be gloda indexed but that gloda has never heard of during compaction. In folder: mailbox://nobody@Local%20Folders/%20Prefiltered%20Inboxes/Inbox%20%28Consolidated%29 sketchy key: 14091443 subject: Refund and Return of Equipment

There was a whole ton of these "sketchy key" error messages, which now I'm betting occurred when I said "yes" to the "do you want to compact this folder" query.

If the folder is somehow corrupt, I can copy all the emails that are in that folder into a new one and try some things if you'd like. This is a fairly new Thunderbird install that I built its structure from scratch and then painfully copied folders into it from my version 2 build, because I was getting some corruptions in the account structure after the upgrade to v3. Those corruptions are gone.

February 2, 2010
11:09 am
Member
Members
Forum Posts: 8
Member Since:
February 2, 2010
sp_UserOfflineSmall Offline

Addendum: I just did a manual compact on that folder and got a slew of these error console messages. I'm betting they are directly related to our bug. But junk processing is still Enabled. I wish it would unenable itself so we could find out when it happens.

February 2, 2010
11:59 am
Admin
Moderators
Forum Posts: 423
Member Since:
July 12, 2008
sp_UserOfflineSmall Offline

It seems then like gloda is also struggling with the status of your message database, which adds to my suspicion that this is a more fundamental issue of the message database, rather than a JunQuilla issue.

Although I hate to recommend this, could you try deleting the database file for that folder giving you troubles, and then reindexing that folder? In doing so, you will lose information about the junk status of the individual messages as well as some gloda information, but if the folder has either all junk or all good that should be OK. You will also probably lose the "analyze junk" status for that folder and need to recreate it.

This used to be common practice, but we are trying to make the message database more stable, so it is rarely needed now. But you seem to have issues that need some radical fix.

The message database file has an extension ".msf" I cannot be responsible for the consequences, so you need to do whatever backps you feel are appropriate.

February 2, 2010
4:29 pm
Member
Members
Forum Posts: 8
Member Since:
February 2, 2010
sp_UserOfflineSmall Offline

Thanks for the recommendation. I'm not afraid of a little .msf. And the junk analysis status of that "catch-all" folder reverted to "inherit" sometime in the past few hours so we shall see. First I deleted the .msf of the catch-all folder, relaunched Thunderbird and set the folder's status to "enabled". Then I checked some other folders near that tree branch and saw that these had also changed (spontaneously) their "process junk" properties to Inherit, so I quit Thunderbird again and deleted their .msf files.

Unfortunately, I fear the problem is more serious. When I quit Thunderbird in order to delete the .msf files for the other itchy folders I DID NOT touch the .msf for the "catch-all" folder. (As we know, its index had just been rebuilt). HOWEVER, when I relaunched Thunderbird, the catch-all folder had reverted to "inherit". This does not look good.

February 2, 2010
4:55 pm
Admin
Moderators
Forum Posts: 423
Member Since:
July 12, 2008
sp_UserOfflineSmall Offline

Here's another suggestion. Folder properties are cached in a database file panacea.dat Sometimes it can get corrupted and cause grief. I understand that you can delete panacea.dat and it will get recreated, though that is not something I have done.

Can you try that? I would also delete the .msf files at the same time to really make sure you are starting from a clean slate.

February 3, 2010
10:11 am
Member
Members
Forum Posts: 8
Member Since:
February 2, 2010
sp_UserOfflineSmall Offline

Gotcha. Thanks. I fear all the "not read" status will get lost, but even then my folder structure is such that I can recreate the "read" status for 99% of my stuff. Pain in the ass, but we'll try it. What about the "rebuild index" command in the properties window? I assume this is not necessary if the .msf file was recently deleted, right?

February 3, 2010
10:21 am
Admin
Moderators
Forum Posts: 423
Member Since:
July 12, 2008
sp_UserOfflineSmall Offline

Most per-message variables have a method they are stored outside of the .msf files. I'm almost certain that "not read" is one of those.  I AM certain that "junk percent" does not have any storage outside of the .msf file, which is why it is a good measure of a catastrophic loss of the .msf files. I did a lot of work in TB3 to prevent loss of data from .msf files, which was quite common in TB2.

Re-indexing is the normal step to take to fix issues. Yes in theory you should not have to do it if you deleted the .msf file. I TB3 it should be a safe operation though that does not lose any data.

panacea.dat does not store per-message information, but per-folder information. They are closely related though as the same .msf file is used for both. I am less familiar with it though than with the .msf files. But in general, I think that you be doing all of this on a backup copy of your profile if at all possible, as this kind of surgery can have unexpected bad effects.

I am going under the theory that you have issues beyond just getting JunQuilla to work. This certianly not be considered normal methods to just get JunQuilla to work.

rkent

February 3, 2010
10:32 am
Member
Members
Forum Posts: 8
Member Since:
February 2, 2010
sp_UserOfflineSmall Offline

panacea.dat turned out to be a BAD file to delete...   I got crashes until I restored that file. (Yes, I deleted it when Thunderbird was not running).

Bottom line: I'm seeing random change of "analyze junk" status on certain folders properties. If this goes on I'm afraid I'll have to get rid of Junquilla, and see if another solution will work. My motivation for installing Junquilla was to prevent mail that's being filtered to a new inbox from being tested for junk.

On Bugzilla, when I reported what I thought was a bug in Thunderbird, you were kind enough to report that mail filtering could be used to set the "not junk" status of any files that are being filtered to new inboxes. Will this truly prevent the junk analyzer from junking any files that are being moved to those new boxes?

February 3, 2010
11:04 am
Admin
Moderators
Forum Posts: 423
Member Since:
July 12, 2008
sp_UserOfflineSmall Offline

I tried deleting panacea.dat on two test profiles, then on my main profile which is very complex and old. In all cases, it was just recreated again, and I saw no noticable change to my Thunderbird operation (and certainly no crashes). That fact that you are crashing just reinforces my belief that there is something seriously wrong with your setup beyond any issues you are having with JunQuilla.

If I was you, or you were my client, at this point I would recreate your whole profile. That is, I would start from a new blank profile create the accounts again, then move the folders with local mail files into the profile. But also if I was you, and convinced that all of this was associated with JunQuilla, I would not do that but just abandon JunQuilla instead.

The good news about crashes is that the logging is excellent. Do you have the crash ID on the crash? You can look up your crash IDS, if any, by setting about:crashes as your default startup screen. With the crash ID, I can look up your crash on the Thunderbird crash-stats website and see where it happened.

"On Bugzilla, when I reported what I thought was a bug in Thunderbird, you were kind enough to report that mail filtering could be used to set the "not junk" status of any files that are being filtered to new inboxes. Will this truly prevent the junk analyzer from junking any files that are being moved to those new boxes?" Yes it should.

February 5, 2010
9:05 am
Member
Members
Forum Posts: 8
Member Since:
February 2, 2010
sp_UserOfflineSmall Offline

" rkent said:

I tried deleting panacea.dat on two test profiles, then on my main profile which is very complex and old. In all cases, it was just recreated again, and I saw no noticable change to my Thunderbird operation (and certainly no crashes). That fact that you are crashing just reinforces my belief that there is something seriously wrong with your setup beyond any issues you are having with JunQuilla."

Right, it is my diagnosis, too. But this profile was recently created from "scratch" so it is practically virgin. That's not to say there aren't potential issues, but I think it is more likely something to do with some unpredictable interaction between JunQuilla and my system configuration, not trying to point fingers. I prefer to try uninstalling Junquilla, change my filters and cross fingers. I can always start from scratch with a new Profile all over again if none of this works for me. It only takes a couple of hours  :-(.

"The good news about crashes is that the logging is excellent. Do you have the crash ID on the crash? You can look up your crash IDS, if any, by setting about:crashes as your default startup screen. With the crash ID, I can look up your crash on the Thunderbird crash-stats website and see where it happened."

I appreciate the offer. I just looked at some of the crashes. I think the only logged crashes are older than the event, and we're talking about chasing something to do with panacea.dat, I think I'd rather let sleeping dogs lie and not bother you. You have better things to do :-).

"On Bugzilla, when I reported what I thought was a bug in Thunderbird, you were kind enough to report that mail filtering could be used to set the "not junk" status of any files that are being filtered to new inboxes. Will this truly prevent the junk analyzer from junking any files that are being moved to those new boxes?" Yes it should."

Thanks. Unfortunately now I'm up to that point as one of my folders keeps on losing the "analyze junk" status... I really wanted to keep Junquilla, but something's not comfortable with it in my system. In another topic thread you replied to me that I should set all the relevant folder properties to "inherit" before uninstalling Junquilla. Yes, I will change them to "inherit", but I am under the impression that those special folder properties (e.g. "analyze junk") were created by Junquilla so can you please tell me exactly how those properties may be interpreted when Junquilla is uninstalled.

Thanks for all your time. In another life I think I would be very happy with Junquilla. I think we all spend 10% of our lives now watching a progress bar!


Forum Timezone: UTC -8

Most Users Ever Online: 41

Currently Online: ritaHinc
2 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

BigMike: 14

David.P: 10

Jeff Wexler: 9

taa: 8

JPRuehmann: 8

bobkatz: 8

Member Stats:

Guest Posters: 217

Members: 2350

Moderators: 2

Admins: 1

Forum Stats:

Groups: 1

Forums: 7

Topics: 375

Posts: 1220

Newest Members:

AlbertKet, Kevintuh, LazaroVag, elinorgb1, Niki1Kevick, AnthonyPaino

Administrators: rkent: 423