Filtering changes for Thunderbird 3.0 beta 3
July 8, 2009 – 10:37 amIt’s been a long time since I posted a blog, being busy with things I wanted to get into Thunderbird 3.0 beta 3 (and Seamonkey 2.0 beta 1). Now that we enter the dark days of the freeze prior to the release, I have some time to update extensions to use new features available in beta 3. But I’d like to give details first of changes in the backend areas where I am working, starting with email filtering in this post.
So here are things that are new in Thunderbird 3.0 beta 3 (Seamonkey 2.0 beta 1) that involve message filtering:
1. Imap filtering on folders that are not the inbox.
If you have a server filter that moves IMAP messages to a folder other than the inbox, previously you could not use TB filters on it. Now you can, controlled by an inherited folder property. There is no user interface for this feature at the moment, but I will add some to FiltaQuilla. See bug 257415.
2. “From, To, Cc, or Bcc” filter/search term
This is a new search term that can be used in advanced search or filters. It does two things: combines all of the normal address fields into a single term, plus adds support for Bcc for the first time.
The support for Bcc needs some comments. Just in case it isn’t clear, Bcc information is not added to outgoing emails that you send (or the incoming emails that you receive), so the only place it really shows up is in emails that you sent, and have kept your own copy of. So the Bcc term does nothing for emails that you receive, only emails in your sent folder. Also, we currently do not have any mechanism to automatically apply filters to messages that you send, so really the main place Bcc will show up in filter/search is in Advanced search, saved searches (virtual folders), or in manual filters (filter-after-the-fact).
3. Junk fields in manual filters
We have now enabled the junk-related search terms of “Junk Status”, “Junk Percent”, and “Junk Status Origin” in manual filters.
There is a subtle change to the behavior of search term editing that was implemented that affected this. Previously, if you caused the filter editor to select an invalid search term (typically by changing the filter context away from “Manual”), that was ignored, and you could save the invalid filter. Now, the invalid terms are grayed out, and attempts to save the filter will result in an error dialog. This behavior was important to allow adding of more complex filtering options.
4. Custom Search Terms
The biggest change that affects filters is the ability for extensions to add custom search terms. This is a complement to the addition of custom filter actions, which were available in beta 2, and are the main point of FiltaQuilla. I expect to do a blog post soon detailing the code necessary for an extension to add a custom search term, plus I will add some custom search terms to FiltaQuilla (probably a regex search on a header).
See bug 495519 for details, including a demonstration extension that adds custom terms.
5. Fix of message copy/move bug.
There has been a long-standing bug involving failed moves or copies, that typically occurred when there were automatic copies or moves of emails, such as are done in filters or junk processing. Bug 497622 hopefully solves this, which should improve the reliability of moves and copies.
What did’t make beta 3, but should be in TB3.0
There are two big changes that have mostly working patches, that did not make beta 3 but hopefully will be available in beta 4.
1. IMAP body filters.
IMAP messages are now downloaded by default, so in theory it should be easy to filter on their bodies. This is bug 127250 (or possibly the closely related bug 67421). This bug is really critical in combination with the new custom search attributes, as it would be very valuable to write filters that analyze each message body, and then take a particular action when something is found there.
2. Post-bayes incoming filter context
Currently, it is not possible to filter on any of the message characteristics that are determined by the bayes filter (including junk status, and the closely related custom traits) automatically while a message is incoming. This is because the bayes analysis, and its related filtering actions, are applied after the standard filters. A post-bayes filter context will allow the user to specify that a particular filter should be applied after the bayes analysis, not before, and then we will be able to enable the junk-related search terms (and an extension could add a filter term that used a custom bayes trait). See bug 198100.
Are the new filters coming to the views menu as well? From/To/Cc/Bcc would be especially helpful there.
I’m not sure what you mean here by adding this to the “views” menu. There is bug 462578 ‘Quick search should have “Subject or contact (From, To, CC or BCC)”‘ which is moving forward, and will add a quick search term that has combined subject and this From/To/Cc/Bcc field. Is that what you mean?
What I meant was having From/To/Cc/Bcc (“AllAddresses”) as a quick search. 462578 combines that with subject.
I built TB from comm-1.9.1 (3.0 beta3pre) and see From/To/Cc/Bcc for searches and customized views, but not for quick searches.
But I guess we’ll all be using the gloda search soon – once we discover that we need to add it to the toolbar and figure out how to search in specific fields.
>But I guess we’ll all be using the gloda search soon.
Because it has never been clear to me what exactly will be the scope of gloda search, I have tended to put little effort myself into the conventional search, instead working with filters where it is clear that gloda will play little or no role in TB3. But there is that one change that will probably happen, making a “universal” field with all addresses and the subject in quick search. I think that is probably enough, and given limitations of additional space or complexity, it is unlikely that a separate field will be created without the subject.
As a followup, I just landed the patch that provides a combined Subject and All Addresses quick search term, called (after much discussion) “Subject, From, or Recipient”