In my last report, I was testing and updating the old Mozilla SOAP framework for use in communicating with Exchange Web Services (EWS). Using the framework in a Thunderbird 3.0 environment, I could talk to the BING search SOAP interface. It took a couple of more weeks to resolve issues associated with communicating with EWS. There were a number of features missing from the old Mozilla SOAP, most importantly:

• attributes in SOAP elements.
• arrays of choices with unlimited number of elements. The property bag of the previous interface can’t handle multiple items with the same name, so I did my own “property list” replacement for it.
• authenticated access to the SOAP service.

I’ve successfully sent a “Get Folder” call to an Exchange Server. The code to define the specific Exchange call looks like this:

let getFolder = new EwsGetFolder;
getFolder.invoke(responseListener);

function EwsGetFolder()
{
 // defaults
 this.FolderBaseShape = "Default";
 this.FolderIds = "inbox";
}

EwsGetFolder.prototype = new SoapBase(kMessages2007sp1Schema);

EwsGetFolder.prototype.invoke =
  function EwsGetFolder_invoke(aSOAPResponseListener)
{
 let FolderShapeList =
   objectPropertyList({BaseShape: this.FolderBaseShape});
 let FolderIdsList = objectPropertyList(
     {DistinguishedFolderId: objectPropertyList(
       {$attributes: objectPropertyList({Id: this.FolderIds}) })});
 let soapCall = Cc["@mozilla.org/xmlextras/soap/call;1"]
                  .createInstance(Ci.nsISOAPCall);
 let parameters = [];
 let FolderShapeType =
   this._schema.collection.getType(
     'FolderResponseShapeType',
     'http://schemas.microsoft.com/exchange/services/2006/types');
 let FolderIdsType =
   this._schema.collection.getType(
     'NonEmptyArrayOfBaseFolderIdsType',
     'http://schemas.microsoft.com/exchange/services/2006/types');
 parameters.push(
   new soapParameter(
     "FolderShape",
     FolderShapeList,
     FolderShapeType,
     "http://schemas.microsoft.com/exchange/services/2006/messages"));
 parameters.push(
   new soapParameter(
     'FolderIds',
     FolderIdsList,
     FolderIdsType,
     "http://schemas.microsoft.com/exchange/services/2006/messages"));
 soapCall.encode(
   Ci.nsISOAPMessage.VERSION_1_1,
   'GetFolder',
   this._schema.targetNamespace,
   0, null, // header blocks
   parameters.length, parameters);
 soapCall.transportURI = kAccount.url;
 soapCall.asyncInvoke(aSOAPResponseListener,
                      kAccount.user, kAccount.password);
}

So I can now successfully do SOAP communication to an Exchange Server (2007 SP1), using the standard Exchange schemas to encode the messages. There is still much needed to clean this up, but I’ve got enough working for now.

So now I turn my attention to the setup of a new account type in Thunderbird. This is the same issue that jcranmer is working on, but he is specifically trying to do everything in javascript, while I am content (for now) to work partly in C++. So I have to build the darn thing.

Ah, the Mozilla build system. Gotta love it. You have to know the difference between :, ::, :=, =~, (), {}, in a zillion different languages (perl, python, autoconf, automake, bash, make at least). When it works, it works like magic. But sometimes you wave your magic wand, say your magic incantation, and nothing happens. That’s what makes it so hard. You spend hours seeing if you can get the spell to work, but sometimes you just have to give up and do it the old fashioned way, knowing that there is probably some trivial change you could have done to make the magic work. But I guess I have no right to complain. Everyone said to NOT do it this way, but did I listen? I’ve been in build hell for a week.

But it turned out to be just build purgatory! Now that I am out, what wonders heaven holds! I can use a real debugger on my code, and not Venkman and dump()! The vast majority of the existing mailnews code is in C++, so I can see what local folders, IMAP, and newsgroups do, and very easily adapt it to my needs. I can subclass the existing implementations!

So as of today, I have a series of subclassed C++ files that contain what will be the future EWS-based objects – though I have not attempted yet to glue them to EWS, just simulations. I have new C++ files for the folder, message service, incoming server, and summary database – along with some basic unit tests. (This is very similar to the approach that the RSS implementation takes).

So what can I do? I can just display an account, its subfolders, and a message header (all simulated not real). It looks like this:

As you can see, I have not faced the issue yet of registering my protocols, so when I click on the message I get a complaint. I’ll be doing that next, along with trying to get the real information from EWS instead of this simulation.

OK, I know, you are not impressed – one simulated message, and that does not even work?  “Everything is hard to do in Thunderbird” is my defense, so cut me some slack, all right?

rkent

In the Thunderbird 3.0 / SeaMonkey 2.0 series, I snuck in a little hidden preference to allow modifications to the way that the TB bayes filter does it tokenization. The main point of this was to allow better transfer of information between SA and TB. I’d like to describe here how to use that feature.

SA’s decision process involves two key steps. First, they evaluate the message with zillions of rules, and tag the message with each rule that the message hits. Second, they have a method of combining all of those tags into a final junk score for the message, which is used to decide whether to tag the overall message as spam.

SA’s rules include many tests that are not done by TB (score one point for SA). Yet when they take all of their rules and combine them together, in most cases they are using broad measures of goodness and spaminess that apply to some concept of a universal email. In contrast, TB’s bayes algorithm will precisely tune the junk analysis to the particular set of emails that a user gets. So if you are in the Viagra business, you can tune your local TB filter to accept those. (score one point for TB).

Wouldn’t it be great if you could combine together the superior rule set of SA, with the superior decision making customization of TB? That is the point of the hidden preference.

SA communicates its message tags to TB in the form of a custom header, X-SPAM-STATUS. A sample value of that from a spam message that I got is:

X-Spam-Status: No, score=4.9 required=5.0 tests=HTML_IMAGE_ONLY_16,
HTML_IMAGE_RATIO_02,HTML_MESSAGE,HTML_SHORT_LINK_IMG_2,
MIME_HTML_ONLY, SPF_PASS shortcircuit=no autolearn=disabled version=3.2.5

This message was not marked as spam by SA, as it just barely missed the score=5.0 cutoff point, so arrives in TB as a suspicious but nevertheless good message. Yet there is lots of information about the tests that SA did in that header. TB should be able to take advantage of that information.

In the default TB configuration, the entire content of the X-Spam-Status header is treated as a single token. So TB will only respond to exact matches to that header. Yet those tests can appear in a variety of combinations, so that is not really taking advantage of the information effectively.

The answer provided in TB3 / SM2 is to provide a hidden preference that allows you to tell TB to break that particular header up into lots of separate tokens, and analyze each of those separately. That is documented in the code here. In the preference, you specify a particular header, and give a list of delimiters for that tokenization. So to process the x-spam-status header as individual tokens including both space and comma as delimiters, you add the preference:

mailnews.bayesian_spam_filter.tokenizeheader.x-spam-status

and assign it the string value ” ,\t\n\r\f”

(That can be hard to set with all of those special characters in it). That includes space, comma, tab, line feed, carriage return, and form feed as delimiters.

You really can’t do this to an existing training corpus without getting strange results. So you must reset the training data and retrain after you do that.

As a sample, the tokens used by TB to process an email with my example x-spam-status is:

This image uses the “Junk Analysis Detail” report from my JunQuilla extension.

Things to notice:

1. The tokens that TB is using to analyze this email are mostly these x-spam-status tokens, as this is a mostly-image spam message with little text for TB to analyze. So the use of the SA tokens greatly enhances the power of TB’s internal filter.
2. The final score for this, 84, ended up with the message marked as junk for me. I run a cutoff of 75, while the default cutoff is 90.
3. Many of the x-spam-status tokens are actually working against us here. For example, x-spam-status:required=5.0 has a score of 23, which moves the message toward the “good” status. Yet that token appears on every single email I receive, so should be neutral. What is going on here?

   This is the issue that I reported in a previous post “bad effects on junk training corpus from change” That is, recently I have been automatically training more messages as Good than Junk. “x-spam-status:required=5.0″ must not exist in my older emails for some reason, so this bias is appearing because 1) the token changed at some point in time, and 2) there is a mismatch in the number of junk and good emails being trained recently. I really need to figure out a solution to this. What I will probably do is to add features to FiltaQuilla to allow me to precisely match the training of junk and good to remove this bias.

I still need to do some tests of this to directly compare the performance of the spam filters with and without the tokenization to be sure this is really a good idea. That is quite tricky to do unfortunately.

In spite of these issues, I would recommend to new installations that you use this alternate tokenization when you are using a Spam Assassin front-end to Thunderbird or SeaMonkey.

====
Check tinderbox for current checkin status and burns

(cd pristine/src) [This is a directory containing a clone of comm-central from mozilla]
hg pull –u

(cd mozpush/src) [This is a directory containing a clone of my local pristine repository]

hg pull –u

(If there are problems, create a new mozpush directory, and do
hg clone /s/tb/pristine/src src)

hg import /f/tb/patch/name.patch
(above starts notepad, give bug & approvals and save)
hg outgoing -p
(check that the patch and comment are correct)
hg push

(cd pristine/src)
hg pull –rebase
(if changes, you may need hg update, or hg rebase -s revision -d revision)
hg outgoing -p

(if everything looks good)
hg push

Go to http://hg.mozilla.org/comm-central/shortlog and get the diff link for the push, something like:

http://hg.mozilla.org/comm-central/rev/324e7b178ad7

(or http://hg.mozilla.org/releases/comm-1.9.1/shortlog )

Go to bug, comment on checked-in attachment with above link.

Mark resolved/fixed, and add the appropriate pristine/src Target Milestone.

(when you checkin to comm-1.9.1, please set the status-thunderbird3.0 field to “.1-fixed” (or whatever the appropriate dot release is)

Follow tinderboxes.

I considered a variety of approaches to this. One extension “Asertiva Thunderbird Extension for Sugar” uses the IBM/Prototype js library for SOAP access. Others recommended that I consider one of the open source SOAP libraries, such as a python-based library, or Apache’s AXIS2 library. Or that I cooperate with the existing project to provide an open-source method of accessing Exchange server.

But I’m not sure of how “open source” I want all of this to be. From my perspective, “open source” as a charitable activity is not successful. We all need to eat, and so the revenue model needs to be clear if a project is going to be more than a phase of life I am going through at the moment. So I would rather keep my options open until I understand that better. Anyway, that’s a long discussion which is beyond the scope of the current posting, which is supposed to be a status update.

I am still in an education phase, trying to understand SOAP and the related protocols, and to figure out what exactly I gain from using any existing library versus doing things more directly from the raw XML. So as both a trial and education step (and against the recommendation of others I might add) I’ve tried to update the old Gecko webservices extension to work in current Gecko 1.9.2, and to work with some current Microsoft SOAP protocols.

Rather than start with EWS, I started with the simpler BING search calls. I used existing Microsoft demos in C#, and could capture the communications using Wireshark to see what I was supposed to be sending and receiving.

Updating the abandoned webservices extension to Gecko 1.9.2

After testing webservices some under an old Firefox 2 build, I upgraded portions of the code to work under a current comm-central trunk build, using Gecko 1.9.2. My requirements are somewhat simpler than the original extension:

1. Most importantly, my target is chrome-based extensions rather than browser code, so a lot of the security issues that FF folks worried about were not important to me.
2. I had no interest also in allowing native JS creation of components, as I could rely on using .createInstance calls instead.
3. At least initially, I am not supporting the reading of wsdl files, nor the associated automatic creation of proxy calls and interfaces. Instead, I read in a schema file, and generate my own code for each method. My understanding is that Microsoft, in their EWS libraries, also does not actually automatically generate method calls from a WSDL-based proxy, but follows this same approach of starting with the schema files.

That allowed me to avoid about half of the existing code, and focus on the /schema and /soap directories of the webservices extension.

Looking at my hg logs, it took one week of development time, and 17 patches to get to the point where I could write a unit test under Gecko 1.9.2, and confirm that I could create webservices components using a unit test. (I’m doing things in a test-driven development fashion, writing XPCSHELL unit tests to try out different features of webservices.)

Learning and testing Gecko webservices

Although there was a little old documentation around on Gecko webservices, ultimately I just needed to learn things the old fashioned way, reading the code and its interfaces and experimenting to see what worked. I took little baby steps, starting first with an XML schema primer and eventually working my way toward duplicating the functionality in some Microsoft Bing search demo C# code. This phase, starting from the first demonstration of loading of SOAP components under Gecko 1.9.2 through testing of encoding and decoding of a Bing search message, took about 2 weeks of coding and 23 patches, with the creation of 22 unit tests in the process.

The main issue that I had to deal with in the existing SOAP code is that it did not support maxOccurs>1 schema types, such as this one from the Bing schema:

<xsd:complexType name="ArrayOfNewsArticle">
  <xsd:sequence>
    <xsd:element minOccurs="0" maxOccurs="unbounded"
                 name="NewsArticle" type="tns:NewsArticle" />
  </xsd:sequence>
</xsd:complexType>

I solved this by using an nsIArray to hold multiple values of the same element.

Sample Code

So now I can do a Bing search in an XPCSHELL test, and decode and test the results. I want to show some of the js that I use, to give some idea of the complexity (or lack thereof) of using this.

I create a class BingSearch, then encode some values to setup the search. A basic call for a search looks like this:

function run_test()
{
 do_test_pending();
 getNews = new BingSearch();
 getNews.Query = "obama";
 getNews.Options = ["EnableHighlighting"];
 getNews.Sources = ["News"];
 getNews.News = {Offset: 0, SortBy: "Relevance", Count: 1};
 getNews.invoke(getNewsListener);
}

The “BingSearch” object is presumably what a sophisticated library would create automatically from the WSDL file. Instead, I create it by hand. Here’s my partial implementation, that does not support all of the allowed inputs to a Bing search, but works for my tests:

function BingSearch()
{
 // defaults
 this.Version = "2.0";
 this.AppId = "<you get this from Microsoft for your application>";
 this.Market = "en-us";
}

BingSearch.prototype = new BingBase();

BingSearch.prototype.invoke = function BingSearch_invoke(aSOAPResponseListener)
{
 let parametersBag = objectPropertyBag({
   Version: this.Version,
   Market: this.Market,
   Query: this.Query,
   AppId: this.AppId,
   Options: arrayPropertyBag("SearchOption", this.Options),
   Sources: arrayPropertyBag("SourceType", this.Sources),
   News: objectPropertyBag(this.News)
 });

 // soap message component
 let soapCall = Cc["@mozilla.org/xmlextras/soap/call;1"]
                  .createInstance(Ci.nsISOAPCall);
 let parameters = [];
 parameters.push(new soapParameter("parameters",
                   parametersBag,
                   this._schema.getTypeByName('SearchRequest')));
 soapCall.encode(Ci.nsISOAPMessage.VERSION_1_2,
                 'SearchRequest',
                 this._schema.targetNamespace,
                 0, null, // header blocks
                 parameters.length, parameters);
 soapCall.transportURI = "http://api.bing.net:80/soap.asmx";
 soapCall.encoding = this._encoding;
 soapCall.asyncInvoke(aSOAPResponseListener);
}

const kSchema = {
 file: 'data/bing20.xsd',
 schemaNamespace: 'http://www.w3.org/2001/XMLSchema',
 targetNamespace: 'http://schemas.microsoft.com/LiveSearch/2008/03/Search'
 }

function BingBase()
{
 if (typeof this._schema == "undefined")
 {
   this._schema = getSchema(kSchema);
   // use the 2001 SOAP encoder
   this._encoding = Cc["@mozilla.org/xmlextras/soap/encoding;1"]
                      .createInstance(Ci.nsISOAPEncoding);
   this._encoding.getAssociatedEncoding("http://www.w3.org/2001/09/soap-encoding", true);
   this._encoding.schemaCollection = this._schema.collection;
 }
}

So far, the complexity of this does not seem unmanageable to me. I’ve only shown the endoding step. Decoding the response consists of a creating a call-specific translator similar to the “BingSearch.prototype.invoke” function above, which relies on the webservices soap library decode method. All of the other functions I’ve created (such as arrayPropertyBag) are not at all specific to the nature of the SOAP interface being used. I am not seeing the need to process the WSDL file automatically and generate proxy functions.

I’m not yet convinced that this resurrection of the old webservices library is the right approach, but I am not seeing any obstacles to using it either. I can generate and decode soap calls fairly efficiently, debug issues that arise, plus I have code that will integrate fairly easily with either javascript or C++ code in a Gecko chrome environment.

Next steps

I’m trying to decide on the next step toward moving this forward. I’m leaning toward attempting a specific EWS application, such as read-only access to an Exchange calendar as a Lightning extension. Another option might be to add some core mailnews hooks to allow me to create either message accounts or addressbooks using an extension – though I’m hoping jcranmer will beat me to it.

The issue here is that I see lots of good work going on by people who are mostly involved in QA, such as wsmwk, WADA, and Ludo, but I as a developer don’t really know how to make the best use of that work.

I assume there is supposed to be a waterfall here, from (bug reporter)->(QA)->(developer)->(code reviewer)->(bug landing). I understand all of the steps of the process except this (QA)->(developer) handoff. I would be curious to hear from people involved in QA about what they view the main outcome of their work is supposed to be, as viewed by a developer.

Here’s what my understanding is of the current process. After bugs are submitted, QA has three main responsibilities: 1) get the bug in the correct component, 2) move the status to NEW or one of the inactive states (DUPE, INVALID, etc.) 3) clarify the bug information to get clear steps to reproduce.

Is this accurate?

Let’s look to see how that is working by looking at my recent work. In the last three months, I fixed eight bugs (that’s a little off my desired pace, but we were frozen a lot of that time). What brought those bugs to my attention?

(3) bugs I reported myself, either due to issues I observed or as a result of following support forums.

(3) bugs are fixes of crashes that wsmwk reported from crash stats

(2) bugs were filed earlier by others. If I recall correctly, both of those bugs were items that I noticed first in support forums, then located the bug in Bugzilla and fixed it.

In no cases did the standard QA waterfall process play a significant role in bringing a bug to my attention. And that concerns me, because I see some very competent and dedicated people working hard on QA, but I don’t seem to be making effective use of that work.

Am I somehow not following the process that I am supposed to be following, or is that process flawed? In theory I am probably in a better position than most people here, because I primarily track items that appear in the mailnews core/filters component.

I wish there was a clear way for the QA people to bring a limited number of bugs to my attention that are 1) important, 2) clearly defined and reproducible, and 3) likely to be fairly easy to fix.

The mailnews core/filters category currently has 326 NEW/ASSIGNED/REOPENED bugs in it. I could probably fix a few per month that were brought to my attention. A reasonable expectation might be that 10% of those are addressed in the next year. How are the QA folks supposed to influence the selection of which 32 bugs actually get my attention?

(posted to http://mesquilla.com and m.d.a.thunderbird, followup to m.d.a.thunderbird please.)

rkent

What I am seeing is that the common, unchanging words in the Received: header, like “received:from” and “received:(exim”, are persistently occurring with a moderate “good” score, such as 36, even after training junk messages with those headers. There are a lot of these little meaningless tokens per message though, and they are dragging down the junk score of junk messages into the Uncertain category.

I think what is happening is this, and it could be caused by any change in your common environment. I started adding new tokens such as “received:from”, without restarting training. Because I also started training temporarily many more good messages than junk, these new tokens are showing up disproportionately as good.

Suppose, for example, I start with 1000 good messages and 1000 junk messages in my corpus, then suddenly add a new token to all incoming emails. Then I train 100 good messages with that new token, and 10 junk messages. The spam corpus will claim that the new token appears in 10% of good emails, but only 1% of junk emails, so the presence of that token is a marker that the message is more likely to be good. Which is not true, since now ALL emails have that new token!

I suppose that one defense against this would be to make sure that the proportion of good and junk emails trained always stays about the same. That is not easy to do, however.

I don’t have any illusions that this has a significant chance of being implemented, and I’m not even sure it’s a good idea myself. But I ask you to suspend disbelief for a minute, and imagine a change to the development culture and process.

An email client is different from a web client in many ways, but one significant way is that there is no real need for a fat uniform core product that developers can target (such as web developers for FireFox). So we are free to allow wide changes in our product configuration that would not make sense for FireFox. There is really no fundamental need for Thunderbird to be presented as a single, fat, feature-laden client.

Instead, ship Thunderbird as a minimal base with a collection of extensions. The extensions could be in a variety of statuses. At one status extreme, “Core” extensions would be enabled by default, would be fully localized, and their updates would be shipped with updates to the core product, rather than through AMO. Many existing core features would be converted into “Core” extensions that could be disabled if desired (for example bayes junk processing, or gloda.) At the other extreme, “Pilot” extensions would be shipped with the core product in a disabled state, would be updated by AMO, and not fully localized. There would also be “Standard” extensions that are shipped with the product, not maintained through AMO, but would not be enabled by default. Lightning might be one example of this. FiltaQuilla or JunQuilla could easily get added to this category in the future, or popular extensions like ThunderBrowse.

So why would you do such a crazy thing? For several reasons.

First, you would have a path to add features to the program that is not as generally disruptive as has been, for example, gloda or the new message header. By not using a new extension, an existing user would not see changes to their workflow that they did not want or appreciate. Also, new features need not delay the release of new versions of Thunderbird, as “Pilot” status extensions could be updated through AMO.

Second, new complex features like gloda, even though they are developed by the core team (well mostly asuth) are in a state of rapid flux, and would really benefit from allowing updates more frequently than even the accelerated release process will allow.

Third, you provide a natural path for outside developers to add contribution to the product without having to completely submerge themselves in the Mozilla culture, or give up complete control of their creation.

Fourth, this really recognizes that the use of an email client is highly personal. Basic users could be presented a basic email client. Advanced users could easily add advanced features. (Existing AMO-based extensions are also good for this, but the quality is not uniform, and they frequently are not kept up to date. And the standard product is still very fat with lots of features that are unneeded by most users.)

Fifth, this would solve the serious issue with Thunderbird of how hard it is for the average user to install addons (because the most popular and important addons would be shipped with the product).

There’s another dimension to this, and that is the developer’s relationship with his or her extension. I know that I feel a responsibility for my extensions that is beyond the responsibility I feel for any core code. You can see that in the documentation that I provide, and my reliability in responding to issues. I think that many other extension developers are like that as well. I’m guessing that they would be delighted to see a higher level of promotion of their work, without the need to cede complete control that incorporation in core might involve.

I suppose I could write a book on what this might look like, but for now let me leave it here.

rkent

1) Spams caught by server-side SpamAssassin: 109

2) Spams caught by local bayes filter after passing SpamAssassin: 49

3) Spam marked by me that got through both filters: 2 (junkpercent scores were 63 and 66)

Total Spam: 160

For the server-side SpamAssassin filter, my spam detection limit is 5.0 This is the stock SpamAssassin filter supplied by default to all accounts by a large, inexpensive web hosting provider (hostmonster).

For my local bayes filter, my spam detection limit is set at 75.

I *never* have emails falsely marked as spam. I train spam reliably using the Uncertain folders in my JunQuilla addon. I have a limit of 300,000 tokens (with a current count of 118,690), 2097 good messages trained, 3748 junk messages trained.

Oh, and I use 2 customizations to tokenization using new hidden preferences available in TB3. First, I tokenize into words the Received header (it is disabled by default), plus I tokenize into words SpamAssassin’s X-SPAM-STATUS header (which is accepted as a single token by default, that is not broken into individual words.) I don’t believe these are very important, however, but I do think that they help a little.

Note to self: blog about customized tokenization settings in TB3, and try to do some analysis.

So far, I have added the following new search terms:

BCC – locate items in the BCC field

Subject Regex – search the subject using a javascript regular expression

Header Regex – search any specific header using a javascript regular expression

Javascript – load javascript in a text field, and program your own search given an nsIMsgDBHdr object

Tag of Thread Head – match a tag in the head of a message’s thread

Tag of Thread Messages – match a tag near the message in its thread (within +/- 10 messages by default)

Address in Thread – match an address near the message in its thread (within +/- 10 messages by default)

This stuff can be useful outside of filters, in fact I am mostly using them personally to define virtual folders. So I’ll probably move them to a new extension, and try to get FiltaQuilla out the door finally.

rkent

(All references to extensions in this posting refer to the 1.0.0 versions, which as of this writing have not been posted to AMO yet. But they should be available in a few weeks.)

Implemented UI

Briefly, inherited properties are a property that can be defined globally, at the server, or at the folder, and its characteristics will be propagated to child objects. This make it easy to specify precisely how the property is applied.

As an example, I have recently implemented a feature “Index in Global Database” in GlodaQuilla which can be used to selectively suppress certain accounts or folders from being accessed by the global database indexer. In the account manager, where indexing can be disabled for an entire account, the UI looks like this:

Each inherited property has default values which are typically set by the base code. In the case of the gloda database indexer, everything but newgroups are indexed by default. Initially each inherited property is set to just use the standard default processing, but if I clear the “default” checkbox, then I can turn off gloda indexing for this account.

If I do that, then go to a first-level folder in the account, I see the following under folder properties:

At the folder level, because I disabled global indexing on the account, it is now shown as disabled on the folder. I could clear the inherit box and selectively enable it on just this folder and its children if I wanted.

This particular UI merges naturally with the existing methods of setting properties in mailnews, but I’m not sure it is optimum for an inherited property. The inherited nature could be more clearly shown, and a particular feature more quickly configured, if I showed a tree of accounts and folders, with checkboxes next to each account to enable or inherit the feature. Maybe in a future version.

Implemented properties

Here are some of the implementations of inherited properties that exist in my extensions:

1. (GlodaQuilla) Index in Global Database – suppresses the running of the global database indexer
2. (FiltaQuilla) Apply Filters to Folder – for Imap folders, allow incoming filters to run on that folder
3. (JunQuilla) Analyze Junk – allow junk processing to be turned on or off. This also allows junk processing to run on RSS or news folders.
4. (TaQuilla) Analyze particular automatic tags.

Issue: Existing mechanisms

Ideally, the inherited property would be the one and only way to manage a program feature. But for existing features, the existing mechanisms remain, which can lead to possible confusion. For example, with JunQuilla’s “Analyze Junk” property, there is existing UI to enable junk processing at the account level. Here the inherited property will always override the default mechanism (but that is mostly because I implemented it in core that way, and I have a little influence on how junk processing is handled in core.) For GlodaQuilla’s “Index in Global Database” the behaviour is different. Existing UI will only allow this to be enabled or disabled globally, and the inherited property does not override this. The inherited property code uses the default server preference as a global enable/disable for a property, so if gloda used that same mechanism instead of an independent preference, this issue would go away. I guess I could say the same thing about junk processing as well.

For FiltaQuilla’s “Apply Filters to Folder”, there is a subtle issue in the inherited nature. I did not implement in core the ability of the inherited property to override the existing default as applied to the Inbox, so incoming filters always run on the inbox. That creates 2 ui issues. First, although I show at the account level the “Apply Filters” option, it does not actually suppress application to the inbox as one would expect. Second, I currently do not show a folder property for “Apply Filters to Folder” for the IMAP inbox since it would not make sense there, so that also means there is no way to enable processing of filters for the children of the inbox. Maybe I should call this feature instead “Apply Filters to non-Inbox Folders” to solve this, or change the core code so that the feature also applied to the Inbox.

Issue: My RDF-inspired property for junk management

Looking ahead to a world where a number of extensions might try to define bayes filter traits, in code I recommended that properties used to manage junk processing use an RDF-inspired globally unique identifier. Then I followed my own advice and defined the identifier that controls junk processing on a folder as: “dobayes.mailnews@mozilla.org#junk” Unfortunately, the existing account manager code does not allow periods in property names, which means I could not use the account manager to manage this. I’ve filed bug 525024 on this issue, and perhaps that can be incorporated after TB 3.0 / SM 2.0.

Issue: Missing inheritance levels

I’ve heard others comment that often they want to set a property on a particular class of folders, say on all Trash folders, or all Sent folders. I’ve considered implementing another level in the inherited properties feature, that would be a folder type. So you would then set a property that would be inherited by any folder of a particular type, and its children – and of course also overridden by the local folder property.

Issue: UI for global property

All of the inherited properties could also be enabled globally using the “mail.server.default.<property>” preference, but I did not give any UI for that in my extensions. I thought that would be too confusing for the user to show those preferences, which would be very similar to existing mechanisms. This is not an issue for properties that use the preference system for server-level issues, but none of the existing server-level preferences are also inherited properties. Perhaps we could move that direction in the future.