To ease my transition back into ews after getting my brain all wrapped around filters for awhile, I thought I would tackle something that is mostly internal to the mailnews code. So I took on the task of getting a basic custom Account Manager interface up for the exchange web services account type.
The account manager interface was apparently written by an ancient advanced intelligence (oops I mean here), that used mysterious technologies that are no longer understood by humankind. Renowned cybernetic archaeologist Joshua Cranmer has tried to unravel some of its mysteries, but that provided me more of a hint of where to look for issues than a roadmap of what I needed to really do. So I was forced into my last resort – reading the code and trying to figure it out. There were some surprises.
First, a brief description of terms. When you first create a new account type (unless you are using the new email interface for Thunderbird 3.0), you enter into the account wizard. That sets up up all of the initial values. After that, if you try to edit those you end up in the account manager. They are closely-related, but distinct, interfaces.
First, how do you add a new account type at all to these interfaces? It turns out that kill-rdf missed a spot, and it is still possible to use rdf to add a new hook into the account wizard. Extensions can have a subdirectory “isp”, and into that subdirectory you can inject an rdf file that contains some information about your account manager hook. The rss implementation gives some hints of how that is done.
Once you add the rdf file, your new account type shows up in the account setup under “add other account …”:
The rdf file that you add contains all sorts of information about the account that you are trying to setup. Through some magic, the rdf items get linked directly (at least in theory) to default values for the main account objects.
But unfortunately, extensibility of the account manager has never really been carefully supported, so things just don’t work out quite the way you would like. Some examples:
- My account type represents a new protocol type, “ews” instead of “imap” or “pop3″. But at one point the server page looks into the standard file “messenger.properties” for strings like “serverType-imap=IMAP Mail Server” which of course do not exist for “serverType-ews”.
- The pages that show in the account wizard are partly hard wired for the existing account types.
- Although the account manager has some attempts at extensibility, most of its options are setup as a list of account types to exclude when showing certain screens. This is maddening if you are adding a new account type, because since your account type previously did not exist, of course it does not show up in all of the exclusion lists.
So this intricately designed infrastructure that the ancients designed is basically unusable at present to add new account types, as far as I can tell. For each little option that you want to add, you need to trace for hours through very abstract code to figure out whether it will work, or if there is some hidden gotcha. Ultimately, it really is easier just to throw it out and start over again.
Fortunately, through the wonders of javascript function replacements, I was able to create enough hooks to get my new account added, without requiring a patch to the core code. Here is a rough outline of the steps that I took:
- First, the account wizard seems to use the existence of identities to decide if your have a mail account type, and if you do it forces you to use the standard mail account wizard types. I managed to override this in javascript.
- After you do that, through some bizarre quirk the rdf property “NC:wizardShortName” is actually a list of the wizard pages that you want to appear for custom account types. So I can add a line to the rdf file:
<NC:wizardShortName>identitypage,ewsServerPage</NC:wizardShortName>
and that will allow me to specify precisely which pages are shown in the wizard. “ewsServerPage” is a custom page that I add though an overlay.
- In the account manager, there is no easy way to control which subpages appear in the account tree for a custom account type. Once again I can use a javascript function hook to manually adjust the available pages. I wanted to remove the default server page (which asks if I want pop3 or imap), and add my own page in its place. I also wanted to be able to remove pages I did not want. That lets me control the list precisely, so that I now get this account tree:
I suppose dmose would like me to organize all of this into a sample extension that others could use. One of the reasons that it is hard to get motivated to do this is that there is no real plan for the future of any of this, yet I hear rumors that a lot of this is scheduled for demolition (rdf, account wizard, even binary extensions that my entire application relies on.) Plus I’m not sure any sane person would want to do the kludges that I am doing to get this to work. Is there even any real demand for custom account types in the mailnews code, beyond what I am doing?
Anyway, the good news for the Exchange Web Services integration project is that I now have basic hooks into the account manager interface, so that I can easily add custom changes to the account manager setup to support Exchange Server. The most basic information (user name and server address) can be entered through the account manager interface, and used to access the correct Exchange server.
serverType-imap=IMAP Mail Server
One of the reasons Mozilla products are used by many is the extensibility! I would think that this SHOULD certainly include being able to add new account types. The Lightning folks have done a great job of making it (relatively) easy to add a new calendar type. I think it would be great if it were as easy to add a new account type to Thunderbird.
Could you give some examples of what some of those other account types might be?
As I understand it, one of the motivations for changes to the folder tree (including such things as removing the ability to show folder size and message count in another column) was to improve its ability to handle a wide variety of potential types of items. But so far that has not happened. I don’t really know if that is just because it is difficult to easily do (as my post and jcranmer’s series shows) or if the demand just does not exist.
Well, first off, why are you perpetuating the ispdata hack stuff? That is the part most likely to be gutted in any future account wizard changes; RSS and Movemail use it mostly because I haven’t taken the time to kick them off of it yet.
Making new account types, as I’ve argued in the past, is one of the things that should be easy to do in Thunderbird. It’s not really true that no one has tried it before: the Webmail extension does try to add new account types in TB (via implementing its own POP/IMAP servers!). I think the lack of extensions is symptomatic of the sheer difficulty of making a new account type–in order to get it to work, you either need to reimplement large portions of core code or you need to distribute binary components. Not to mention all the grief the core code gives you in trying to circumvent the closed system.
One thing that I’ve come to think in recent months is that to convince extension authors to use a feature, you need to have a usable prototype extension. The folder pane was designed to be extensible, but as I recently discovered, it was broken: no one noticed it because no one tried to make an extension!
In short, I think that if a comprehensive tutorial on making new account types were presented to potential extension authors, enough to make it relatively easy to do the common backend tasks, the uptake would eventually come. Even if this is not true, we would have a wonderfully documented subset of code.
Johsua: “Well, first off, why are you perpetuating the ispdata hack stuff?” Because that is the currently present mechanism for adding a new account type? And it works to some extent, except as you say without any real extensions using it the deficiences are unknown.
“One thing that I’ve come to think in recent months is that to convince extension authors to use a feature, you need to have a usable prototype extension.” Preaching to the choir here (JunQuilla and FiltaQuilla are really demonstrations of core features). But there are serious obstacles to that in practice. Two issues: first, I’ve argued unsuccessfully that core developers should actually package new features as extensions when possible, both as a trial period for the feature and as a way to make sure that they eat the dog food of using extensions as a way to add features. But I’ve not gotten any traction with that. Second, and this is fairly subtle, I think that the world is divided into “core developers” and “extension writers” in many people’s minds – with “extension writers” a different, probably lower, class of people. Put too much effort into extensions and you move into that lower class.
“I think that if a comprehensive tutorial on making new account types were presented … ” As I said in my post, it is hard to get motivated to do that when there is no agreed on direction for the future of the core code. It would be very easy to document in detail features that are only valid for 3.2a1pre. I’ve never figured out how to get any agreement on roadmaps for the future in mailnews development areas that multiple people are likely to have opinions on.
Personally I’ve been waiting for jcranmer’s patches to land to start work on using Twitter as an account type in Thunderbird.
It would also allow an extension such as WebMail to not run its own server, since the account just request over HTTP protocols and be parsed right away, instead of converting an HTTP protocol to POP3.
Another example is OpenChange which has some documentation about creating a MAPI protocol as an extension for Tb: http://wiki.openchange.org/index.php/Thunderbird_Plugin and http://wiki.openchange.org/index.php/SoC/Ideas#Thunderbird_Integration (among other places).
I don’t know if there is other active interest for other accounts, but I’m confident people would make other ones if it was easy enough. There’s a few (mostly unused) protocols Tb doesn’t implement, but other examples besides that? Perhaps Facebook messages (or other sites that have custom messaging system), web forums (per jcranmer’s example extension), Google Wave, maybe integration of VoIP in some way and I’m sure there are others; as well as things that don’t necessarily come in a “message format” but could be forced into one (comments on a Wiki talk page, etc.)
Patrick: OK I’m convinced, there is a demand for other account types.