My kind of application needs a database

January 24, 2009 – 10:33 pm

The last couple of weeks, I’ve been focused on killing some backend database bugs in Thunderbird that seemed to have popped up recently – including bugs 471682, 472446, and 471130. I’ve still got more database-related issues that I want to see solved before TB3. I am determined to make sure that the message database in Mozilla Mailnews is robust and reliable – which has not always been a priority in the past.

Maybe this would be a good time to say why I care.

There’s a class of applications that I want to pursue, that are probably best described as variations on the personal information manager theme. Right now I see two possible directions, which are distinct in my mind but have a lot of possible overlap. I come at this as a refugee from Ecco, as someone who has never quite figured out how to get organized in life once the world went toward internet-based communication (primarily email), and tools like Ecco did not manage to keep up. There was a time when I was an organized person, with Ecco at the hub. I’d like to get there again.

My first possible direction is the easiest to explain – it’s a contact manager similar to ACT, Goldmine, or SugarCRM. The main goal would be to integrate all of the many ways that we now interact with each other into a centralized application, where you could manage your activities to develop and maintain personal relationships in a deliberate manner. I’d like to be able to see all of the public communications that someone makes – blogs, chat, newsgroups, web-based systems like bugmail etc. – along with any personal communications that we have, to more easily develop and nurture a wide net of relationships. I’ve been calling this application by the codename Stalker.

The second direction is more vague, but is focused on managing personal visions, goals, projects, and tasks. I’ve been inspired in the past by both the high-level life views presented by for example Steve Covey’s books, as well as the more practical approach in the Getting Things Done direction. I can envision an application that allows you to manage your life’s direction at all levels, from the high-level view of personal vision and goals, drilling down into projects, and ultimately resulting in the nitty-gritty of the minute-by-minute work required to actually move things forward.

There are existing applications that do both of these things. But my experience has been that they ultimately fail because of the problem of information fragmentation. There are just too many little buckets where we keep information about our tasks and contacts. When you have to start spending lots of time to keep all of these information formats aligned with each other, then the focus breaks down, and the systems become impossible to maintain.

That’s where the Mozilla platform comes in. So much of our life’s activities are now focused on internet-based activities. Any application that hopes to manage these needs to be, first and foremost, an outstanding client for internet-based interactions. So why not start with the email and web application, and add on to it the personal information management piece? As the Mozilla platform continues to migrate toward Javascript as the primary programming language, then virtually everything that you would want from a personal information management application can be done as an extension to the core Mozilla applications. You get an excellent web browser and email client as the base – and then add the information management on top.

And that’s where the database comes in. Personal information management will rely on robust database technology. This will be particularly true when you take the next step, and make the project and contact information shareable by a group – which is where you need to go particularly if you hope to ever earn income from it. I want to have a seat at the table when the direction of the database on the platform is discussed and developed, because it is critical to my vision of where I would like to see some new applications.

2 Responses to “My kind of application needs a database”

  1. Kris Walker says:

    I’d be really interested in hearing some ideas on a datastore that overcomes data fragmentation.

    First, how do we combine data from different domains, into one domain? And then how to we modify it, and then tease it back apart to send it off to the remote service it came from (assuming that many are push/pull services, and not just consuming services). I think it’s worth taking a look at web hooks for some ideas.

    Second, I’ve sea-sawed back and forth on the implementation between a description model, like RDF, data entities (like OO classes or constructors), and an object-relational mapping scheme. I really like RDF, but I’ve never seen it implemented in a reasonable way. On the other hand, relational databases seem to be overstaying their welcome in the age of fragmentation.

    However it is implemented, I think the data models must be extensible and dynamic for this to work.

  2. rkent says:

    RDF is certainly the technology that tries to tie together multiple web datastores. But is has become “uncool” for a variety of reasons. I’ve seriously looked though at porting the Redland RDF library into the Mozilla environment to give me a more robust software subsystem to play with that.

    But I picture the war against data fragmentation to be one of integrating many sources into a common framework, such as the Mozilla mailnews (Thunderbird) platform. As for data storage, though I haven’t tried it yet, generally I picture using a local SQL-based engine (such as the SQLITE-based gloda in Thunderbird) to provide the integration capability, with the canonical storages all being simple attribute/value pairs (or triples) stored in a variety of different formats and locations.

    But I have not actually faced the real issues yet.

Leave a Reply