Mail::Box 2, Second Progress Report

software for e-mail handling in Perl

15 July 2003, Mark Overmeer, MARKOV Solutions.

In the last two months

A period of two months is rather short when a large conference and some days off are included. In my case, these where joined into one extended stay in Florida to visit YAPC::NA.

Participation of YAPC::NA

At arrival, there was only one slot of 90 minutes allocate for my talks, which I used to present the Mail::Box distribution to the American audience.

During the three days of the conference, some other small spots fell open of which I claimed three more talks: 20 minutes for OODoc (Object Oriented Documentation) and twice 5 minutes for Hash::Case and Object::Realize::Later, other modules that I wrote.

• Mail::Box (90 minutes tutorial)

  The presented Mail::Box talk was based on the talks given at the previous YAPC::EU and German Perl Workshop, but adapted to include more of the new features and some problem areas. As far as I can tell, all of the about 30 spectators saw this presentation for the first time.

  The presentation will continue to develop, showing more examples and demonstrating some powerful features, which will result in a close to new presentation for YAPC::EU (compared to that of YAPC::EU last year). Those slides will be published on the Net.

Mail::Box documentation

After the transition of the documentation system to use OODoc was completed in the previous period, the benefits of the new set-up became visible. Without the fear to lose overview on the documentation, the manual can grow much larger now. As main feature, larger parts of the documentation were reorganized, which can been seen at This page will grow into a large doc/faq/example collection.

Small developments

The new Outlook DBX and message rebuild functionalities didn't cause any problems for users, but they have already found a few users.

Some improvements where made to help people with large mailboxes: it is now very simple to remove unimportant header fields, sets of "resent group" headers, and mailing list related headers from memory and folders.

A major memory leak was fixed with a contributed module [written by Todd Richmond] which replaced the use of the leaking external module. This may save up to 40% of memory! Mail::Box is still demanding considerable amounts of memory, which is mainly caused by the way Perl is organized internally.

An other way to save memory is the new message "destruct" function, which frees-up the message from memory and at the same time catches the situations where the message data is accidentally being used after destruction. This is a clean hack, and only useful for people with large folders.

Major developments

Progress has been made in the list of tasks in the Mail::Box phase 2 project plan, however not all are already visible to the users. The main reason behind this is that visiting and preparing three conferences within 5 month consumes that much time that development can be started but not be finished: fixing the last bugs, creating enough tests, and writing the general documentation of each new feature takes some extra effort and concentration that in cannot be combined.

Two major parts of the plan are therefore still in my development tree as alpha code: unicode headers which parse all the features of e-mail addresses and mailing list header support are close to ready. The IMAP implementation has started, as well as the destription how to use Mail::Box with databases.

• Unicode headers for addresses

  In parallel to Mail::Box, I maintain a distribution named User::Identity, which stores user related data. On this moment, that software is stepwise integrated with Mail::Box as alternative for some older external modules.

  An example shows what you can do with it (even without detailed knowledge of Perl, this should be understandable):

        use User::Identity;
        my $patrik = User::Identity->new
         ( name      => 'patrik'
         , full_name => "Patrik_Fältström"
         , charset   => "ISO-8859-1"
         );
        $patrik->add(email => "him@home.net");
  
        my $b = $msg->build(To => $patrik);
  
        $b->get('To')->print;
  

  This prints

        To: =?ISO-8859-1?Q?Patrik_F=E4ltstr=F6m?= >him@home.net>
  

  As you can see from the example, the user does not need to take care about character set encoding in message headers anymore: only once tells the generally prefered character set. This is a major gain. The example already works for real.

• Mailing list headers

  The Mail::ListDetector module, which is available in the Perl archive CPAN, can be used with Mail::Box although it was never designed that way: the simple interface it requires is also available in Mail::Box.

  However, the module is not powerful enough for my plans, and it seems that the developer of the module lost interest in progress. Therefor, I have decided to reimplement this code, using the knowledge which is stored in it.

  The software is close the completion. It can detect about 10 different mailinglist implementation and provides a uniform interface to take the data which the lists put in the headers of the messages.

• Mail::Box and IMAP

  After discussing an IMAP implementation with a few people on the conferences and at the mailinglist, I made the choice to use the Mail::IMAPClient distribution to base the IMAP integration on. The available documentation (RFCs and manuals) are read, and the developments are currently on the user authentication process. IMAP connection is only in the early stages.

  [Yuval Kojman] published a design for an IMAP/POP server based on Mail::Box at his site.

• DBI

  About five people joined in a debate on linking Mail::Box to databases. On the moment I try to get a view on two or three different designs in an attempt to figure-out how people organize the database tables.

Releases

By reason of the "conference season" I could not continue a two weeks release cycle. The number of changes per release went up, and the number of releases went down.

version 2.044: to be released next week

version 2.043: Thu Jul 10 10:18:51 CEST 2003

version 2.042: Tue May 20 14:32:05 CEST 2003

Mailinglist

The "conference season" and the "holiday season" are probable causes for a decay in the number of messages on the Mail::Box mailinglist:

July till 14th	: 20
June	: 22
May	: 69

My personal trace contains 110 messages directly related to Mail::Box and many dozens for related modules since May 15, the date of the revious report.

Plans for the near future

In the last part of this project, I plan to finish the remaining subjects of the plan. During Summer, most other activities are reduced to a very low level, so there will probably be enough time to finish the targets set.

An offer by LinuxJournal for a publication about Mail::Box will be studied further. Not to forget the YAPC::EU conference and its aftermath.