This weekend, I installed Scalix on a client's mail server. I had installed Scalix before, but it was a clean install with no old e-mail to migrate. This time, however, I had a lot of old e-mail to migrate.

Most of the users used Eudora for Windows. Others used the openwebmail web mail client. As a result, I had e-mail from three different sources to populate with:

  • Eudora locally-saved mailboxes (an almost mbox-format)
  • IMAP folders on the server, in mbox format
  • Standard mbox mailspool boxes (inboxes).

In preparation, I had read on a page on the Scalix site there were scripts to import mbox files into Scalix, but as it turned out, these scripts were for restoring a proprietary-to-Scalix backup mailbox format. Bleh.

The next best solution was to use an e-mail client that could import mbox mailboxes and move the messages to an IMAP folder. Thunderbird only imports a few formats on Windows and only imports Netscape Communicator on Linux. Bleh.

So, I tried kmail. It worked beautifully. At one point, I had three instances of kmail running in three different VNC sessions on the server funneling messages into Scalix. The only problem with kmail is that it required constant babysitting as I moved from mailbox to mailbox and user to user.

So, while kmail was cranking along on a large mailbox, I looked at CPAN to see what modules were available for dealing with IMAP and mbox file formats. It didn't take long before I had a script that would upload a user's messages. That worked great for the mbox mailspool files- there was one file per user.

Next were the IMAP/webmail mailboxes. I had copied them all into separate folders for each user, so to automate it, I needed a script that could go into each user folder and upload the messages from each of the mailboxes it found there. However, there were some mailboxes I didn't want to upload (e.g. Spam, Virus, Trash boxes). Others, I needed to rename to go into Scalix equivalents.

The following Perl script is what I came up with.


#!/usr/bin/perl

use Smart::Comments;
use Mail::IMAPClient;
use Mail::Box::Mbox;

my $users = [
    {   username   =>  'user1',
        password   =>  'password1', },
    {   username   =>  'user2',
        password   =>  'password2', }, ];

my $folder_translation = {
    'sent-mail'     =>  'Sent Items',
    'sentmail'      =>  'Sent Items',
    'saved-drafts'  =>  'Drafts',
    'saved-messages'=>  'Drafts',
};

my $ignore_folders = [ 
    'spam-mail', 
    'virus-mail', 
    'mail-trash', 
    'Junk',
    'Junk E-mail', 
    'Trash', ];

foreach my $user (@$users) { 
    if( -d "/export/imap/$user->{'username'}") {

        warn "Looking at $user->{'username'}\n";
        my $imap = Mail::IMAPClient->new(  
            Server => 'mail.example.com',
            User    => $user->{'username'} . '@example.com',
            Password=> $user->{'password'},) or 
            die "Could not log in as $user->{'username'}";

       
        opendir DH, "/export/imap/$user->{'username'}";
        my @files = readdir DH;

        FILELOOP:
        foreach my $file (@files) {
            if($file !~ m/^\./) {   # Skip hidden
                foreach my $ign (@$ignore_folders) {
                    if($file eq $ign) {
                        warn "Ignoring $file\n";
                        next FILELOOP;
                    }
                }
                my $translated_name = $file;
                foreach my $tr_key (keys %$folder_translation) {
                    if($tr_key eq $file) {
                        $translated_name = $folder_translation->{$tr_key};
                    }
                }

                # Ready to upload messages
                warn "Uploading $file to $translated_name\n";

                my $folder = Mail::Box::Mbox->new(
                    folder => '/export/imap/' . $user->{'username'} .  '/' . $file);

                my @folders = $imap->folders();
                if(! grep /^$translated_name$/, @folders) {
                    $imap->create($translated_name);
                }

                foreach my $msg ($folder->messages) { ### Uploading msgs |===[%]           |
                    my $uid = $imap->append($translated_name, $msg->string);
                    if(! $uid) {
                        warn "Could not append message\n";
                    }
                }

                $folder->close() or warn "Could not close Mbox connection $@\n";
            }
        }
        closedir DH;

        $imap->disconnect() or warn "Could not close IMAP connection $@\n";
    }
    else {
        next;
    }

}

4:07 AM under linux , mail , open source software , perl , perl programming , scalix