We are moving ~60 user mailboxes from a Postfix/Dovecot/Maildir setup into Exchange 2007. I am aware of Microsoft's Transporter Suite, which will allow for an IMAP-to-Exchange conversion of mailboxes in bulk. However, there is a concern about how to clean up a botched process, should there be an unforeseen issue. Apparently, attempting to remove a corrupted Exchange mailbox for a user also results in a rather scary-sounding dialog that implies the user account will be removed as well, not just the corrupted box itself. While the original email store (in Maildir format) is available, we would need the ability to remove the bad box(es) in question and repeat the process, so this method is not a viable option.
I have grave concerns at this point about affecting a batch- or bulk-style transition without massive manual intervention. We are trying to avoid visiting 50+ desktops and manually moving email by using Outlook or Thunderbird as a "pivot" agent; while this is 100% guaranteed to work, it will take weeks or months to move over 2.5 million messages one account at a time.
So I have begun to look into other tools and methods, both open-source and commercial.
The first one I tried was OfflineIMAP, which as it turns out, is not very Exchange-friendly; a subtle effect of the migration process results in exhaustion of Exchange's named properties due to a unique header being generated for each email moved. A patch is available for this that changes the unique header into a single, generic header that has a unique identifier after it, avoiding the exhaustion issue. Yet, after applying the patch, there are still some issues. A pity, really, as it would have been nice to keep them in sync.
On the commercial side, I took a brief look at Transend Migrator in a trial mode. The results were mixed and there appears to be little documentation delivered with the binary. This is hardly encouraging, especially for something that end-users will readily notice if it goes horribly wrong.
There are other issues as well. The mailboxes are stored as Maildir, but the box names conform to Thunderbird's expectations (we've been on TBird for several years). Many box names do not match (Sent vs. Sent Items, Trash vs. Deleted Items, etc.) and will need to be translated effectively as they are ported. Yes, we pamper our end-users that much - a successful migration will consist of informing them we are switching to Outlook and giving everyone a small 30 minute class, followed by people openning their new Exchange-based mailboxes for the first time.
Someone, somewhere surely has something that works...
EDIT: A follow-up
The larch script, written in Ruby, has provided the easiest solution, although with a few caveats. Here's a heavily condensed version of what happens:
- Obtain the user's username and password. Have the password reset after the transition.
- Connect to the user's box via IMAP and ensure that the INBOX is subscribed, and that all messages are marked as read. There is an issue on the Exchange side that can prevent messages from successfully importing if the message is unread and in a certain state...
- Change the postfix transport map to point to the exchange server, which causes all new deliveries to arrive at the Exchange server instead of the current email server.
- Use the
larch
command to move the email over, omitting the user's Trash can. We had to capitulate and omit this because there were users turning the Trash into a personal filing system. - Note any messages that didn't transition. Re-examine and attempt to recopy them again. This will only move the newer messages.
- Create a record in MySQL that instructs Dovecot to go into proxy mode for that user, and point the record at the Exchange server. This allows existing clients to connect unchanged until we are ready to set up their Outlook install.
- Re-connect to the user's Inbox and check to see that all messages are copied.
I've done many such migrations on much larger systems; 300+ mailboxes and I've migrated in many directions: Courier -> Cyrus, Exchange -> Cyrus, Courier -> Exchange, and Cyrus -> Exchange...
Our tool of choice is called imapsync; an open source Perl script. It uses the actual IMAP protocol to handle migrations thus alleviating the need to deal with underlying specifics to each implementation (aildir formats, mailbox annotations such as DONT (.) vs. slash (/) seperators.
IMAPSYNC is also re-entrant safe. You can run it multiple times over the same accounts and it will only copy what has not been copied the first time through or any new emails that may have arrived.
We typically dump all the user accounts to a BASH script that, in the end, is executed like so:
Where host1 is where you're copying from host2 is where you're copying to.
You can get IMAPSYNC from here: http://freshmeat.net/projects/imapsync/
The best way to go is using imap!
You COULD let each user move the messages himself by giving him both accounts in Outlook.
I did a migration from Cyrus to Exchange 2003 using a separate linux box with the Debian package "uw-mailutils" and this command:
mailutil transfer -verbose {imap.xxx.org:143/imap/user=m_xxx}INBOX {exchange.xxxx.org:143/imap/[email protected]} INBOX
mailutil is from the UW-IMAP project. Every Linux distro should contain it.
When it failed (which it didn't) I would just go into the exchange account and delete "INBOX"
I've used the "imapsync" program before to mmigrate from an IMAP server to Scalix (an Exchange-like system). Think rsync for IMAP mailboxes.
Jim.
We have a huge large mail system, and routinely ingest users from other mailsystems (typically Notes or Groupwise). One common thread in all of these migrations is that the data migration piece is always a disaster. There are always intractable problems, VIP issues end up taking priority and the normal user population suffers. Migration tools have bugs that never get fixed because nobody does a Notes->Exchange migration twice!
What we're looking at for a future strategy is a mostly greenfield mailbox, with users identifying email that must be retained. For you its easy -- keep the dovecot servers running for 6 months, setup Outlook profiles pointing to the old imap system (I believe this is scriptable with Outlook 2007) and let users migrate in their own time.
You'll find that most folks really don't need what they say they need, and after 6 months, any legit need to move data to the new system will be done.
Another option is to use a mail archiving solution. We have to archive all email due to litigation, and oftentimes you can "ingest" messages from an old system into a hosted archiving solution. Check out Postini and Microsoft Hosted services.
Use the Transporter Suite. It was written specifically for this purpose.
Basically, you import a CSV file of the IMAP users and passwords, and it will import the data correctly. You can also import users one at a time if you are scared of doing a batch process. It's actually pretty hard to botch an import.
Do not use imapsync or any other IMAP to IMAP based sync engine. It sort of works, but much of the metadata will be screwed up -- specifically dates. Even with the --syncinternaldates option, Exchange will still overwrite the dates to the current date, and you will have a botched migration.