Tiger to Leopard Server Migration, Part Three
Picking up where the last article left off, this article looks at setting up the mail service in Leopard, including migrating mail data from Tiger.
Configuring the Mail Service
In the Server Admin application, I jumped right to the Settings part of the Mail section and filled out things appropriately. In the “Domain name” field I entered the fully qualified domain name of the server in its role as a mail server; this is the same address that is in the MX records in DNS. For example, “mail.netmojo.ca”.
The “Host name” field contains the fully qualified domain name of the server itself. “leopardserver.netmojo.ca”, for example.
I checked off Enable IMAP, SMTP, and “Copy undeliverable mail to:”, with an appropriate postmaster email address. In the Filters tab, I enabled Junk mail and virus scanning to my taste.
I would like to use the mailling list feature some time, but I’ll come back to that later, so for now I left it disabled.
In the Logging tab, I increased the verbosity of each section, since I like to know what is going on with my server, and I monitor the logs as much as I can. To save on disk space, I also checked off the Archive option, setting it for 7 days.
In the Advanced tab, I checked off all of the authentication methods for SMTP and IMAP, except for CRAM-MD5, which my clients won’t be using. In the SSL settings, I selected “Use” for SMTP and “Require” for IMAP. I selected a self-signed SSL certificate that I created with the “Manage Certificates” feature of Server Admin.
Migrating IMAP Email: Bad Documentation
I discovered that the Upgrading and Migrating book contains an egregious error.
In the “Migrating from Mac OS X Server v10.4″ chapter, on page 40, it describes how to migrate the mail database. Step 8, “Run the mail database upgradedb script” shouldn’t be there. That script is for Panther.
The mail database between 10.4 and 10.5 does not need to be modified – basically, no migration is necessary, just copy the files over. The advice in the Upgrading and Migrating manual contradicts advice in the Mail Service Administration manual:
Moving Mail Messages from Apple Mail Server to Mac OS X
Server v10.5If you have upgraded your server from a version prior to Mac OS X Server v10.3 and you have an existing Apple Mail Server database, you must migrate your mail database to Mac OS X Server v10.5 Mail service. If you are upgrading from Mac OS X Server v10.3 or v10.4, no migration is necessary.
To migrate the data, all you need to do is copy it over.
Copy The IMAP Files From Tiger to Leopard
If your IMAP files are in the default location, you can back them up to an archive file using tar. On your 10.4 Server:
cd /
mkdir /Volumes/disk2/imap
tar czvf /Volumes/disk2/imap/var-imap.tgz var/imap
tar czvf /Volumes/disk2/imap/var-spool-imap.tgz var/spool/imap
On my server, “/Volumes/disk2″ is a disk that is shared between my 10.4 install and my 10.5 install.
On the 10.5 Server, make sure that the Mail service is not running, and backup the same directories:
sudo bash
cd /var
mv imap imap.original
mkdir imap
chown _cyrus:mail imap
ls -l | grep imap
drwxr-xr-x 2 _cyrus mail 68 Nov 14 18:16 imap
drwxr-xr-x 16 _cyrus mail 544 Nov 7 19:52 imap.original
Do the same thing for /var/spool/imap. Then, extract the archived files:
cd /
tar xvf /Volumes/disk2/imap/var-imap.tgz
tar xvf /Volumes/disk2/imap/var-spool-imap.tgz
One bit of advice that I did follow from the Upgrading and Migrating manual is step 9 on page 41: reconstruct the mail index files:
/usr/bin/cyrus/bin/reconstruct -i
Enable Alternate SMTP Port
Many big ISPs now block outgoing traffic on port 25, to fight spam and irritate their customers. A work-around is to accept mail on the alternate SMTP port, 587, also known as the “submission” service. It is easy to enable: open /etc/postfix/master.cf in a text editor, and uncomment the line:
submission inet n - n - - smtpd
If your mail server is already running, you need to restart it for the changes to take place (run “postfix reload”). Remember to allow connections to this port in your static firewall rules, at some point.
Setup Requisite Email Aliases
Every UNIX server has a few standard email addresses: root@hostname, and postmaster@hostname, for example. You need to tell the server where to send email that the system sends to these addresses. To do that, you need to edit the /etc/postfix/aliases file.
Edit at least the first aliases, at the top of the file:
# Person who should get root's mail. Don't receive mail as root!
root: brentk
# Basic system aliases -- these MUST be present
MAILER-DAEMON: postmaster
postmaster: root
This tells the system to send email addressed to root@ to brentk@, and email going to MAILER-DAEMON@ to postmaster@, and email going to postmaster@ to root@. So ‘brentk’ gets everything.
If you have any other email aliases, you could add them to this file too. Mail doesn’t need to go to a local user; instead of just ‘brentk’, I could have put ‘brentk@gmail.com’ (not my address — apologies to the imposter) instead.
Once you have edited the aliases file, you need to run the “newaliases” command.
Start the Mail Service
Before starting the Mail service, I want to make sure that the server does not accept any incoming email — I’m not done configuring the server yet, and I don’t want it accepting new mail from external sources. To be sure that the server doesn’t accept any email, I added a temporary firewall rule to block port 25 in general, but to allow it from me:
ipfw add 00002 allow tcp from 123.456.789.111 to me dst-port 25
ipfw add 00002 allow tcp from 123.456.789.111 to me dst-port 587
ipfw add 00003 deny tcp from any to me dst-port 25
ipfw add 00003 deny tcp from any to me dst-port 587
In Server Admin, when I started the Mail service, I got the warning that “Service port may be restricted”. I watched the logs as it started up, and everything looks good.
Nov 14 18:36:13 leopardserver master[695]: process started
Nov 14 18:36:14 leopardserver ctl_cyrusdb[700]: verifying cyrus databases
Nov 14 18:36:14 leopardserver ctl_cyrusdb[700]: skiplist: recovered /var/imap/mailboxes.db (288 records, 27996 bytes) in 0 seconds
Nov 14 18:36:14 leopardserver ctl_cyrusdb[700]: skiplist: recovered /var/imap/annotations.db (0 records, 144 bytes) in 0 seconds
Nov 14 18:36:17 leopardserver ctl_cyrusdb[700]: done verifying cyrus databases
Nov 14 18:36:17 leopardserver master[695]: Cyrus POP/IMAP Server v2.3.8 ready for work
Nov 14 18:36:17 leopardserver ctl_cyrusdb[731]: checkpointing cyrus databases
Nov 14 18:36:18 leopardserver ctl_cyrusdb[731]: done checkpointing cyrus databases
Nov 14 18:40:54 leopardserver master[847]: process started
I fired up Mail.app (client), and connected to IMAP. It logged in fine, and all of my mailboxes and email messages are there. I can see from the Mail Access logs that I logged in using plaintext+TLS, so my authentication data did not go over the wire in the clear.
Trouble with SMTP
I encountered a little problem when I tested sending email. Fortunately, /var/log/mail.log told me exactly what the problem was:
Nov 14 19:40:23 leopardserver postfix/tlsmgr[1828]: warning: no entropy source specified with parameter tls_random_source
Nov 14 19:40:23 leopardserver postfix/tlsmgr[1828]: warning: encryption keys etc. may be predictable
Nov 14 19:40:23 leopardserver postfix/smtpd[1827]: warning: cannot get private key from file /etc/certificates/mail.netmojo.ca.key
Nov 14 19:40:23 leopardserver postfix/smtpd[1827]: warning: TLS library problem: 1827:error:0906406D:PEM routines:DEF_CALLBACK:problems getting password:pem_lib.c:105:
Nov 14 19:40:23 leopardserver postfix/smtpd[1827]: warning: TLS library problem: 1827:error:0906A068:PEM routines:PEM_do_header:bad password read:pem_lib.c:401:
Nov 14 19:40:23 leopardserver postfix/smtpd[1827]: warning: TLS library problem: 1827:error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib:ssl_rsa.c:709:
Nov 14 19:40:23 leopardserver postfix/smtpd[1827]: cannot load RSA certificate and key data
The SSL certificate was not being loaded. The reason was because I had added a password to it. I edited the certificate using the Certificate Manager feature of Server Admin, removed the password, and ran ‘postfix reload’. When I retried sending with my mail client, I got a pop-up message asking whether to accept the self-signed certificate. I accepted it, and the mail was sent.
Curious how IMAP worked with TLS, but SMTP didn’t. I guess Postfix is more picky than Cyrus.
Authenticating with Kerberos
In my next article, I will look at setting up client computers to authenticate to the server. Part of this will be getting Kerberos (”single sign-on”) services working, which will also provide an even more secure mail login.
November 27th, 2007 at 09:54
great articles,
eagerly awaiting part 4
December 3rd, 2007 at 08:55
Same here, very good stuff and much appreciated. I have only joined the Mac server admin world since after Tiger was released and these articles have made the migration process much easier.
Why couldn’t apple’s documentation put it that clearly?
Keep up the excellent work.
December 4th, 2007 at 01:22
Thanks for the support. Part 4 is coming soon!
I did post two other articles on some leopard server configuration issues:
* http://www.netmojo.ca/blog/2007/12/01/setting-up-sieve-and-vacation-messages-on-mac-os-x-server/
* http://www.netmojo.ca/blog/2007/11/21/locked-out-of-leopard/
Cheers,
Brent
December 18th, 2007 at 08:01
Hello, thanks for the very instructive work! I can’t wait for your part 4 as I am going absolutely mad about theis AFP & Kerberos thing. I just can’t figure it out…..!!
January 18th, 2008 at 00:34
Great series of posts, they make me far more confident in what I’m about to do in a couple of hours. Wish me luck ;-) BTW, have you considered migrating mail boxes via the mailbfr script of http://osx.topicdesk.com/ ?Anybody going that way, I suggest they do a full backup of all mail settings on Tiger, then restore only mail data on Leopard and tweak the config files manually.
October 17th, 2008 at 12:26
I did this and even postalias /etc/postfix/aliases before newaliases
but it doesn’t seem to have any effect. Postmaster mail is not ent to the OD user I redirected it to. anything else I might be missing?
(OSX 10.5.5 server)
October 17th, 2008 at 12:55
What does /var/log/mail.log say when you send an email to root@?
October 17th, 2008 at 12:58
You could also try reloading postfix after running newaliases:
sudo postfix reloadOctober 20th, 2008 at 04:59
Hi Brent,
There seems to be a paradox. Postfix is running but claims not to be.
I did reload and sent a mail to root.
Oct 17 20:31:03 server postfix/postfix-script[18964]: fatal: the Postfix mail system is not running
Oct 17 20:32:06 server postfix/postalias[19040]: fatal: open /etc/aliases.db: Permission denied
Oct 17 20:33:56 server postfix/postfix-script[19135]: fatal: the Postfix mail system is not running
Oct 17 20:39:04 server postfix/postfix-script[19334]: fatal: the Postfix mail system is not running
Oct 17 20:40:10 server postfix/postfix-script[19450]: fatal: the Postfix mail system is not running
Oct 17 20:45:53 server postfix/postfix-script[19700]: fatal: the Postfix mail system is not running
Oct 17 20:46:07 server postfix/postalias[19754]: fatal: open database
/etc/postfix/aliases.db: Permission denied
Oct 17 20:46:44 server postfix/postfix-script[19784]: fatal: the Postfix mail system is not running
Oct 20 13:53:03 server postfix[13322]: error: to submit mail, use the Postfix sendmail command
Oct 20 13:53:03 server postfix[13322]: fatal: the postfix command is reserved for the superuser
Oct 20 13:53:44 server postfix/postfix-script[13361]: fatal: the Postfix mail system is already running
October 20th, 2008 at 08:27
Willard, note the permissions errors. Prefix the command with ’sudo’.
However, by now it doesn’t matter, because postfix polls the alias database, and any changes that you have made would have taken effect. Perhaps you didn’t run newaliases with privilege either. Try ’sudo newaliases; sudo postfix reload’. Note also, whether there are any error messages after you run newaliases.
October 20th, 2008 at 08:49
Hi Brent,
Somehow the mail.log stopt populating with data
the mailaccess.log shows this (doesn’t look good to me)
Oct 20 10:27:12 server pop3[5100]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 10:31:09 server pop3[5265]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 10:32:12 server pop3[5312]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 10:34:32 server imap[5397]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 10:37:12 server pop3[5504]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 10:41:09 server pop3[5675]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 10:42:02 server ctl_cyrusdb[5710]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 10:42:02 server ctl_cyrusdb[5710]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 10:42:02 server ctl_cyrusdb[5710]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 10:42:02 server ctl_cyrusdb[5710]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 10:42:02 server ctl_cyrusdb[5710]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 10:42:02 server ctl_cyrusdb[5710]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 10:42:12 server pop3[5721]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 10:47:12 server pop3[5915]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 10:51:09 server pop3[6080]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 10:52:12 server pop3[6123]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 10:57:12 server pop3[6318]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 11:01:09 server pop3[6483]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 11:02:12 server pop3[6531]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 11:07:12 server pop3[6720]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 11:11:09 server pop3[6892]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 11:12:02 server ctl_cyrusdb[6935]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 11:12:02 server ctl_cyrusdb[6935]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 11:12:02 server ctl_cyrusdb[6935]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 11:12:02 server ctl_cyrusdb[6935]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 11:12:02 server ctl_cyrusdb[6935]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 11:12:02 server ctl_cyrusdb[6935]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 11:12:12 server pop3[6946]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 11:42:02 server ctl_cyrusdb[8113]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 11:42:02 server ctl_cyrusdb[8113]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 11:42:02 server ctl_cyrusdb[8113]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 11:42:02 server ctl_cyrusdb[8113]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 11:42:02 server ctl_cyrusdb[8113]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 11:42:02 server ctl_cyrusdb[8113]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 12:12:02 server ctl_cyrusdb[9304]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 12:12:02 server ctl_cyrusdb[9304]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 12:12:02 server ctl_cyrusdb[9304]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 12:12:02 server ctl_cyrusdb[9304]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 12:12:02 server ctl_cyrusdb[9304]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 12:12:02 server ctl_cyrusdb[9304]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 12:42:02 server ctl_cyrusdb[10485]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 12:42:02 server ctl_cyrusdb[10485]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 12:42:02 server ctl_cyrusdb[10485]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 12:42:02 server ctl_cyrusdb[10485]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 12:42:02 server ctl_cyrusdb[10485]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 12:42:02 server ctl_cyrusdb[10485]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 13:12:02 server ctl_cyrusdb[11680]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 13:12:02 server ctl_cyrusdb[11680]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 13:12:02 server ctl_cyrusdb[11680]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 13:12:02 server ctl_cyrusdb[11680]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 13:12:02 server ctl_cyrusdb[11680]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 13:12:02 server ctl_cyrusdb[11680]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 13:22:16 server imap[12084]: TLS server engine: No CA file specified. Client side certs may not work
Oct 20 13:42:02 server ctl_cyrusdb[12863]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 13:42:02 server ctl_cyrusdb[12863]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 13:42:02 server ctl_cyrusdb[12863]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 13:42:02 server ctl_cyrusdb[12863]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 13:42:02 server ctl_cyrusdb[12863]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 13:42:02 server ctl_cyrusdb[12863]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 14:12:03 server ctl_cyrusdb[14225]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 14:12:03 server ctl_cyrusdb[14225]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 14:12:03 server ctl_cyrusdb[14225]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 14:12:03 server ctl_cyrusdb[14225]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 14:12:03 server ctl_cyrusdb[14225]: DBERROR: couldn’t checkpoint: Invalid argument
Oct 20 14:12:03 server ctl_cyrusdb[14225]: DBERROR: sync /var/imap/db: cyrusdb error
Oct 20 14:17:00 server ctl_cyrusdb[14765]: DBERROR: critical database situation
Oct 20 14:17:09 server ctl_cyrusdb[14830]: cleaning up corrupted database files in: /var/imap/db
Oct 20 14:17:09 server ctl_cyrusdb[14830]: —-: removed file: __db.001
Oct 20 14:17:09 server ctl_cyrusdb[14830]: —-: removed file: __db.002
Oct 20 14:17:09 server ctl_cyrusdb[14830]: —-: removed file: __db.003
Oct 20 14:17:09 server ctl_cyrusdb[14830]: —-: removed file: __db.004
Oct 20 14:17:09 server ctl_cyrusdb[14830]: —-: removed file: __db.005
Oct 20 14:17:09 server ctl_cyrusdb[14830]: —-: removed file: log.0000000001
October 20th, 2008 at 08:50
let me switch on notification so I don’t miss you….
October 20th, 2008 at 09:43
It looks like you have some Cyrus IMAP problems. Start with the Mail Service Admin manual, page 57.
October 20th, 2008 at 10:55
Hi,
I just did everything that’s in there, in fact I already did it before too. Is there a way to get rid od the corrupted db etc and start from scratch. I have backed up everything so I’m willing to take the risk.
thanks