For an Microsoft Exchange server, is it possible to encrypt the email database such that the sysadmin cannot see the emails?
In other words, the admin would be responsible for all aspects of running the Windows server and Exchange process, but would not be able to see the contents of any one email (except those sent to him, obviously). Only another individual (e.g. company owner) would be able to see all emails contained in the database.
The "supported" answer to what you're looking for involves using a public key infrastructure (PKI) and using the built-in encryption and digital signature functionality in Microsoft Outlook to encrypt/sign messages "client side". Anyone on the Internet sending you messages would need to encrypt email they're sending to you.
This is strictly a client-side issue. There are no mechanisms in current versions of Exchange to handle encryption of email server-side.
A better solution would be to audit access. Totally agree with sentiments above - an administrator is by definition, someone who you must trust. However, audit trails can be used to check that their level of privilege isn't being mis-used.
In short, no. Sorry. As others have noted, if you don't trust your admin change admins. You can always setup physical controls at the server, no RDP, two sets of eyes at console.... But really, you need to trust IT.
And BTW- from an encryption standpoint, you are really more asking if mailboxes can be encrypted. This would involve some sort of asymmetric, PKI based encryption style. I think there are some solutions out there. But still, SMTP comes in plaintext....