I have a simple Time Machine Server configuration with 1 destination. Users are defined in Open Directory (but the machines that will be backing up to TMS do not login via Open Directory).
Updated info: Each user has their own machine and are administrators on their own machine. They will log into the Time Machine Server using their own Open Directory username and password. Encryption of TM backups will not be used.
Given this configuration, are backups secured per user when using Time Machine Server on OS X Server (Mountain Lion)?
It depends on what you mean by "per user". There are two layers of security for the backup, based on different users (plus a third layer if you enable encryption of the backup). When you configure a client to back up to a server with Time Machine, you authenticate to the server with a name and password. That client's backup is stored as a disk image on the server with the file permissions set to allow only that user (and root) access to it. The disk image can also, optionally, be encrypted. Within that image, the backed-up files' original permissions are copied, so that on the client, users can only recover files they'd normally have access to. Let me go over these in a bit more detail (in a different order) and explain the implications:
Permissions on the client: If you go into Time Machine's recovery mode on the client, it respects the original permissions of the backed-up files -- a user logged into the client will not be able to examine or recover files they wouldn't have had access to on disk. However, enforcement of these permissions is taking place on the client; if someone has admin acess (or subverts security) on the client, they'll be able to bypass permissions within the backup (just as they can on the client's own disk). Also, if someone gets direct access to the backup disk image on the server, they can mount it in any mode they want (e.g. with file ownership disabled), and get full access to the backup.
Permissions on the server: the TM service is basically AFP file sharing, with an added "use this to back up" advertisement. To back up, the client uses a memorized username and password to mount the Backup folder, then create/mount the disk image within it, and store files into the image. There are several security implications here:
If for example Joe sets up his computer to back up using his server account, and then Bill also creates an account on Joe's computer, Joe would be able to get at Bill's backed up files by simply mounting the shared Backup folder, then mounting the image with file ownership disabled.
Joe's server password is stored in the System keychain on his computer (so that it can reconnect to the server to make backups). This means that anyone with admin access (legitimate or otherwise) to Joe's computer can recover his server password.
Because of this, I tend to recommend that the server account used for Time Machine should not be a regular user account, but a seperate per-computer account used only for backup (and preferrably restricted to only Time Machine and File Sharing in the service access controls), and certainly not sharing a password with any other account.
If the server account used for TM ever has its password changed, the client will not be able to back up or recover files until the entry in its System keychain is updated with the new password. This is another reason not to use a regular user account…
Encryption: If encryption is enabled when TM is set up, the disk image will be encrypted (in addition to the server-based file permissions). The encryption password is also stored in the client's System keychain, so anyone with admin access on the client can recover it (again, don't reuse passwords). This does mean that if the server's security is subverted, the contents of the backup image will remain secure. However, you should make sure that the encryption password is recorded and stored securely, because you're going to need it if the client is lost or its HD crashes or… Unlike the server filesharing password, there's no way to reset or bypass this layer of the security.
UPDATE: Given the added details, yes, each client's backup will be secured based on their Open Directory account. Caveats: anyone with admin access to the server will be able to get at everyone's backups, and the OD account's password will be stored in the client Mac's System keychain (recoverable by anyone with admin access to the client).