We have an environment of a few thousand users running applications on about 40 clusters ranging in size from 20 compute nodes to 98,000 compute nodes. Users on these systems generate massive files (sometimes > 1PB) controlled by traditional unix permissions (ACLs usually aren't available or practical due to the specialized nature of the filesystem).
We currently have a program called "give", which is a suid-root program that allows a user to "give" a file to another user when group permissions are insufficient. So, a user would type something like the following to give a file to another user:
> give username-to-give-to filename-to-give ...
The receiving user can then use a command called "take" (part of the give program) to receive the file:
> take filename-to-receive
The permissions of the file are then effectively transferred over to the receiving user.
This program has been around for years and we'd like to revisit things from a security and functional point of view.
Our current plan of action is to remove the bit rot in our current implementation of "give" and package it up as an open source app before we redeploy it into production.
Does anyone have another method they use to transfer extremely large files between users when only traditional unix permissions are available?
As xryl669 says you can use a directory to actually share the files. It should look like this:
The give command becomes
The take command looks something like this:
If the emitter is really willing to give the file away, you can use an SUID binary that moves the file to a directory that's writeable by all and has the sticky bit (like
/tmp
), then changes ownership to the new owner.chown(3)
already takes care of removing theset-user-ID
andset-group-ID
bits for you. This way the new owner can do what he wants with the file, including moving it.This directory writeable by all can belong to the user's home directory, in case you want to use multiple filesystems for home directories and want to make sure you don't cross filesystem boundaries as performance would immediately be terrible. In this case you'll probably want to make sure the recipient knows when a new file is offered.
E-mails would do the trick. A more Unixy solution would be a
/etc/profile
that lists your newly delivered files. Added bonus if you offer this feature withpam_echo
(eg withfile=/tmp/deliveries/%u
, seepam_echo(8)
). As with anything PAM-related, you'd want to check that all your implementations offer such a module first.You could use a system with a shared directory, (possibly without execute perms.), where things for a given user are archived with a specific file name structure (
to-$username_from-$username.tar
, for example). Give makes the file andchowns
it to the target user; take extracts the file and removes it.If you want to do it as an actually move (IE, change file location and permissions; no copying because of the giant file size), you might be able to get away with moving to a shared directory with -x perms (so nobody can list files there), and the same
chown
method.mv
,chown
/mv
.I'd suggest to rewrite the app to indeed mimic a "give" and "take", but rather "push" and "pull" it from a protected directory. Your directory can be only accessible for the push/pull app - which handles the file moves. Alternatively, your app/script can create a random, temporary directory with permissions being set for the sender and receiver only.
Wish to have more security? You can PGP encrypt/sign the file (using the public key of the receiver).
In terms of redoing it from a "security and functional point of view", I'd strongly suggest not to create SUID programs. If you don't drop privileges in a proper way, you can virtually access any file on the system. If your program is buggy (buffer overflow, etc...) - one can exploit this to gain root access on your system.
This is probably no use to you but for reference cp --reflink source target does thin copies of files using copy-on-write.
This means you could copy the file outright and only changed blocks would actually be copied. Unlike a hard link the new file has its own inode and metadata meaning you can then provide the copy of the file to the new user using standard chown stuff.
As far as I am aware this is a feature that is only available on OCFS2 and btrfs currently. I guess that does solve your problem but seeing as the availability of it is not widespread it probably wont be useful.