Based on what you all see out there what is a recommended configuration of /tmp on a server system and why. I've had discussions on these points over the years sometimes with basic disagreements.
The following are basically the questions I see. Some might suggest that these questeions be asked with several questions, however, I think it might be easier for administrators if this information was under one heading. I'm sure this will be informative.
Specifically for /tmp:
Should ln -s /var/tmp /tmp?
Should /tmp be preserved between reboots or not?
Should /tmp be on a real disk area or allowed to be implemented basically on the SWAP area (or tmpfs)?
Should /tmp be on a different disk from the / (root) disk?
Would you place /tmp on a different disk controller from the / (root) disk?
Any rules of thumb for the size of /tmp?
How would you manage /tmp space while the system is up? Delete all files > certain age? Leave area alone until it reaches a %age of max?
Should any procedural items be placed into effect to govern this area?
Specifically for /tmp:
In the case of a complete in-memory disk image (think "live boot CD") this might be acceptable, as every byte of RAM needs to be squeezed. Otherwise, unless you are hard-pressed for disk space, no. /var has its own peculiarities and mixing /tmp with /var/tmp may have unintended consequences when performing systems maintenance. It also adds an extra dependency in that /tmp must be mounted for /var/tmp to function properly; not everything needs /tmp and you may have a situation where you want to migrate it to a different partition or drive, but can't, because you don't want to unmount /var.
No. If you are relying on this as a consistent behavior then you will, sooner or later, encounter issues.
When it's heavily used, this is a temptation - "we'll put /tmp into a RAM disk, it'll speed up access, and when the system reboots/shuts down, there's nothing to clean up". However, if you are thinking of implementing temp space as a RAM disk that will be swapped, then I would consider the ramifications of your system's swap space usage by other programs. If swap is there as a form of "emergency overflow" for when the system is in dire straights and needs it, the last thing you need is to have swap space consumed by a runaway process filling /tmp, consuming memory, causing pressure on the VM subsystem to swap to disk. Between swap activity, and the additional I/O streaming into the RAM disk (which in turn may cause additional page-ins to satisfy a seek() ) your system will quickly become I/O bound.
Preferably, yes, although it's not necessary. If you make heavy use of it, or have a constant workload that requires it, then definitely yes. Hypothetical example: a database that dumps temp files to /tmp would gain a slight speedup by introducing /tmp to a separate spindle (ie. drive).
If you have a requirements for recoverability or speed, then it should be considered.
It should accomodate 2x your expected workload. By this, I mean that if you have local users regularly using this space, sooner or later someone will do something silly and attempt to fill it up. Having a slight overage will allow you to avoid strange "issues" with programs that stop because their temp files have filled up what space is left.
If this is a "common services" installation, where the server provides one or more network services, but does not host users, then this will probably be on the low side. If this is a multi-user installation, this will be on the high side (yes, there are still places that host actual users and not just their network services).
Look into the tmpwatch command, I think you'll find it suits this part of your question(s) nicely. The command simply deletes any files past a certain age in hours. Depending on how fast it fills up, you could do 30 days, 45 days, 90 days, etc.
I would recommend the following:
The rest is a matter of your specific needs.
An awful lot depends on the particular application load you're using. Some application servers (SunONE, old netscape stuff) write a few hundred to a few thousand files to /tmp - in that situation, you really don't want it to be a mounted ramdisk, and there's no reason to preserve it between reboots.
Servers are becoming less general and more special purpose- this type of question (and the similar "how do I partition my system" question) really depends on your loadout.
I did recently have one server that was rebooted for the first time after 4 years or so- it got stuck halfway through the boot deleting all the files in /tmp - there was so much cruft there that it took a good hour to clean it up. Definitely a good idea to clean it periodically if your box doesn't reboot very often.
From experience I recommend not mixing your /var/tmp and /tmp directories.
The reason being /var is (hopefully) where all your log, cache and service data (e.g. databases) will reside. It is generally a good idea to place /var on a separate partition so if a significant data event (e.g. lots of logging or database writes) occurs your root and /tmp partitions will still have free space to reliably operate.
For example I have literally just returned from a site where this practice was not followed (i.e. everything was on one partition) and as a result of log build up the entire system was brought to its knees. If a sane partitioning layout had been followed, the /var partition would have run out of space, but the server would have stayed responsive.
Only answering some points...
I would put /tmp on a different partition than /, just so that I could mount / ro during normal operation. Separate disk? That depends on how much use /tmp and / get and whether they get in each other's way.
As for parts 1 and 2, on my box (Mac OS X) the /var/tmp is not cleaned out daily but /tmp is, so they have different policies and therefore should not be symlinked together. I'm not sure that cleaning /var/tmp daily wouldn't break anything and would want to investigate that before becoming more vociferous - it's only 172k at the moment.
One item I would consider - in reference to your last question - is to make /tmp noexec,nosuid - that is, no executables will be able to run from /tmp, and no suid binaries will be able to switch userid. However, this might affect some programs, so test it before you rely on it completely - I think ssh was even one of these, but I forget.
This is a security measure that will improve your security in regards to /tmp.
Another thing: a lot of users will try to use /tmp as permanent temporary storage: any enforced cleanup of /tmp has to be done along with education of the users: remind them that /tmp is temporary and anything placed there can disappear at any time.
Have you considered what happens when /tmp fills up in this situation? It's not the sort of thing you do twice. I've done it once (on Solaris /tmp will eat up all available RAM and swap), and it brought the server to it's knees.