I'm setting up a new server and am debating between the LTS and latest versions. I normally run the newest stuff (my current server for which this is a replacement is latest), but I want 64 bit this time (lots of RAM!).
I want stability but I also want things to just work, and 64 bit has given me headaches in the past, but it was mainly with desktop setups. Is the older, but more stable and longer supported LTS version better to go with, or latest and greatest, but be-prepared-to-upgrade-in-6-months version the best?
Honestly, this depends a lot on what you're doing with the server, and what your expectations are.
If you're building a box that can handle short periods of (scheduled) downtime without problem (updates/upgrades) and isn't running a mission critical service, running Ubuntu's latest stable is going to be fine. Just remember that it will require updates if it's going to remain in production for a length of time, and plan appropriately.
If you're building a box that needs maximum uptime, or if you plan to run commercial software that might offer limited compatibility between versions, then an LTS release is going to be your best bet. This will allow you to run longer (with security updates available) before you have to upgrade or retire the box. Also, as mentioned, many commercial software companies won't support all releases, but will support LTS to simplify things on their side.
Now, as this is for a server, I would tend to lean more towards the LTS release. Server hardware rarely changes extensively once it's in production, and it's easier to verify that hardware before putting it into use. Additionally, on the server side 64 bit support is quite good (in my experience) and very stable. I doubt you'll have a need to upgrade frequently just for better 64 bit support.
If you're at all interested in the newest software don't run the LTS version, run 9.04. As for the problems with 64-bit. I don't believe you'll run into many for a server, the problems are mostly around java and the flash plugin. If you're running software entirely from Ubuntu, and you know it has drivers for everything you need, you should be fine.
I would say that you can use the new 64-bit 9.04 version and just prepare that in the future you can upgrade that version to the next LTS version that comes out.
LTS means that the server and its components get supported for 6 years. You always have the option to upgrade the whole system and its rather easy with Ubuntu. Just be careful when you do that and back your system up. What will probably happen is 90% will work fine and some 10% of your apps might complain. If you don't use anything complicated - like a standard LAMP stack - I wouldn't really worry.
Just remember that if for whatever reason you use 32-bit, you can use more memory then 3gbs (I think 16GB) but your application can only access a maximum of 2Gbs at a time. Meaning, if you run 1 instance of MySQL, it can only access 2Gbs. But if you run 2 instances of MySQL (one on port 3306 and one on 3307) then both instances will use a maximum of 4Gbs of memory.