I have a ubuntu development server at work. It is an old rack server that is located somewhere in the company's dungeon, where nobody ever goes. The only way it can work is as a so-called headless server (i.e. with no monitor/keyboard connected and only accepting network connections).
Obviously, if you just need terminal access ssh is more than enough. I, however, would also like to connect to the graphical interface from time to time. At the moment I am using the built in VNC functionality but I am pretty sure this is neither the most efficient nor the most secure way of approaching this issue.
I have done a bit of research on the issue but failed to come to any definite conclusions. I read about trying to forward the X environment over ssh, which would at least solve my security concerns. Can anyone share their experiences in setting this up? Is there any other way that might be worth looking at?
I almost exclusively connect to this server from a Windows machine. I don't know if that might be a problem for some of the methods suggested.
Yes, X forwarding over ssh is a beautiful thing indeed. It allows you to use graphical applications on an app by app basis and have windows handled by your own desktop environment. You do not even need a desktop environment installed on the server.
You do need to set up some authentication things for it to work though. I believe you need xauth for that.
It's SO much faster than VNC as well. VNC was always rather laggy in my experience.
Edit: I have no experience using this method via Windows, but I found this tutorial for you if you're interested.
You could use freenx instead of vnc. Freenx transmits x-commands (with caching) instead of bitmaps.
Xming and XDMCP is a brillant option.
Edit:
Xming is a x-server for windows, which is based on Cygwin and has the ability to share the clipboard and implements different desktop layouts as well.
XDMCP is a simple and - important - unencrypted protocol (don't use it over the internet) to connect a X-displaymanager and a x-server.
I don't know about the "best" way, I guess YMMV, but here's a fairly comprehensive overview of tools at your disposal: http://www.mynitor.com/2010/02/07/15-remote-desktop-solutions-for-linux/
I particular here's my experience:
nomachine nx - Impressive speed. In its native mode it feels like you're in front of the console. It supports starting new sessions as well as shadowing the console (but shadowing is slower). It supports detaching and attaching to sessions. Clipboard sharing only worked one way for me and I wasn't able to fix it. nomachine offers free server and client packages with some licensing restrictions.
FreeNX is built on the nomachine nx libraries
x2go - Impressed with this also especially sound redirection but I didn't use it for as long as nomachine nx because I found out about it later.
x11vnc - Great vnc server that is able to attach to the console session. Used it in conjunction with nomachine nx. I'm not sure if it can be run in headless mode but I'm listing it in case it can, because for me it was the fastest vnc server around.
xrdp - Looked promising but it looks to be unmaintained. The OpenSuse nomad solution is based on xrdp and I hear nomad is the best remote desktop experience you can get on linux.
teamviewer - The linux version is still beta, runs under wine and consumes some CPU but I used it a lot quite recently and I was pleased with it. Not sure if it works on a headless server.
Also consider xpra, which allows you to detach to an running session from somewhere else, like "screen for X".
There's also Window Shifter, a front end for xpra, which should work also for Windows, see the demo.
While inherently insecure, you could use XDMCP over a local network. I use it frequently to access virtual machines. Its just like logging on to your desktop, except you chose the remote server.
I use x11vnc, which works very well for me. It lets you keep a persistent graphical session between connections, so you can disconnect and reconnect and everything will be just the way you left it. It also supports tunneling over SSH. See these Ubuntu community docs for a short description and some instructions on setting it up.
I have successfully used freenx on Ubuntu using the Windows client from http://nomachine.com.
For me this was to allow me to use an old tablet PC (that struggled even with a clean install of Windows XP) as a front end into a more powerful Ubuntu desktop machine.
The only issue I had was that you needed to turn the "Visual Effects" to none to get decent performance.
I was going to comment on txwikinger's answer suggesting nxserver, but I don't have enough reputation.
Setting up for remote X access can be daunting, and involves multiple steps.
I use x2go, which is load-and-go, and exceptionally easy to use. It gives you a full desktop, just as if you had connected a new screen and keyboard to your server. It has full X functionality.
x2go is NOT a "remote desktop" solution. You get a completely new session. However, you can disconnect and reconnect from the same or different machine with the x2go client; your desktop is persistent. Best of all, the desktop on your client machine is completely scale-able in real-time simply by resizing the window.
x2go uses ssh for transport, so your data is encrypted. This simplifies traversing firewalls. Speed doesn't feel impacted by the use of ssh for transport.
x2go client software is available for Linux and Windows, and Mac.
Very nice free and open source software. I use it every day.