I occasionally look at sites like netcraft and am curious if there is an unobtrusive way to ping a server and see what publicly facing software it is running? Are sites like netcraft using some sophisticated heuristics to infer their data or can certain kinds of (non-abusive) requests lead to straightforward answers?
NMAP allows you to do OS detection and service identification. I don't know how netcraft does it -- and using nmap could very quickly move out of the 'unobtrusive' category. But you can test it in-house for sure...
I think NetCraft use the Server: header returned by the webserver. No special tools are needed to do this
It's based on the ICMP fingerprint in the packets that you get back (if you get them back). Different IP stacks reply different to echo's and that gives you a hint to what is on the inside. Check this out.