Connection to accounts on my VPS hang for up to 20 seconds before a connection is established

To check if it is a DNS or a TCP-connection issue I'd install tcpdump on the server, Wireshark on your client and run some measurements yourself. This needs some console skills though, but it's worth a try.

We will capture raw traffic from your network interface and look into the bowels of the wire to figure out, if it's a DNS or a TCP-problem. However, this won't solve the root cause of your problem, it will only give you pointers were to look further.

This whole posting is based on the assumption, that it's either a DNS reverse-lookup timeout problem or a deeper problem on the network.

After installing tcpdump, figure out on which physical network interface the HTTP requests get to your server. Next, figure the public IP-address of the machine you are working from - you need this to filter out unwanted noise if you run the connection test.

Let's call your server's IP-address A and your client's IP-address B.

Assuming the interface which holds the IP-address of this vhost is named eth0, you should run tcpdump as user root like this:

# tcpdump -n -s 1500 -w /tmp/tcpdump.pcap -i eth0 \(host A and host B and tcp port 80\) or udp port 53

(replace A and B with your IP-addresses; if you use https instead of http replace tcp port 80 with tcp port 443)

The program will start and will capture all traffic between your client machine and your server, as well as DNS packets (pretty much all of them I'm afraid, but we come back later to that) and store the content to the file /tmp/tcpdump.pcap.

Then, fire up your request. After you observed the problem, go back to the console and stop tcpdump by pressing CTRL+C.

Now I suggest you install the program Wireshark on your client machine for deeper analysis.

After installation, download the file from your server to your client machine and load the file in Wireshark through File -> Open.

To check if the TCP-handshake is OK, look for the first three packets. If capturing network traffic was OK, look for the first packet which has the TCP SYN flag set. Mark this using your mouse and choose Analyze -> Follow TCP Stream to filter out this single TCP-connection (we do this because usually you will have quite a lot TCP-connections from a single request to a webserver, but we only want to look into one for now).

A windows will open, just close it again, we don't need that. Now highlight the first packet and chose Edit -> Set time reference (Toggle). This will make all timestamps in the whole window relative to the first packet you marked so that we can have a look if timing is OK. Also set View -> Time Display Format -> Seconds since previous displayed packet if it's not the default.

Look at the first four packets. You will notice, they will have certain TCP-flags set:

1. First packet from your client to the server has the SYN flag set.
2. Second packet from your server to the client has the SYN and ACK flag set.
3. Third packet from your client to the server only has the ACK flag set.
4. The fourth packet now will have your first HTTP GET request.
5. The fifth packet is the server ACKnowledging that the HTTP GET was received
6. The sixt packet is the first data to arrive
7. And this is even more data; if you look into the lower part of Wireshark you see the content.

Have a look at the timestamps of these seven packets. They all should follow very very close to each other; it should be looking something like this:

You can see all these packets are following very closely to each other, they are just hundreds of milliseconds apart. If this is the case for you, the TCP-connection works per se and we look more into DNS. But if this is not OK for you, if you see long delays or even TCP retransmissions or messages about "duplicate ACK" and the such, then you need to look into a deeper networking problem. Back to your VPS-provider then.

However, if this turns out to be OK, we look into DNS. For this we want to check about the following fact:

• Does the DNS server respond in a timely manner for questions?
• What exactly is the DNS question which is causing problems?

I will follow on the following hyptothesis:

• The server sees your client's IP-address and wants to know it's DNS-name
• For this it will send out a so called IN PTR question to the name server
• I could imagine that this so called "reverese lookup" either fails or the nameserver answers to slowly.

So, now you have to do a little number-shuffeling. Assume your client's IP-address is 1.2.3.4. Now reverse the numbers (4.3.2.1) and add the suffix .in-addr.arpa to it. You've now built the correct query for Wireshark. In the Filter-field, say:

dns.qry.name == "4.3.2.1.in-addr.arpa"

In the example below I was asking for the full qualified name of my server at 78.47.204.196, so my filter is dns.qry.name == "196.204.47.78.in-addr.arpa":

In my case you see an interesting effect. My nameserver at 192.168.1.1 did not respond fast enough for my client's taste, so it sent out another request, although this time using IPv6. However, that doesn't really matter, it would just send out the same request again, if you don't have IPv6. You see that the real answer - yalla.hackers-r-us.org - comes 2.96 seconds after the request. It's slow, but the answer is there.

You know need to check the following things:

• Does an answer to the request arrive at all?
• How often is the same question issued over and over again?

If you see the request over and over sent out again in the course of 30 seconds, you know it's a DNS issue.

Sorry for the lengthy posting, but from what you wrote I assumed you're not too deep into these things.

Your instinct that it might be a DNS issue could well be right. On first connection, the VPS tries to reverse-resolve the incoming IP address for the logs, and if the DNS servers it's configured to use don't respond, it'll just sit there and wait for a while.

You could try changing the VPS's DNS servers to 8.8.8.8 and 8.8.4.4 (Google's open DNS resolvers) to see if that makes things any better. Alternatively, ask the VPS provider for its local DNS settings, and make sure they're set correctly in /etc/resolv.conf (or wherever those settings are in WHM/cPanel).

If you were connecting over SSH to the host, I'd agree with the DNS suggestion. Howver, in this case, you're simply testing latency to serve an http request.

Looking at the first site in the chrome browser, I get latency info like:

2.04 s (onload: 2.04 s, DOMContentLoaded: 1.42 s)

The second site is:

4.59 s (onload: 4.40 s, DOMContentLoaded: 4.21 s)

Another tool I like to use is called check host.

Site 1:

http://check-host.net/check-report/1dc3fb

Site 2:

http://check-host.net/check-report/1dc3fa

Some other things right off the bat site 1:

www.google-analytics.com

Site 2:

https://twitter.com/statuses/user_timeline/spare_brain.json?callback=twitterCallback2&count=5

May I suggest also enrolling in the fantastic google webmaster tools:

https://www.google.com/webmasters/tools/home?hl=en