We have upgraded some of our routers to Ubuntu 16.04 and are now getting some performance problems with DNS. It seems that packets are sometimes truncated, but I have no clue what else I can do:
This are the messages from logfile:
Jun 8 10:33:01 proxy named[2827]: success resolving 'b1sync.zemanta.com/A' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:33:05 proxy named[2827]: success resolving 'b1sync.zemanta.com/AAAA' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:33:05 proxy named[2827]: success resolving 'px.owneriq.net/A' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:33:25 proxy named[2827]: success resolving 'deliveryengine.synchroscript.adswizz.com/AAAA' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:33:42 proxy named[2827]: success resolving 'acl.stayfriends.de/A' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:34:36 proxy named[2827]: success resolving './NS' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:34:38 proxy named[2827]: success resolving 'boden-de.resultspage.com/AAAA' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:34:50 proxy named[2827]: success resolving 'cdn.optimizely.com/AAAA' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:34:56 proxy named[2827]: success resolving 'cdn.syndication.twimg.com/A' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:35:21 proxy named[2827]: success resolving 'plus.google.com/A' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:35:25 proxy named[2827]: success resolving 'd.agkn.com/AAAA' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:35:47 proxy named[2827]: success resolving 'googleads.g.doubleclick.net/A' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:37:09 proxy named[2827]: success resolving 'e6858.dsce9.akamaiedge.net/A' (in '.'?) after disabling EDNS
Jun 8 10:40:43 proxy named[2827]: success resolving 'r1---sn-4g5e6nl7.gvt1.com/A' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
Jun 8 10:42:12 proxy named[2827]: success resolving 'tedbaker.tdefender.net/A' (in '.'?) after disabling EDNS
Jun 8 10:42:14 proxy named[2827]: success resolving 'tile-service.weather.microsoft.com/A' (in '.'?) after disabling EDNS
Jun 8 10:42:34 proxy named[2827]: success resolving 'e5886.x.akamaiedge.net/A' (in '.'?) after disabling EDNS
Jun 8 10:42:41 proxy named[2827]: success resolving 'i.salecycle.com/AAAA' (in '.'?) after disabling EDNS
Jun 8 10:42:48 proxy named[2827]: success resolving 's.mopub.com/A' (in '.'?) after disabling EDNS
Jun 8 10:42:53 proxy named[2827]: success resolving 'postback.pointwise.co/A' (in '.'?) after disabling EDNS
Jun 8 10:43:22 proxy named[2827]: success resolving 'detectportal.firefox.com/AAAA' (in '.'?) after disabling EDNS
Jun 8 10:43:31 proxy named[2827]: success resolving 'www.evi.com/A' (in '.'?) after disabling EDNS
Jun 8 10:43:34 proxy named[2827]: success resolving 'tg.symcd.com/AAAA' (in '.'?) after disabling EDNS
Jun 8 10:43:41 proxy named[2827]: success resolving 'googleads4.g.doubleclick.net/A' (in '.'?) after disabling EDNS
Jun 8 10:43:41 proxy named[2827]: success resolving 'googleads4.g.doubleclick.net/AAAA' (in '.'?) after disabling EDNS
Jun 8 10:43:42 proxy named[2827]: success resolving './NS' (in '.'?) after disabling EDNS
Jun 8 10:43:55 proxy named[2827]: success resolving 'ping.avast.com/A' (in '.'?) after disabling EDNS
Jun 8 10:43:59 proxy named[2827]: success resolving 'm2932843.iavs9x.avg.u.avcdn.net/AAAA' (in '.'?) after disabling EDNS
Jun 8 10:44:22 proxy named[2827]: success resolving 'www.stylight.de/A' (in '.'?) after disabling EDNS
Jun 8 10:45:16 proxy named[2827]: success resolving './NS' (in '.'?) after disabling EDNS
Jun 8 10:45:21 proxy named[2827]: success resolving 'www.ist-track.com/A' (in '.'?) after disabling EDNS
Jun 8 10:46:30 proxy named[2827]: success resolving './NS' (in '.'?) after disabling EDNS
Jun 8 10:46:39 proxy named[2827]: success resolving 'ocsp-ds.ws.symantec.com.edgekey.net/A' (in '.'?) after disabling EDNS
Jun 8 10:47:33 proxy named[2827]: success resolving 'download.cdn.mozilla.net/AAAA' (in '.'?) after reducing the advertised EDNS UDP packet size to 512 octets
I have been looking around the net for a solution. For example, this nice page: Why does BIND log messages about disabling EDNS or reducing the advertised packet size?
- I have set the packet size to 512, but then I get also the "disabling EDNS" problem.
- I have found that it is possible to disable EDNS in general, but this is for sure not the solution.
- I tested with different nameservers.
- I also tested with a different dns service (dnsmasq)
With the reply size test, I have found also that the destination DNS servers support a size of 4096. But sometimes the size gets reduced.
rst.x1008.rs.dns-oarc.net.
rst.x1968.x1008.rs.dns-oarc.net.
rst.x2454.x1968.x1008.rs.dns-oarc.net.
"74.125.73.76 DNS reply size limit is at least 2454"
"74.125.73.76 sent EDNS buffer size 4096"
"Tested at 2017-06-08 09:07:07 UTC"
rst.x4090.rs.dns-oarc.net.
rst.x4058.x4090.rs.dns-oarc.net.
rst.x4064.x4058.x4090.rs.dns-oarc.net.
"74.125.47.151 DNS reply size limit is at least 4090"
"74.125.47.151 sent EDNS buffer size 4096"
"Tested at 2017-06-08 09:06:18 UTC"
rst.x1008.rs.dns-oarc.net.
rst.x1253.x1008.rs.dns-oarc.net.
rst.x1447.x1253.x1008.rs.dns-oarc.net.
"2a00:1450:400c:c02::103 DNS reply size limit is at least 1447"
"2a00:1450:400c:c02::103 sent EDNS buffer size 4096"
"Tested at 2017-06-08 09:06:22 UTC"
rst.x4090.rs.dns-oarc.net.
rst.x4058.x4090.rs.dns-oarc.net.
rst.x4064.x4058.x4090.rs.dns-oarc.net.
"74.125.47.139 DNS reply size limit is at least 4090"
"74.125.47.139 sent EDNS buffer size 4096"
"Tested at 2017-06-08 09:07:13 UTC"
rst.x1008.rs.dns-oarc.net.
rst.x1253.x1008.rs.dns-oarc.net.
rst.x1447.x1253.x1008.rs.dns-oarc.net.
"2a00:1450:400c:c02::103 DNS reply size limit is at least 1447"
"2a00:1450:400c:c02::103 sent EDNS buffer size 4096"
"Tested at 2017-06-08 09:06:22 UTC"
The problem now is that I can't find the root problem. The two machines which have the problem have different NICs (one Intel and the other one Broadcom) — so I don't think it is a driver problem.
One machine has a DSL connection and the other one has 2 gateways (cable and ethernet in a failover config). So all of them have no router in front (only the second, with the ethernet, but it is only the failover link, and it happens also on both links).
I have also made a pcap dump, and found some "TCP Retransmisson" and "TCP Spurious Retransmission" — but don't know whether they are the problem. And with tcpdump I can see a lot of "bad udp cksum" — but not on Wireshark.
You have to enable eDNS. This is the solution.