Home / server / Questions / 335359
Accepted
Daniel Quinn
Asked: 2011-11-29 00:29:02 +0800 CST

How is it possible that I can do a host lookup but not a curl?

  • 772

Has anyone ever seen this before? Note that this happens not only with google.com, but with every domain I try. It's a wireless connection (WEP), but I'm not sure how that would be relevant:

$ curl -v google.com
# This takes about 60s to return
* getaddrinfo(3) failed for google.com:80
* Couldn't resolve host 'google.com'
* Closing connection #0
curl: (6) Couldn't resolve host 'google.com'

$ wget google.com
--2011-11-28 14:44:08--  http://google.com/
Resolving google.com... failed: Name or service not known.
wget: unable to resolve host address `google.com'

$ ping google.com
PING google.com (209.85.148.147) 56(84) bytes of data.
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=2 ttl=54 time=136 ms
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=3 ttl=54 time=34.0 ms
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=4 ttl=54 time=34.3 ms
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=5 ttl=54 time=42.5 ms
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=6 ttl=54 time=44.7 ms
64 bytes from fra07s07-in-f147.1e100.net (209.85.148.147): icmp_req=7 ttl=54 time=34.5 ms
^C
--- google.com ping statistics ---
8 packets transmitted, 6 received, 25% packet loss, time 7007ms
rtt min/avg/max/mdev = 34.063/54.376/136.026/36.758 ms


$ host google.com
google.com has address 209.85.148.106
google.com has address 209.85.148.147
google.com has address 209.85.148.99
google.com has address 209.85.148.103
google.com has address 209.85.148.104
google.com has address 209.85.148.105
google.com mail is handled by 30 alt2.aspmx.l.google.com.
google.com mail is handled by 40 alt3.aspmx.l.google.com.
google.com mail is handled by 50 alt4.aspmx.l.google.com.
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.

$ host google.com 192.168.1.201
Using domain server:
Name: 192.168.1.201
Address: 192.168.1.201#53
Aliases: 

google.com has address 209.85.148.103
google.com has address 209.85.148.104
google.com has address 209.85.148.105
google.com has address 209.85.148.106
google.com has address 209.85.148.147
google.com has address 209.85.148.99
google.com mail is handled by 40 alt3.aspmx.l.google.com.
google.com mail is handled by 50 alt4.aspmx.l.google.com.
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.
google.com mail is handled by 30 alt2.aspmx.l.google.com.

$ cat /etc/resolv.conf 
# Generated by NetworkManager
nameserver 192.168.1.201

$ cat /etc/hosts
127.0.0.1       localhost
::1             localhost

$ netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.1.254   0.0.0.0         UG        0 0          0 wlan0
127.0.0.0       127.0.0.1       255.0.0.0       UG        0 0          0 lo
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 wlan0

Basically any application, including Firefox, can't work to do name lookups. What's more, if I take the wifi offline and plug in an ethernet cable, everything is fine.

domain-name-system wifi curl

8 Answers

  1. Using this: https://www.centos.org/modules/newbb/viewtopic.php?topic_id=39343

    I found a key command that helped me troubleshoot:

    [root@localhost ~]# wget -6 URL Failed
[root@localhost ~]# wget -4 URL Worked

    It's something to do with the default ipv6 stack that's causing problems with certain utils. Disable ipv6 to resolve.

    • 10

  2. Check your /etc/nsswitch.conf. If the hosts line says something like

    hosts:      files dns

    I'm as confused as you. But if it says something like

    hosts:      files

    then the fact that DNS is working (see output of host command) won't help curl, which is doing name resolution via the standard OS libraries, which have been told not to use the DNS.

    • 8
  3. Best Answer

    Perhaps you have some very weird and restrictive SELinux (or grsecurity...) rules in place?

    If not, try strace -o /tmp/wtf -fF curl -v google.com and try to spot from /tmp/wtf output file what's going on.

    • 8

  4. I had the same problem - host, nslookup resolves ok, curl - can't on the same hostnames.

    After tcpdumping communication I found that curl tries to establish TCP (in addition to UDP) connection to DNS port, which was closed in my router. After tcp port 53 was enabled curl started to work flawlessly.

    Another strange thing is that this problem does not surface if dns server is regular bind installation. If I use embedded into router DNS server, curl suddenly tries to use TCP ports even if it already received (!) answer via UDP 2ms before. I suppose this is bug.

    • 5

  5. There could be an error in your /etc/resolv.conf file that nslookup tolerates but curl does not.

    The question asked was "How is it possible that I can do a host lookup but not a curl?"

    This is possible because curl uses getaddrinfo() to resolve the FQDN, whereas nslookup does not. Instead, I believe nslookup parses /etc/resolv.conf using some other function or library, or via its own custom code. I did not look at source code to verify this, but you can prove it by adding whitespace in front of the nameserver token in /etc/resolv.conf. nslookup can parse this but getaddrinfo() cannot.

    
Example /etc/resolv.conf
 nameserver 8.8.8.8

    If your resolv.conf has this error, or other errors that are tolerated by nslookup but not getaddrinfo(), then you can resolve an FQDN with nslookup, but you will not be able to use curl on that FQDN.

    Fix: as root, edit /etc/resolv.conf and remove any leading whitespace on the nameserver line.

    • 2

  6. If this occurs for anyone trying to set up DNS for an AWS EC2 instance, make sure to also enable the IPv6 rules (::/0) for HTTP and HTTPS in the security group used by that instance.

    • 2

  7. I had this same problem on my VE (running on my laptop) today, and found it quite surprising. Dig and NSlookup works, but curl fails.

    So for instance:

    # curl -v google.com
* getaddrinfo(3) failed for google.com:80
* Couldn't resolve host 'google.com'
* Closing connection #0
curl: (6) Couldn't resolve host 'google.com'

    But when I saw David T's post here, I decided to try it with curl. So while this fails:

    # curl  google.com -6
curl: (6) Couldn't resolve host 'google.com'

    This succeeds:

    # curl  google.com -4
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

    The -6 specifies that curl uses IPv6 and the -4, to use IPv4. I get the same errors when using wget, so definitely some problems with the IPv6 stack on the Host.

    All the other modifications to the nsswitch.conf file and other BIND conf files did not help as the problem is not with this utilities.

    • 1

  8. Was your curl installation smooth? If possible try re-installing curl.

    Try curl -v google.com to get more verbose output for debugging.

    e.g.:

    curl -v dnserror.test
* getaddrinfo(3) failed for dnserror.test:80
* Couldn't resolve host 'dnserror.test'
* Closing connection #0
curl: (6) Couldn't resolve host 'dnserror.test'

    Are you getting similar output?

    • 0