Home / server / Questions / 908141
In Process
Willem
Asked: 2018-04-18 14:41:30 +0800 CST

Docker pull: TLS handshake timeout

  • 772

I get this consistenly (Ubuntu 16.04 LTS):

$ docker pull nginx
Using default tag: latest
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: TLS handshake timeout

However curl TLS works fine (apart from the auth error):

$ curl https://registry-1.docker.io/v2/
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}

And even a small golang program (to mimic docker) works fine:

package main
import (
    "fmt"
    "io/ioutil"
    "net/http"
)
func main() {
    resp, err := http.Get("https://registry-1.docker.io/v2/")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        panic(err)
    }
    fmt.Println("Got: ", string(body))
}

The pcap for the docker TLS timeout request:

reading from file docker-timeout.pcap, link-type LINUX_SLL (Linux cooked)
00:38:54.782452 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [S], seq 26945613, win 29200, options [mss 1460,sackOK,TS val 1609360 ecr 0,nop,wscale 7], length 0
00:38:54.878630 IP registry-1.docker.io.https > my-ubuntu.52036: Flags [S.], seq 2700732154, ack 26945614, win 26847, options [mss 1460,sackOK,TS val 947941366 ecr 1609360,nop,wscale 8], length 0
00:38:54.878691 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [.], ack 1, win 229, options [nop,nop,TS val 1609384 ecr 947941366], length 0
00:38:54.878892 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1609384 ecr 947941366], length 155
00:38:55.175931 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1609459 ecr 947941366], length 155
00:38:55.475954 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1609534 ecr 947941366], length 155
00:38:56.076327 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1609684 ecr 947941366], length 155
00:38:57.280103 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1609985 ecr 947941366], length 155
00:38:59.684095 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1610586 ecr 947941366], length 155
00:39:04.492102 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1611788 ecr 947941366], length 155
00:39:04.879468 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [F.], seq 156, ack 1, win 229, options [nop,nop,TS val 1611884 ecr 947941366], length 0
00:39:04.976015 IP registry-1.docker.io.https > my-ubuntu.52036: Flags [.], ack 1, win 105, options [nop,nop,TS val 947943890 ecr 1609384,nop,nop,sack 1 {156:157}], length 0
00:39:04.976073 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1611909 ecr 947943890], length 155
00:39:05.275922 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1611984 ecr 947943890], length 155
00:39:05.876104 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1612134 ecr 947943890], length 155

What could possibly be going wrong?

linux-networking

10 Answers

  1. net/http: TLS handshake timeout means that you have slow internet connection. Default value of connection timeout is too small for your environment. Unfortunately docker don't have any settings that allows you change connection timeout. You may try to create your own registry cache somewhere else and pull images from it.

    • 34

  2. I experience the same problem. Then answer of Azamat Hackimov pointed me in the right direction. My machine is somewhat slow, especially at boot time, when I want to launch the service. Therefore the short timeout kicks in and kills my request.

    This is my workaround:

    docker pull $IMAGE || docker pull $IMAGE ||  docker pull $IMAGE || docker pull $IMAGE

    Simply hammer the server with request. Usually the second one is successful for me.

    • 19

  3. If you are using a private registry, you need to place the certificate for that under:

    /etc/docker/certs.d/registryname/ca.crt

    registryname will change accordingly.

    Also, please change your MTU size to 1300, this was also one thing I did to resolve the error. Registry one I believe you might have already done.

    Command for MTU change:

    ip link set dev eth0 mtu 1300

    MTU size is important to check to avoid this error if your internet speed is really good.

    • 8

  4. I had an equal Problem, by using docker run hello-world 1st time, which results in downloading an image using https://registry-1.docker.io/v2/, which end to 

    docker: Error response from daemon: Get https://registry-1.docker.io/v2/: proxyconnect tcp: net/http: TLS handshake timeout.

    Searching the web for hours and found out, that this happens at some users with ubuntu 18.04 and the current docker release, behind an proxy. A workaround is to remove all the https-proxy configuration in order to leave only the http-proxy configuration, to force an http (not https) download.

    Don't know, what the real reason is.

    (by the way: I had an equal "TLS handshake" problem with composer and packagist. This was because of an missing cacert.pem file, which was not provided by ubuntu by default. Maybe this docker-problem is going into the same direction?)

    • 7

  5. In my case my server was behind the nat and proxy and set to auto detect proxy what i have done on current terminal i have export proxy settings 

    root@k8master:~/runner# export http_proxy="http://192.168.10.208:3128"
root@k8master:~/runner# docker pull gitlab/gitlab-runner:latest
latest: Pulling from gitlab/gitlab-runner
7b722c1070cd: Pull complete 
5fbf74db61f1: Pull complete 
ed41cb72e5c9: Pull complete 
7ea47a67709e: Pull complete 
ae336ceeca88: Pull complete 
f9f79780e6cf: Pull complete 
67e622273f37: Pull complete 
bc84c40af701: Pull complete 
69e36092e9de: Pull complete 
Digest: sha256:b1f5387942aaaf8c220f6613a1e96ba2cbcb6c58a5e47ca0df8ae3216720a15e
Status: Downloaded newer image for gitlab/gitlab-runner:latest
    • 5

  6. You can get the TLS handshake timeout error if your docker daemon proxy is not configured correctly.

    # verify docker daemon proxy configuration
/etc/systemd/system/docker.service.d/proxy.conf

# flush changes
sudo systemctl daemon-reload

# restart docker service
sudo systemctl restart docker

    For more details, see https://docs.docker.com/config/daemon/systemd/#httphttps-proxy

    • 2

  7. None of above answers can solve my problem, however I found that below https://github.com/helm/helm/issues/5220 works for me!

    After that change the IT appartment of my company has found a solution. I used the https_proxy environment variable with https:// url to our proxy. This works for most of the tools we are using but not for helm or the newer kube. They seem to have some problems with the TLS handshake. We switched from https:// to a http:// url (e.g https_proxy=http://myproxy) and now everything works fine.

    • 1

  8. For my case, I simply changing the internet connection to different connection. Turns out my current internet service provider is kinda slow

    • 1

  9. When I switched the network connection to a faster one, the issue resolved.

    • 1

  10. What worked for me was to use a different network interface. Instead of connecting through ethernet (wired), I switched to wifi. Problem solved.

    I was on a fresh install of Raspbian Stretch, by the way.

    • 0