Home / server / Questions / 947718
In Process
Antony Gibbs
Asked: 2019-01-06 08:04:52 +0800 CST

Varnish greedy buffering

  • 772

I have a setup with nginx proxy (ssl) to varnish (5.2) to apache on a debian linux.

varnish passing response without buffering to nginx (not buffering either)

sub vcl_backend_response {
  if ( std.integer(beresp.http.Content-Length, 0) > 10000 ) {
    set beresp.do_stream = true;
  }
}

If one downloads a big file (Ex: 8G), download start instantly (so stream seams to work) but varnish ram usage keeps growing.

Seems like varnish pulls form apache as fast as possible (faster then the output to the outer world) and steals all the ram needed.

If I get nginx to proxy directly to apache I don't have this issue.

Is there a way to tell varnish to limit it's buffering in this case?

Am I missing out something?

UPDATE

$ curl -i https://example.com/test.dmg | head

HTTP/1.1 200 OK
Server: nginx
Date: Sat, 05 Jan 2019 21:26:43 GMT
Content-Type: application/x-apple-diskimage
Content-Length: 8743924593
Connection: keep-alive
Last-Modified: Wed, 28 May 2014 16:18:56 GMT
Accept-Ranges: bytes
Age: 0
X-Cache: MISS
Strict-Transport-Security: max-age=15768000; includeSubDomains; preload
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
varnish

1 Answers

  1. Despite your VCL and apparently working stream, your backend likely does not send Content-Length header.

    You may want to ensure it does. Check here.

    • 0