Home / server / Questions / 901125
Accepted
Lucio Crusca
Asked: 2018-03-12 13:24:57 +0800 CST

Nginx as reverse proxy not caching

  • 772

I'm trying to configure Nginx as a caching reverse proxy. The origin server is Apache and it hosts a WordPress instance, if that matters.

The reverse proxy functionality is working as expected, but the cache does not seem to work. If I get the same static resource twice in a row, I get x-proxy-cache: MISS twice.

assodigitale.it is the domain, 138.201.87.123 the origin server IP address and 138.201.87.124 the Nginx proxy IP address.

The origin server seems to reply allowing proxy to cache the resource:

$ curl --connect-to ::138.201.87.123:443 --http2 -I https://assodigitale.it/wp-content/uploads/2018/03/aereo.jpg
HTTP/2 200 
date: Sun, 11 Mar 2018 20:59:39 GMT
server: Apache/2.4.25 (Debian)
content-length: 32989
strict-transport-security: max-age=31536000; includeSubdomains; preload
last-modified: Wed, 07 Mar 2018 09:34:41 GMT
etag: "80dd-566cf44ca2952"
accept-ranges: bytes
vary: Accept-Encoding
cache-control: max-age=1209600, public
x-content-type-options: nosniff
content-type: image/jpeg

The first request to the proxy server results in a MISS, as expected:

$ curl --connect-to ::138.201.87.124:443 --http2 -I https://assodigitale.it/wp-content/uploads/2018/03/aereo.jpg
HTTP/2 200 
server: nginx/1.13.9
date: Sun, 11 Mar 2018 21:04:00 GMT
content-type: image/jpeg
content-length: 32989
strict-transport-security: max-age=31536000; includeSubdomains; preload
last-modified: Wed, 07 Mar 2018 09:34:41 GMT
etag: "80dd-566cf44ca2952"
vary: Accept-Encoding
cache-control: max-age=1209600, public
x-content-type-options: nosniff
x-proxy-cache: MISS
strict-transport-security: max-age=4838400; includeSubDomains; preload
accept-ranges: bytes

The second request to the Nginx proxy shoud result in a HIT, but it results in another MISS:

$ curl --connect-to ::138.201.87.124:443 --http2 -I https://assodigitale.it/wp-content/uploads/2018/03/aereo.jpg
HTTP/2 200 
server: nginx/1.13.9
date: Sun, 11 Mar 2018 21:05:52 GMT
content-type: image/jpeg
content-length: 32989
strict-transport-security: max-age=31536000; includeSubdomains; preload
last-modified: Wed, 07 Mar 2018 09:34:41 GMT
etag: "80dd-566cf44ca2952"
vary: Accept-Encoding
cache-control: max-age=1209600, public
x-content-type-options: nosniff
x-proxy-cache: MISS
strict-transport-security: max-age=4838400; includeSubDomains; preload
accept-ranges: bytes

Here is the relevant part of my nginx config:

proxy_cache_path /srv/cache/nginx levels=1:2 keys_zone=revproxy:2000m inactive=2880m use_temp_path=off;
proxy_cache_key "$scheme$request_method$host$request_uri";
proxy_cache_methods GET HEAD;
proxy_cache_valid any 1m;
proxy_cache_valid 200 1440m;

server {
    listen 443 ssl http2;
    ssl on;
    server_name assodigitale.it;

    ssl_certificate /etc/letsencrypt/live/assodigitale.it/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/assodigitale.it/privkey.pem;

    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;

    ssl_protocols TLSv1.2;
    ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
    ssl_prefer_server_ciphers on;

    location / {
        proxy_cache revproxy;
        add_header X-Proxy-Cache $upstream_cache_status;
        add_header Strict-Transport-Security "max-age=4838400; includeSubDomains; preload";

        proxy_pass  https://138.201.87.123;
        proxy_cache_use_stale updating error timeout http_500 http_502 http_503 http_504;
        proxy_cache_bypass $http_x_forceflushcacheurl;
        proxy_cache_lock on;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_hide_header Upgrade;
        proxy_buffering off;
        proxy_connect_timeout       600;
        proxy_send_timeout          600;
        proxy_read_timeout          600;
        send_timeout                600;
        proxy_ignore_headers Set-Cookie;

        http2_push_preload on;
        client_max_body_size 64M;
    }
}

The /srv/cache/nginx directory has 755 permissions and www-data owner, and Nginx runs as www-data. In fact Nginx does write its folders in there, namely 0 1 2 3 4 5 6 7 8 9 a b c d e f, but the overall space taken is now 344Kb, for a site that is quite large and that has much more than casual traffic.

Trying the same curl commands above, but with pages instead of images, yelds just the same result, it's always a MISS.

Why is Nginx refusing to cache resources?

nginx

3 Answers

  1. Best Answer

    you should set proxy_buffering on, otherwise nginx will not to cache the response!

    the official document said:

    When buffering is disabled, the response is passed to a client synchronously, immediately as it is received. nginx will not try to read the whole response from the proxied server. The maximum size of the data that nginx can receive from the server at a time is set by the proxy_buffer_size directive.

    • 5

  2. I've copied the configuration from another similar Nginx proxy I run, adapted it to the website and it now works.

    This is the configuration I'm currently using:

    proxy_cache_path /srv/cache/nginx levels=1:2 keys_zone=revproxy:2000m inactive=2880m use_temp_path=off;
proxy_cache_key "$scheme$request_method$host$request_uri";
proxy_cache_methods GET HEAD;
proxy_cache_valid any 1m;
proxy_cache_valid 200 1440m;    
server {
    listen 443 ssl http2;
    ssl on;
    server_name assodigitale.it;

    ssl_certificate /etc/letsencrypt/live/assodigitale.it/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/assodigitale.it/privkey.pem;

        ssl_protocols        SSLv3 TLSv1 TLSv1.1 TLSv1.2;    
        ssl_ciphers RC4:HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers on;
        keepalive_timeout    60;
        ssl_session_timeout  10m;

    location / {
            proxy_cache revproxy;
            add_header X-Proxy-Cache $upstream_cache_status;

            proxy_pass  https://138.201.87.123;
            proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
            proxy_cache_use_stale updating error timeout http_500 http_502 http_503 http_504;
            proxy_cache_lock on;
            proxy_set_header        Host            $host;
            proxy_set_header        X-Real-IP       $remote_addr;
            proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_hide_header      Upgrade;
            proxy_buffering on;
            proxy_connect_timeout       600;
            proxy_send_timeout          600;
            proxy_read_timeout          600;
            send_timeout                600;
            client_max_body_size 64M;
    }
}

    While this can be an answer because it solves the problem, I don't understand why it works (or, more precisely, why the previous configuration didn't), so I'm not going to accept my own answer.

    Maybe someone can spot the particular difference in the two configurations that makes the cache work: that would be an acceptable answer.

    • 1

  3. We had a similar problem. We were using nginx as a proxy cache for an s3 bucket (so we could support IP whitelisting). We noticed that we weren't sourcing form an https endpoint even though our proxy was serving on https. Seems like nginx didn't like that. Once we switched from http to https, no problems.

    • 1