So my coworker set us up a fancy new caching proxy system and then promptly went on vacation. Now I'm getting complaints from our developers/designers that many static resources are being cached for far longer than any of the configuration I've seen dictates.
For example a certain logo file has been changed as of the 13th, but the version from the 9th is still being returned, despite the setting: proxy_cache_valid 200 1h;
which should only cache it for 1 hour.
As far as I can see the upstream server is giving Nginx the header Expires: Sat, 14 Feb 2015 19:33:58 GMT
and the cache expiry is just running with that regardless of the fact that the Last-Modified:
header has changed. I've had a peek at the upstream server's logs and the proxy does not make any attempt to check the status of the file.
How can I get Nginx to check for updated content?
The response headers from the cache:
# curl -v -XHEAD 'http://foo.company.com/inc/skins/pt-1r/schemes/default/img/logo.png'
* About to connect() to foo.company.com port 80 (#0)
* Trying 1.2.3.4... connected
* Connected to foo.company.com (1.2.3.4) port 80 (#0)
> HEAD /inc/skins/pt-1r/schemes/default/img/logo.png HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.16.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: foo.company.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.0.15
< Date: Thu, 15 Jan 2015 19:37:19 GMT
< Content-Type: image/png
< Connection: keep-alive
< Last-Modified: Fri, 09 Jan 2015 00:04:42 GMT
< Content-Length: 19198
< Cache-Control: max-age=2592000, public, must-revalidate, proxy-revalidate
< Expires: Thu, 12 Feb 2015 22:54:22 GMT
< Vary: User-Agent
< Content-Language: en
< X-Cache-Status: HIT
< Accept-Ranges: bytes
As opposed to directly from the server:
# curl -v --header "Host: foo.company.com" -XHEAD http://10.1.2.3/inc/skins/pt-1r/schemes/default/img/logo.png
* About to connect() to 10.1.2.3 port 80 (#0)
* Trying 10.1.2.3... connected
* Connected to 10.1.2.3 (10.1.2.3) port 80 (#0)
> HEAD /inc/skins/pt-1r/schemes/default/img/logo.png HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.16.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Accept: */*
> Host: foo.company.com
>
< HTTP/1.1 200 OK
< Date: Thu, 15 Jan 2015 19:33:58 GMT
< Server: Apache
< Last-Modified: Tue, 13 Jan 2015 23:04:44 GMT
< Accept-Ranges: bytes
< Content-Length: 45255
< Cache-Control: max-age=2592000, public, must-revalidate, proxy-revalidate
< Expires: Sat, 14 Feb 2015 19:33:58 GMT
< Vary: User-Agent
< Content-Type: image/png
< Content-Language: en
proxy.conf
# Store cached date here
proxy_cache_path /var/lib/nginx/cache levels=1:2 keys_zone=cache:128m inactive=1d max_size=1g;
# Use cache defined above
proxy_cache cache;
proxy_cache_key $scheme$host$request_uri;
# Only cache positive responses
proxy_cache_valid 200 1h;
proxy_cache_valid 301 302 5m;
# Temp path for when buffers overflow
proxy_temp_path /var/lib/nginx/temp;
# Buffer data (must be on to allow caching)
proxy_buffering on;
proxy_buffer_size 128k;
proxy_buffers 100 128k;
# Set some headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
# Die if backend takes too long to connect
proxy_connect_timeout 5;
# Allow adding abcnocache=1 to URLs to skip the cache
proxy_cache_bypass $arg_abcnocache;
# Add a header showing the cache status
add_header X-Cache-Status $upstream_cache_status;
the site's config:
server {
server_name foo.company.com *.foo.company.com ;
listen 80;
access_log /var/log/nginx/foo.company.com-access.log;
error_log /var/log/nginx/foo.company.com-error.log;
location / {
proxy_pass http://10.1.2.3;
proxy_redirect default;
}
}
nginx.conf for good measure:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
gzip on;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_proxied any;
gzip_vary off;
gzip_types text/plain text/css application/x-javascript text/xml application/xml application/rss+xml application/atom+xml text/javascript application/javascript application/json text/mathml;
gzip_min_length 1000;
gzip_disable "MSIE [1-6]\.";
server_names_hash_bucket_size 64;
types_hash_max_size 2048;
types_hash_bucket_size 64;
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
Your backend responds with
So does the original answer, and this overrides the
proxy_cache_valid
setting:So all the nginx does - it runs with the cached copy of the object, because your backend said it's valid.
must-revalidate
andproxy-revalidate
does nothing when the cache entry is considered valid. And yours is. So you should really redirect this complaint back to developers.