Home / server / Questions / 638910
Accepted
Nathan Osman
Asked: 2014-10-23 10:10:22 +0800 CST

Nginx silently dropping header lines that exceed 1128 bytes

  • 772

I have Nginx set up as a reverse proxy to a uWSGI application (a Django app). Due to the nature of the content that the Django app is hosting, sometimes the URLs get long. Really long. Because of this, the Referer HTTP header can also sometimes get long.

Here's the problem. Nginx drops the Referer received from the client if it exceeds 1128 bytes (discovered by trial and error). Anything 1128 bytes or smaller gets passed to the upstream server. Everything larger gets dropped. This is a problem because Django's CSRF protection mechanism requires an intact Referer.

Here's the relevant section of my Nginx config. file:

location / {
    proxy_pass http://127.0.0.1:8000;
    proxy_set_header Host              $host;
    proxy_set_header Referer           $http_referer;
    proxy_set_header X-Real-IP         $remote_addr;
    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

I looked through the Nginx documentation and couldn't find a setting that controlled the maximum length for client headers. Frustrated, I even began pouring over the source code for Nginx, which didn't turn anything useful up either.

What do I need to do to get Nginx to pass Referer unmodified to the upstream server?

Version Information

  • Ubuntu 14.04 Server 64-bit
  • nginx/1.4.6 (Ubuntu)
  • uWSGI 1.9.17.1-debian
  • Django 1.7.0
nginx

2 Answers

  1. Best Answer

    Xavier's answer and the ensuing discussion led me to the real cause of this issue: uWSGI. It appears that uWSGI was stripping the header:

    [WARNING] unable to add HTTP_REFERER

    The solution was painfully simple - when launching uWSGI, I needed to simply include the following option:

    --buffer-size 8192

    All of the headers make it through to Django now and CSRF verification succeeds.

    • 4

  2. Nginx header read size is controlled by two directives :

    • client_header_buffer_size [buffer_size] which is the default buffer that would fit in most cases

    • large_client_header_buffers [count] [buffer_size] which are additionnal buffers allocated on demand if it doesn't fit in client_header_buffer_size buffer. You must tailor this last directive to your case and make sure than buffer_size is greater than any line passed in the request (headers + URL with HTTP method and HTTP version).

    • 1