Home / server / Questions / 246764
Accepted
Alexander Gladysh
Asked: 2011-03-13 17:05:18 +0800 CST

nginx: dump HTTP requests for debugging

  • 772
  • Ubuntu 10.04.2
  • nginx 0.7.65

I see some weird HTTP requests coming to my nginx server.

To better understand what is going on, I want to dump whole HTTP request data for such queries. (I.e. dump all request headers and body somewhere I can read them.)

Can I do this with nginx? Alternatively, is there some HTTP server that allows me to do this out of the box, to which I can proxy these requests by the means of nginx?

Update: Note that this box has a bunch of normal traffic, and I would like to avoid capturing all of it on low level (say, with tcpdump) and filtering it out later.

I think it would be much easier to filter good traffic first in a rewrite rule (fortunately I can write one quite easily in this case), and then deal with bogus traffic only.

And I do not want to channel bogus traffic to another box just to be able to capture it there with tcpdump.

Update 2: To give a bit more details, bogus request have parameter named (say) foo in their GET query (the value of the parameter can differ). Good requests are guaranteed not to have this parameter ever.

If I can filter by this in tcpdump or ngrep somehow — no problem, I'll use these.

ubuntu http nginx

3 Answers

  1. Best Answer

    Adjust the number of pre/post lines (-B and -A args) as needed:

    tcpdump -n -S -s 0 -A 'tcp dst port 80' | grep -B3 -A10 "GET /url"

    This lets you get the HTTP requests you want, on the box, without generating a huge PCAP file that you have to offload somewhere else.

    Keep in mind, that the BPF filter is never exact, if there are a large number of packets flowing through any box, BPF can and will drop packets.

    • 32

  2. I don't know exactly what you mean with dump the request but you can use tcpdump and/or wireshark to analyze the data:

    # tcpdump port 80 -s 0 -w capture.cap

    And you can use wireshark to open the file and see the conversation between servers.

    • 5

  3. If you proxy the requests to Apache with mod_php installed you can use the following PHP script to dump the requests:

    <?php
$pid = getmypid();
$now = date('M d H:i:s');
$fp = fopen('/tmp/intrusion.log', 'a');

if (!function_exists('getallheaders')) 
{ 
    function getallheaders() 
    { 
           $headers = ''; 
       foreach ($_SERVER as $name => $value) 
       { 
           if (substr($name, 0, 5) == 'HTTP_') 
           { 
               $headers[str_replace(' ', '-', ucwords(strtolower(str_replace('_', ' ', substr($name, 5)))))] = $value; 
           } 
       } 
       return $headers; 
    } 
} 

function ulog ($str) {
    global $pid, $now, $fp;
    fwrite($fp, "$now $pid {$_SERVER['REMOTE_ADDR']} $str\n");
}

foreach (getallheaders() as $h => $v) {
    ulog("H $h: $v");
}
foreach ($_GET as $h => $v) {
    ulog("G $h: $v");
}
foreach ($_POST as $h => $v) {
    ulog("P $h: $v");
}
fclose($fp);

    Note that since you're using nginx the $_SERVER['REMOTE_ADDR'] may be pointless. You'll have to pass the real IP to Apache via proxy_set_header X-Real-IP $remote_addr;, and you can use that instead (or just rely on it being logged via getallheaders()).

    • 0