Is there a way to get the total size (in bytes) of the request header that the browser is sending to the server for each request?
As far as I can see Nginx supports by default 2 log files: error_log (tracks issues related to the Nginx server itself) and access_log (tracks requests processed by Nginx). Whilst it is possible to control the format of access_log with the log_format directive, I have been unsuccessful at finding a way of logging only certain requests to a separate file, and therefore would like to ask the question on SF as a reference for future readers:
Is there a way to log certain requests to a different log file than the one defined by access_log?
FYI the reason behind this question is that I have a rule that denies access to unwanted crawlers with a 200 (because 403 would give them a hint that they're being blocked), and filtering those requests out of the access_log becomes more difficult.
I using WAMPServer on my computer for testing and development. I forgot and left it online for a few days and I notice a bunch of random requests that aren't even from my IP. Here are some examples.
77.73.69.127 - - [29/Apr/2012:08:22:20 -0700] "HEAD /manager/html HTTP/1.0" 200 -
58.218.199.250 - - [29/Apr/2012:08:31:54 -0700] "GET http://www.verysurf.com/proxyheader.php HTTP/1.1" 404 213
58.218.199.147 - - [29/Apr/2012:08:35:37 -0700] "GET http://www.travelimgusa.com/ip.php HTTP/1.1" 200 1340
58.218.199.250 - - [29/Apr/2012:10:03:53 -0700] "GET http://61.152.144.145/judge.php HTTP/1.1" 200 1355
58.218.199.227 - - [29/Apr/2012:12:04:07 -0700] "GET http://59.53.91.9/proxy/judge.php HTTP/1.1" 200 1335
58.218.199.250 - - [29/Apr/2012:13:08:29 -0700] "GET http://59.53.91.9/proxy/judge.php HTTP/1.1" 404 213
58.218.199.250 - - [29/Apr/2012:13:08:29 -0700] "GET http://59.53.91.9/proxy/judge.php HTTP/1.1" 200 1335
A lot of them are from this 58.218.199.250 IP.
Another IP I noticed has been trying to access my database manager.
200.196.48.40 - - [28/Apr/2012:16:12:32 -0700] "GET /index.php HTTP/1.1" 200 4599
200.196.48.40 - - [28/Apr/2012:16:12:33 -0700] "GET /admin/index.php HTTP/1.1" 404 213
200.196.48.40 - - [28/Apr/2012:16:12:33 -0700] "GET /admin/pma/index.php HTTP/1.1" 404 217
200.196.48.40 - - [28/Apr/2012:16:12:34 -0700] "GET /admin/phpmyadmin/index.php HTTP/1.1" 404 224
200.196.48.40 - - [28/Apr/2012:16:12:34 -0700] "GET /db/index.php HTTP/1.1" 404 210
200.196.48.40 - - [28/Apr/2012:16:12:35 -0700] "GET /dbadmin/index.php HTTP/1.1" 404 215
200.196.48.40 - - [28/Apr/2012:16:12:35 -0700] "GET /myadmin/index.php HTTP/1.1" 404 215
200.196.48.40 - - [28/Apr/2012:16:12:35 -0700] "GET /mysql/index.php HTTP/1.1" 404 213
200.196.48.40 - - [28/Apr/2012:16:12:36 -0700] "GET /mysqladmin/index.php HTTP/1.1" 404 218
200.196.48.40 - - [28/Apr/2012:16:12:36 -0700] "GET /typo3/phpmyadmin/index.php HTTP/1.1" 404 224
200.196.48.40 - - [28/Apr/2012:16:12:37 -0700] "GET /phpadmin/index.php HTTP/1.1" 404 216
200.196.48.40 - - [28/Apr/2012:16:12:37 -0700] "GET /phpMyAdmin/index.php HTTP/1.1" 404 218
200.196.48.40 - - [28/Apr/2012:16:12:38 -0700] "GET /phpmyadmin/index.php HTTP/1.1" 403 222
And that's all this IP was doing. Well it returned a 404 since the permissions is local only. And of course all these IPs are from Brazil, China, and Russia... Should I be worried about these random requests or is it normal? Are these bots or crawlers?
I know that you can use nginx HttpGzipModule to gzip responses. Can you als use it to enable nginx to handle gzip-compressed requests, i.e. ones with the request header: Content-Encoding: gzip ?
Note: Apache, with the help of mod_deflate can handle gzip-compressed requests with the directive SetInputFilter DEFLATE.
Related
I installed nginx on Ubuntu Hardy and immediately realized that the PHP scripts were getting empty request arrays, ie whether GET or POST nothing was coming through, even the $_REQUEST array was empty.
When I switch back to Apache it was all fine.
The installation nginx version is 0.6.35-0ubuntu1.1~hardy1 and PHP is PHP Version 5.2.4-2ubuntu5.10.
What could be wrong?