Home / user-94486

kaji's questions

Martin Hope
kaji
Asked: 2012-05-23 00:11:33 +0800 CST
  • 2

I was trying to mimic this logstash.net/docs/1.1.0/tutorials/metrics-from-logs

I have following setup

nginx(app server) ==sends the increment==>Etsy statsD=====>Graphite

This setup is working fine since graph is being plotted just fine but not correctly. According to above configuration, I understand the graph step value should be integer but I am getting floating points numbers in Y axis(hits) as per the attached graph and I am completely baffled as to how do i interpret this graph.

Logstash agent configuration

input {
  file {
    type => nginx_web
    path => ["/var/log/nginx/access.log"]
  }

filter {
 grok {
   type => nginx_web
   pattern => "%{IP:ClientIP} (?:%{HOST:ClientHost}|-) (?:%{USER:ClientUser}|-) \[%{GREEDYDATA:TimeStamp}\] \"(?:%{WORD:Verb} %{URIPATHPARAM:Request} HTTP/%{NUMBER:HTTPversion}|%{DATA:UnparsedRQ})\" %{NUMBER:Response} (?:%{NUMBER:Bytes}|0) (?:%{QUOTEDSTRING:HTTPReferrer|\"-\"}) %{QUOTEDSTRING:HTTPUserAgent}"
 }

output {

 statsd {
        type => "nginx_web"
        host => "X.X.X.X"
        increment => [
                        "nginx.response.%{Response}"
                     ]
    }
}

I am using following URL API to generate this graph

render/?width=600&height=320&hideLegend=1&from=-60minutes&until=-0minutes&target=stats.logstash.*.nginx_web.response.*

enter image description here

Can somebody point me in right direction as to where to go to remedy my issue ?

graphite logstash statsd
Martin Hope
kaji
Asked: 2012-02-21 23:23:02 +0800 CST
  • 0

I have following configuration in my linode slice

NB <----->web1+web2+web3 Here's the configuration screenshot from linode: enter image description here

We need NB in http mode since we need few things that plain tcp mode does not give.

Now my problem is : when i run something like webpagetest.org for benchmarking purpose, i always get message that my domain is running static assets without keepalives. In TCP mode, keepalive is fine.

I understand haproxy is working as nodebalancer in linode. I just wanted to know "is this a known limitation of NB/haproxy or am i doing anything wrong ????"

keepalive haproxy linode
Martin Hope
kaji
Asked: 2011-09-13 12:27:19 +0800 CST
  • 1

We have webserver setup as follows

Loadbalancer (HAPROXY) <---> Nginx <---> App servers

To make Nginx aware of real clients that are making connections , I compiled Nginx with HttpRealIpModule and did the configuration changes. HAPROXY is also configured to forward XFF header. Nginx is getting right at least partially( my observation) if the client is browsing with pubic IP address. In case of clients which has private IP and behind another client side proxy like SQUID, I am getting the private IP of a client rather than the PUBLIC IP from which the private IP is being NATTED. According to Nginx Module documentation[ http://wiki.nginx.org/HttpRealIpModule ], HttpRealIpModule should be getting last ip in the X-Forwarded-For header for replacement. Here's the NGREP dump for a client with private IP behind squid proxy

T 192.168.255.1:53777 -> 192.168.153.164:80 [AP]
.1315664739.21; __utmb=82961925.3.10.1315664739; __utmc=82961925;         __utmz=82961925.1315664739.21.21.utmcsr=domainname.com|utmccn=        (referral)|utmcmd=referral|utmcct=/users/sign_in; SnapABugHistory=206
#..If-None-Match: "cc1ff315751caad21aea3dfd5392a56a"..Via: 1.1 atmosphere.sprout.lan:3128    (squid/2.7.STABLE9)..X-Forwarded-For: 192.168.3.111..Cache-Control: max-age=259200..X-Forwarded-For:    116.90.235.11....

XFF header here is atmosphere.sprout.lan:3128 (squid/2.7.STABLE9)..X-Forwarded-For: 192.168.3.111..Cache-Control: max-age=259200..X-Forwarded-For: 116.90.235.11.

The last ip seen here in the header is 116.90.235.11 yet the log file of nginx shows the client's private ip 192.168.2.111

is this normal ??

nginx haproxy