Mediocre Gopher's questions

We have a strange problem. Our remote server shows over 900 connections coming from our office over port 80:

# ss -n | grep ESTAB | grep $OFFICEIP | grep :80 | wc -l
935

Netstat shows a similar number:

# netstat -n | grep $OFFICEIP | grep :80 | wc -l
922

(Although interestingly the netstat number periodically changes, either going up or down a bit.)

We simply cannot find where these connections are coming from. Our office router doesn't show them in its tables, and doing a tcpdump (tcpdump -ieth1 -qnnvvSXs 1500 port 80) on the remote host shows zero traffic on that port at all, let alone from our office ip.

In addition, conntrack doesn't show the connections as existing either:

# conntrack -L | grep $OFFICEIP | grep 80
conntrack v0.9.13 (conntrack-tools): 38025 flow entries have been shown.

This is happening on a box running CentOS 6, kernel version 2.6.32-431.29.2.el6.x86_64. If anyone has any ideas as to where these discrepancies are coming from it would be greatly appreciated.

If I run the following:

chef-client --force-formatter --logfile STDOUT 2>&1

And then control-c it mid-run (or encounter any sort of error) I see this nice error message:

[2014-09-12T00:00:30-04:00] FATAL: SIGINT received, stopping

================================================================================
Recipe Compile Error in /var/cache/chef/cookbooks/users/recipes/fen.rb
================================================================================


SystemExit
----------
exit


Cookbook Trace:
---------------
  /var/cache/chef/cookbooks/users/recipes/default.rb:11:in ``'
  /var/cache/chef/cookbooks/users/recipes/default.rb:11:in `from_file'
  /var/cache/chef/cookbooks/users/recipes/fen.rb:15:in `from_file'


Relevant File Content:
----------------------
/var/cache/chef/cookbooks/users/recipes/default.rb:

etc...

[2014-09-12T00:00:30-04:00] ERROR: Running exception handlers
[2014-09-12T00:00:30-04:00] ERROR: Exception handlers complete
[2014-09-12T00:00:30-04:00] FATAL: Stacktrace dumped to /var/cache/chef/chef-stacktrace.out
Chef Client failed. 0 resources updated

But if I do the following:

chef-client --force-formatter --logfile STDOUT 2>&1 | tee /tmp/chef.log

And then control-c, I see all the log messages I was seeing before, with the exception of that nice fatal message telling me what was happening. The file I'm piping too also doesn't show the fatal message.

So apparently chef-client is detecting whether or not it's piping to an actual file descriptor, and suppressing output if it is. Does anyone know of a way to stop it from doing this?

I'm running a CentOS6 instance and have an upstart job that depends on sshd to already be running. However when I boot up the box that job fails to start, I'm guessing because sshd isn't actually running yet. Is there a way I can delay upstart jobs from starting until all normal init scripts have started?

This is a pretty basic question but I can't find any answers on it. I know you use mongos to interface with a sharded collection. However, if I have a replica set which is not sharded how do I set up mongos so that my application can use both the primary and secondaries in that set? Is it even necessary to use mongos or will having my application's mongo driver (php in this case) connect to one server in the set have it automatically enumerate the other members in the set and connect to them as well?

I'm currently trying to get nginx to add a header to the response when it is sending some kind of 50* error. I already have an add_header directive on the http block, and that gets respected for all requests except it seems errors. I also tried the following in one of my vhosts:

location /mediocregopheristhecoolest {
    add_header X-Test "blahblahblah";
    return 502;                                                                                                    
}       

Going to that page gives me a 502, but no header. Is this simply something nginx doesn't do, or am I doing it wrong?

I have the following nginx server config:

server {
    listen 80;
    server_name example.com
    root   /server/root;

    index index.php;

    error_page 404 = /index.php;

    location ~ \.php$ {
        try_files $uri =404;

        proxy_pass         http://127.0.0.1:8080;
        proxy_redirect     off;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_set_header   X-Request-URI    $request_uri;
    }

}

The behavior that I want is that when nginx encounters a request for a file which does not exist it instead shows the index.php page by way of a 404 page. The problem is it seems that apache (which is what's being proxied back to) is still trying to resolve the original request when it gets the request. If I go to http://example.com/blahblah, I get back the error:

The requested URL /blahblah was not found on this server.

Which is an apache error. How can I make it so that index.php is shown as the 404 page just the same as if it was a static file?

I've just started with wsgi and am trying to get a simple uwsgi server up and running. I've set up a virtualenv environment and activated it. Inside lib I have a file hello.py with the contents:

def application(environ, start_response):
    status = '200 OK'
    output = 'Hello World!'

    response_headers = [('Content-type', 'text/plain'),
                        ('Content-Length', str(len(output)))]
    start_response(status, response_headers)

    return [output]

And I'm running:

uwsgi --http :8000 --wsgi-file lib/hello.py --add-header "X-test: hi"

to start the server.

My problem is that the server isn't sedning the body. When I curl to localhost:8000 I can see the X-test header, so I'm definitely hitting uwsgi. Additionally, if I change "200 OK" to something else I can see that in curl as well.

I'm fairly sure I followed the tutorial correctly (it seemed pretty straightforward), can anyone spot what I'm doing wrong? Could it be that I'm using python3? I installed uwsgi through pip inside my virtualenv, if that matters at all.

I have openresty set up on a server (nginx bundle which includes the lua module) and I'm trying to create a script which has specific odds of setting a cookie on a user's browser. My code looks like this:

    location =/index.php {
        set $random_num 0;

        rewrite_by_lua '

                marth.randomseed(1);
                nvx.var.random_num = math.random(0,3);

        ';

        add_header Set-Cookie "random_num=$random_num; path=/; domain=...com;";
    }

I know my random seed function isn't actually very random, but I figured I'd deal with that later. At the moment I'm just trying to get nginx to set a random number, but doing so yields this error:

2012/07/11 11:27:20 [error] 5492#0: *44 lua handler aborted: runtime error: [string "rewrite_by_lua"]:3: attempt to ind
ex global 'marth' (a nil value)
stack traceback:

Can anyone tell me what I'm doing wrong, and if there's any other way to get a random number in nginx?

I'm currently migrating our ssl setup from apache to nginx, but I've hit a roadblock. In our apache config we have SSLInsecureRenegotiation on due to some incompatibility issues with safari. I can't find the equivalent command in nginx. Is there one?

I have the following as my knife.rb:

log_level                :info
log_location             STDOUT
node_name                'user'
client_key               '/home/user/.chef/user.pem'
validation_client_name   'chef-validator'
validation_key           '~/.chef/validation.pem'
chef_server_url          'url:4000'
cache_type               'BasicFile'
cache_options( :path => '/home/user/.chef/checksums' )
cookbook_path [ './', './site-cookbooks' ]

I can do "knife cookbook list" and all that fun stuff just fine, but when I go to edit I get:

# knife node edit test.domain.com -c knife.rb -e vim
ERROR: ArgumentError: Attribute chef_environment is not defined!

I have an environment set up:

# knife environment list
  _default
  production

and the node I'm trying to edit is on that environment. I have tried using the -E param, as well as adding an "environment" and "chef_environment" parameter to my knife.rb (the docs are a bit ambiguous as to which I should use), but to no avail. Anyone have any advice on this?

The setup I'm going for is as follows: I want nginx to look for a file in the /test subdirectory, and if it's not there, serve it from it's normal uri. I figured this would be fairly straightforward with try_files. Here's the setup I came up with:

server {
    listen       80;
    server_name  test-server;

    location / {
        root   /opt/www;
        index  index.php;
        try_files /test$uri $uri;
    }
}

This half-works. If I go to http://test-server/something, and there is a /test/something file, it will serve the /test/something file. However, if there isn't a /test/something file, but there is a /something file, it will return a 500 Internal Server error. My best guess is that there's some kind of recursive looping going on, but I don't know what the alternative would be.

I'm in the process of migrating a setup from apache to nginx. In this process I've come across this rewrite rule in an .htaccess file:

RewriteRule ^/?(?:(?!one|two|three|four).?)+/?$ http://somewhere.else.com [R=301,L]

I'm pretty good at regex usually, but this is way beyond me. For starters, I didn't know embedded parenthesis were even allowed. Can someone explain this regex to me? And if it's an apache only thing, how I can replicate it in nginx?

I have a script which rsync's a local directory to about 10 remote servers. It basically has a list of the servers and for loops through each of them doing the rsync. To speed up the process I have it fork for each of the rsyncs, so that all 10 happen in parallel. The problem is the last few of the servers come back with a "ssh_exchange_identification: Connection closed by remote host". This doesn't happen when doing the rsyncs individually or in serial, and putting a half second delay in between each fixes the problem (usually, sometimes the last server in the list still returns the error).

Does anyone know why this would be happening? I assume it's an issue with rysnc and not my script, as the script just does a system call to rysnc anyway.

Basically I have a couple cookbooks I've written, and currently they all have the same definition file copied on each. Obviously this is a bad situation, as it makes adding new cookbooks more difficult, and changing that definition file requires changing three or four other files to be exactly the same.

Is there a way that, if I have a separate cookbook who's purpose is to basically just contain that definition, I can import that definition file into the recipes that need it?

I'm working on a test setup for nginx+apache. I have nginx listening on port 80, apache on port 8080. Nginx is set up to deal out the static content, apache is there to deal out the dynamic content (at least until we can upgrade to php 5.3). My apache setup works fine, here's the server section for nginx:

server {

    listen           80;
    server_name  mediocregopher-test;
    index test.php;

    location ~ \.php$ {
        proxy_pass   http://127.0.0.1:8080;
    }

    location ~ /\.ht {
        deny  all;
    }

    location / {
        root   /var/www/html;
    }
}

The issue is, my index page (test.php) needs to be proxy'd to apache as "127.0.0.1:8080/test.php" when someone requests "mediocregopher-test:80/". With this setup this isn't happening. I'm fairly new to nginx, but I've looked around and couldn't find any setups that would solve this (although the problem is so simple it seems like it would be). Any suggestions?

I'm using nginx 1.0.1, if that's at all relevant.

I'm wondering if there's any way to find the tcp so_sndbuf of an actively running process (such as apache or nginx). I know that tcp_wmem sets the default value, but I'm not sure if the server programs I'm working with set their own.

My purpose for this is that I'm trying to set the Initial Window (IW) (see this link for why) for tcp connections. I looked at this paper from google, and they recommend changing your tcp_wmem default value to MSS*IW. I'm worried, however, that programs might be setting their own so_sndbuf instead of using tcp_wmem, so I want to double check.

I'm using linux kernel 2.6.18-194.26.1.el5 (CentOS).

We recently started getting the following error on one of our nginx boxes:

2011/05/25 16:35:51 [alert] 3580#0: accept() failed (24: Too many open files)

Checking /etc/security/limits.conf, we have this:

*                soft    nofile          900000
*                hard    nofile          900000

but when we did cat /proc/{pid}/limits it showed the file limit being 1024. When we restarted nginx the issue was fixed and /proc/{pid}/limits showed 900000. I'm thinking this may have been caused because the machine was rebooted, and on boot-up nginx was started before limits were applied. However everything I've been reading about how limits and pam work suggest that this isn't really how ulimits work. Does anyone know what's going on here?

Edit: Sorry, should mention os and stuff. We're running CentOS with kernel 2.6.18-194.26.1.el5, and nginx 1.0.1

I have zabbix set with an item to monitor a log file on a zabbix client:

log["/var/log/program_name/client.log","ERROR:","UTF-8",100]

And a trigger to determine when that log file get's more ERRORs:

{Template_Linux:log["/var/log/program_name/client.log","ERROR:","UTF-8",100].change(0)}#0

This trigger gets tripped when the log file gets ERRORs the first time, but then that first trigger just sits around for ever in Monitoring->Triggers. My understanding is that the next time the server checks the value of log["/var/log/program_name/client.log","ERROR:","UTF-8",100] and sees that it hasn't changed that the trigger would go away. Obviously this isn't the case. Could someone explain why this first trigger isn't going away?

Ultimately my goal is to receive an email whenever ERRORs are added to that log file, but I would like to understand how triggers are working first.

I need confirmation on a question regarding layer 2 security. I've been doing research and I just need to know that I understand it correctly.

Basically, if you have a LAN with a few machines connected to a switch, and those machines are broadcasting LLDP or ARP or something like that, is it true that those broadcasted packets will never escape the LAN (sans faulty software in the switch or machines and things of that nature)?

I know this is a basic question, but I haven't been able to find a direct answer to it and am hoping someone could just give a really brief one. Thanks!