rahul286's questions

After debugging for 6-hours - I am giving this up :|

We have a nginx+php-fpm+mysql in LAN with almost 100 wordpress (created and used by different designers/developers all working on test wordpres setup)

We are using nginx without any issues from long.

Today, all of a sudden - nginx started returning "504 Gateway Time-out" out of the blue...

I checked nginx error log for a virtual host...

2010/09/06 21:24:24 [error] 12909#0: *349 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.0.1, server: rahul286.rtcamp.info, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "rahul286.rtcamp.info"
2010/09/06 21:25:11 [error] 12909#0: *349 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 192.168.0.1, server: rahul286.rtcamp.info, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "rahul286.rtcamp.info"
2010/09/06 21:25:11 [error] 12909#0: *443 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 192.168.0.1, server: rahul286.rtcamp.info, request: "GET /info.php HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "rahul286.rtcamp.info"
2010/09/06 21:25:12 [error] 12909#0: *443 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.0.1, server: rahul286.rtcamp.info, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "rahul286.rtcamp.info"
2010/09/06 22:08:32 [error] 12909#0: *1025 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.0.1, server: rahul286.rtcamp.info, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "rahul286.rtcamp.info"
2010/09/06 22:09:33 [error] 12909#0: *1025 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.0.1, server: rahul286.rtcamp.info, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "rahul286.rtcamp.info"
2010/09/06 22:09:40 [error] 12909#0: *1064 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 192.168.0.1, server: rahul286.rtcamp.info, request: "GET /info.php HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "rahul286.rtcamp.info"
2010/09/06 22:09:40 [error] 12909#0: *1064 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.0.1, server: rahul286.rtcamp.info, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "rahul286.rtcamp.info"
2010/09/06 22:24:44 [error] 12909#0: *1313 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.0.1, server: rahul286.rtcamp.info, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "rahul286.rtcamp.info"
2010/09/06 22:24:53 [error] 12909#0: *1313 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 192.168.0.1, server: rahul286.rtcamp.info, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "rahul286.rtcamp.info"

As I run php-fpm on port 9000 via TCP mode, I ran "netstat | grep 9000" and noticed something unusual... (Pasting partial output here for ease of read)

tcp        9      0 localhost:9000          localhost:36094         CLOSE_WAIT  14269/php5-fpm  
tcp        0      0 localhost:46664         localhost:9000          FIN_WAIT2   -               
tcp     1257      0 localhost:9000          localhost:36135         CLOSE_WAIT  -               
tcp     1257      0 localhost:9000          localhost:36125         CLOSE_WAIT  -               
tcp        9      0 localhost:9000          localhost:36102         CLOSE_WAIT  14268/php5-fpm  
tcp        0      0 localhost:46662         localhost:9000          FIN_WAIT2   -               
tcp      745      0 localhost:9000          localhost:46644         CLOSE_WAIT  -               
tcp        0      0 localhost:46658         localhost:9000          FIN_WAIT2   -               
tcp     1265      0 localhost:9000          localhost:46607         CLOSE_WAIT  -               
tcp        0      0 localhost:46672         localhost:9000          ESTABLISHED 12909/nginx: worker
tcp     1257      0 localhost:9000          localhost:36119         CLOSE_WAIT  -               
tcp     1265      0 localhost:9000          localhost:46613         CLOSE_WAIT  -               
tcp        0      0 localhost:46646         localhost:9000          FIN_WAIT2   -               
tcp     1257      0 localhost:9000          localhost:36137         CLOSE_WAIT  -               
tcp        0      0 localhost:46670         localhost:9000          ESTABLISHED 12909/nginx: worker
tcp     1265      0 localhost:9000          localhost:46619         CLOSE_WAIT  -               
tcp     1336      0 localhost:9000          localhost:46668         ESTABLISHED -               
tcp        0      0 localhost:46648         localhost:9000          FIN_WAIT2   -               
tcp     1336      0 localhost:9000          localhost:46670         ESTABLISHED -               
tcp        9      0 localhost:9000          localhost:36108         CLOSE_WAIT  14274/php5-fpm  
tcp     1336      0 localhost:9000          localhost:46684         ESTABLISHED -               
tcp        0      0 localhost:46674         localhost:9000          ESTABLISHED 12909/nginx: worker
tcp     1336      0 localhost:9000          localhost:46666         ESTABLISHED -               
tcp     1257      0 localhost:9000          localhost:46648         CLOSE_WAIT  -               
tcp     1336      0 localhost:9000          localhost:46678         ESTABLISHED -               
tcp        0      0 localhost:46668         localhost:9000          ESTABLISHED 12909/nginx: wo             

There are plenty of "CLOSE_WAIT" & "FIN_WAIT2" pairs as highlighted below (in above output):

tcp     1337      0 localhost:9000          localhost:46680         CLOSE_WAIT  -               
tcp        0      0 localhost:46680         localhost:9000          FIN_WAIT2   -

Please note port 46680 in above.

I enabled mysql slow queries error log, but it didn't work.

As of now restarting php5-fpm every minute via a cronjob (see command below) keeping everything running "smoothly" but I hate patchwork and want to solve this...

1 * * * * service php5-fpm restart > /dev/null

I searched extensively on Google - got no help. As mentioned, this a test-server in LAN, CPU load is never crossed 0.10 and memory usage is also below 25% (System has 2GB RAM and ubuntu-server installed) So if you find its time-confusing to help me out, please atleast drop a hint.

Thanks in advance for help.

-Rahul

(note - this is reposting of - http://forum.nginx.org/read.php?11,127694)

Update: I found answer, which is posted below.

First, I am not sure if Varnish can be used as web-cache to speedup internet in small offices.

We have an office with 20 systems around and we frequently browse same set of websites.

I setup squid and it worked fine for sometime. But I couldn't get to cache youtube videos which was my main aim.

I came recently across Varnish while playing with nginx but I see it is being used as web-accelerator for backend server.

Can we use Varnish instead of nginx for web-cache in transparent way? I mean to say that people in my office will not need to configure proxy on their machine.

Thanks,

-Rahul

I just moved our blog-network to ngnix (from apache) which is powered by WPMU. Everything is running smoothly and fine.

During my quest to optimize server to the max level - I came across this thread on nginx forum http://forum.nginx.org/read.php?2,2649

I would really like to know: Answer to the original question...

For example, to prevent the warning when serving a 6144 kilobyte file, which of the three methods would yield the best performance?

1. A large number of small buffers: fastcgi_buffers 768 8k
2. A small number of large buffers: fastcgi_buffers 8 768k
3. A near balance between the buffers and size: fastcgi_buffers 64 96k

And also one more thing: In my ngnix.config - my setting is:

fastcgi_buffers 64 4k;

i.e. 256K buffer size (I am on 32-bit platform). Does this mean if my PHP scripts generates response larger than 256K, nginx will throw some error?

I know ngnix forum seems better place to ask this but as no ngnix forum is more active than serverfault, I'm posting this question here.

I hope I will get some inputs/guidance from experts here.

Thanks,

-Rahul

We just completed complex network design for our new office. It has 2 ADSL routers connected to a Dual WAN Load Balancer router. Load Balancer is connected to 2 16-port Switches which connects 30 PCs. Also one 16-port switch is connected to another 16-port switch which in turn connects to the Load Balancer.

So my PC have logical path: PC >> SWITCH A >> SWITCH B [Optional] >> Load Balancer >> ADSL Modem [one of two available in network]

As I was facing some weird problems, I decided to run diagnose. My internet is working fine. Actually HTTP POSTs and FILE UPLOADS sometimes gets timed out.

Traceroute to external server (same output I get for Google/Facebook/etc). Number of hops remains 15.

[rtcamp@main ~]$ traceroute rtcamp.com
traceroute to rtcamp.com (70.32.85.76), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  rtcamp.com (70.32.85.76)  362.911 ms  364.550 ms  366.284 ms

Traceroute to Load Balancer router

[rtcamp@main /]$ traceroute 192.168.0.1
traceroute to 192.168.0.1 (192.168.0.1), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

My biggest problem is. We have created a public server for our subdomain like sub.example.com. Now sub.example.com works from outside world, but cannot be reached from network within.

I think if I can get normal traceroute output, things will be solved.

Any solution or idea?

Thanks,

-Rahul

Added on 10 September

Details of our network setup

1. We have network of 192.168.0.x
2. 192.168.0.1 is load balancer
3. 192.168.1.1 is ADSL modem A
4. Another ADSL modem is in bridge mode
5. We have PC's from 192.168.0.2 to 192.168.0.50 (PC get IP address dynamically)
6. 192.168.0.101/2 are for server in LAN. Its only one server with 2 LAN cards so 2 ip address.
7. 192.168.0.200 is Wi-fi router and 192.168.0.201 onwards IP address are for laptops connected to wi-fi router. Wifi router gets LAN IP 192.168.0.100 from Load balancer as well on its ethernet interface.

What I am doing...

Trying to implement single-sign-on for our organization’s all machines, blogs, wiki’s, CRM, HRM, project management tools, SVN, etc, etc...

We have OpenLDAP installed and configured on our dedicated server running CentOS. I used phpLdapAdmin to add organization structure and info about various users, clients, resources.

An example entry for a user...

DN is :: cn=Bill Gates, ou=users, dc=example, dc=com

userid :: bill.gates

mail :: [email protected]

userpassword :: as2%$%66789ds (some md5 cryptic value)

Where I am now...

OpenLDdap is working fine. Bind test also ran successfully.

What I want to do...

Bind using higher privilege user and then search users by their entered userid or mail, which are slightly different from CN. Point is I want to authenticate user against an attribute which is not part of RDN.

Where I am stuck...

1. I can’t bind using userid as its not apart of DN. Is this allowed in general?
2. I can bind using different LDAP user say, high-privileged user and do ldap_search to get a unique record based on filter userId but then md5 of user entered password doesn't match with userPassword field. OpenDLdap encrypts using some salt. I don't want to remove salt either. Is there any way out?

My questions (finally)

1. Can we do ldap_bind on an attribute which is not part of RDN?
2. Can we send a unencrypted password (I will use HTTPS tunnel for security) to OpenLDAP server and ask OpenLDAP to encrypt is and check against userpassword filed? This is normally we do in all web-apps, right?
3. (off-topic) Can userid contain . (dots) and spaces. We need some char between “First Name(givenName)” and "Last Name(sn)". Which one is safe character? I mean which non-aplha-numeric character is allowed in usernames by world's all (or most) application?

My biggest concern is to go with an approach, which can be easily made to work with most apps. There are many web-based apps, desktop softwares, etc which we will be modifying during our mission single sign-on!

Thanks for reading... and thanks in advance for some help!

-Rahul

I keep hearing about some PHP (opcode) caches like - APC, XCache, Memcache, eAccelerator, etc.

But I couldn't ever figure out how to go about choosing one. Apart from performance benefit, which a caching system is supposed to deliver, which other factors should be a point of concern.

Like why you will say X cache system is better than Y? I am less worried about relative performance gain. Small differences between any two systems matter less.

If a generic answer to my question is not possible, here are few pointers. I use dedicated VPS with Mediatemple (with root access). RAM is 512 MB (physical) + 400MB (swap) I am concerned about WordPress and its cousins WordPress-MU and BuddyPress. 90% of our codes/sites fall into WordPress family.

Thanks in advance for some help.