I do realise this question has been asked about 6 years ago Bind nginx on the same port for tcp and http
However, has anything changed, is it possible for me to do something like:
stream {
listen 80;
location / {
protocol http;
}
location /mysql {
protocol tcp;
}
}
Or must thetcp and http modules still be separate?
I know how to scan one directory:
clamscan -r /home
but for the life of me I cannot seem to get multiple directories working and I cannot seem to Google this simple problem.
I have tried:
clamscan -r /home /otherdir
clamscan -r /home,/otherdir
to no avail.
What am I doing wrong?
I have this defining an alias (latest iteration of it):
location ~ /xxx/(.*).php($|/) {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /xxx/web/blog/xxx/$1.php;
include fastcgi_params;
}
location ~ /xxx(.*) {
autoindex on;
alias /xxx/web/blog/xxx$1;
}
To try and define a wordpress install within the directory of xxx of xxx.com (for example).
Now when I take out the PHP path I get the index.php file downloaded to my computer but as soon as I add PHP fpm, as defined by the php location, I get:
FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream
In the logs. I have tried debug mode in error logs but it all looks good.
I do have a php location already doing:
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
try_files $uri =404;
# # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /xxx/web/live/backend/web/$fastcgi_script_name;
include fastcgi_params;
}
I have checked the output of /xxx/web/blog/xxx/$1.php using return 200 "/xxx/web/blog/xxx/$1.php"; and it looks perfect.
I should add that I have been searhcing on SE for about 3 hours and not a single post on this actually solved my problems yet, including:
So why won't this work?
I cannot seem to get Nginx basic auth working from behind a Rackspace cloud balancer.
First I am using this code (it is also using real IP module here which was one of the mannny solutions I tried) within my nginx.conf:
http {
set_real_ip_from 1.1.1.2;
#real_ip_header X-Forwarded-For;
satisfy any;
allow 1.1.1.2;
deny all;
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/htpasswd;
And this is the fields I see in my header within PHP:
["HTTP_X_FORWARDED_FOR"]=> string(14) "1.1.1.2"
And for some reason Nginx does not work on that by default.
I have tried quite a things but I cannot seem to get it to work.
Any ideas?
I need the upload progress extension on my production server running Nginx, the only problem is that to do that I must actually recompile the whole of Nginx.
This means uninstalling the repo Nginx, installing from source (and all the maintenance that comes with it) and compiling with the module option pointing to the downloaded upload progress source.
I did however, find a less painful way of doing it by installing nginx-extras however this is the -V signature post-installation:
configure arguments: --prefix=/etc/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --with-debug --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-sha1=/usr/include/openssl --with-md5=/usr/include/openssl --with-mail --with-mail_ssl_module --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-auth-pam --add-module=/build/buildd/nginx-1.1.19/debian/modules/chunkin-nginx-module --add-module=/build/buildd/nginx-1.1.19/debian/modules/headers-more-nginx-module --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-development-kit --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-echo --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-http-push --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-lua --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-upload-module --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-upload-progress --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-upstream-fair --add-module=/build/buildd/nginx-1.1.19/debian/modules/nginx-dav-ext-module
And I am slightly concerned that one of these extras could cause problems on a production server.
Does nginx work in such a way that if I do not include the options for the extensions in extras it won't use them or I won't notice them or should I go to the effort of manually compiling Nginx?
My initial test on my dev server tells me that these don't make too much visible changes until they're used but I am unsure if I am missing something.
I want to allow 1GB uploads on a specific subdomain of my site called upload.xxx.xx.xx.
Currently I am trying something like:
server {
if ( $host = 'upload.xxx.xx.xx' ) {
client_max_body_size 1000M;
}
}
But of course it gives me an error about the client_max_body_size not supposed to be there.
Is there a way to accomplish what I want without duplicating the two vhosts, all the settings are the same otherwise, or on the flip side should I not bother to go to the trouble of making sure the user can only upload greater than 10MB on upload subdomain?
I have been reading this link: http://blog.martinfjordvald.com/2010/08/file-uploading-with-php-and-nginx/ (a blog whose posts are repeatedly linked from the official Nginx site) which says:
Each time Apache accepts an upload one of these processes is going to be busy for the duration of the upload.
Now I will admit I am a noob at Nginx but doesn't it also suffer the problem that one of its own workers will be used for the duration of the upload, OK, fair enough it isn't a PHP instance with the RAM/CPU problems that come with it but it is still limited (i.e. x number of connections), right?
I am not sure if the advantage of using Nginx here is quite explained.
The Nginx documentation itself is not, currently, very clear on exactly how it truly processes a request so if I have something incredibly wrong about how Nginx works then I hope someone will fix it instead of downvoting.
I have basic HTTP auth configured to a specific Location in my Apache config like:
<Location /system>
AuthType Basic
AuthName "Password Required"
AuthUserFile /to/passwd/file
Require valid-user
Order allow,deny
Allow from xxx
Allow from xxx
Satisfy Any
</Location>
with /system aliasing a folder which goes to a path /system.
Now I have a URL in this /system path of /system/user/synchro which I need to be public to everyone. It should be noted that the destination of the path is not a physical file but rather a rewritten URL.
Is there a way to nest a Location (or something) to be able to open up this one path but keep the rest locked down?
I have been doing some reading and have come to the understanding that if you wish to use a launchConfig with Ubuntu you will need to install the cfn-init file yourself which I have done:
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"SpotPrice" : "0.05",
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" },
{ "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" },
"Arch" ] } ] },
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"InstanceType" : { "Ref" : "InstanceType" },
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash\n",
"apt-get -y install python-setuptools\n",
"easy_install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-1.0-6.tar.gz\n",
"cfn-init ",
" --stack ", { "Ref" : "AWS::StackName" },
" --resource LaunchConfig ",
" --configset ALL",
" --access-key ", { "Ref" : "WorkerKeys" },
" --secret-key ", {"Fn::GetAtt": ["WorkerKeys", "SecretAccessKey"]},
" --region ", { "Ref" : "AWS::Region" }, " || error_exit 'Failed to run cfn-init'\n"
]]}}
But I have a problem with this setup that I cannot seem to get a decent answer to. I keep getting this error in the logs:
Jun 15 12:02:34 ip-0 [CLOUDINIT] __init__.py[DEBUG]: config-scripts-per-once already ran once
Jun 15 12:02:34 ip-0 [CLOUDINIT] __init__.py[DEBUG]: handling scripts-per-boot with freq=None and args=[]
Jun 15 12:02:34 ip-0 [CLOUDINIT] __init__.py[DEBUG]: handling scripts-per-instance with freq=None and args=[]
Jun 15 12:02:34 ip-0 [CLOUDINIT] __init__.py[DEBUG]: handling scripts-user with freq=None and args=[]
Jun 15 12:02:34 ip-0 [CLOUDINIT] cc_scripts_user.py[WARNING]: failed to run-parts in /var/lib/cloud/instance/scripts
Jun 15 12:02:34 ip-0 [CLOUDINIT] __init__.py[WARNING]: Traceback (most recent call last):#012 File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/__init__.py", line 117, in run_cc_modules#012 cc.handle(name, run_args, freq=freq)#012 File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/__init__.py", line 78, in handle#012 [name, self.cfg, self.cloud, cloudinit.log, args])#012 File "/usr/lib/python2.7/dist-packages/cloudinit/__init__.py", line 326, in sem_and_run#012 func(*args)#012 File "/usr/lib/python2.7/dist-packages/cloudinit/CloudConfig/cc_scripts_user.py", line 31, in handle#012 util.runparts(runparts_path)#012 File "/usr/lib/python2.7/dist-packages/cloudinit/util.py", line 223, in runparts#012 raise RuntimeError('runparts: %i failures' % failed)#012RuntimeError: runparts: 1 failures
Jun 15 12:02:34 ip-0 [CLOUDINIT] __init__.py[ERROR]: config handling of scripts-user, None, [] failed
Jun 15 12:02:34 ip-0 [CLOUDINIT] __init__.py[DEBUG]: handling keys-to-console with freq=None and args=[]
Jun 15 12:02:34 ip-0 [CLOUDINIT] __init__.py[DEBUG]: handling phone-home with freq=None and args=[]
Jun 15 12:02:34 ip-0 [CLOUDINIT] __init__.py[DEBUG]: handling final-message with freq=None and args=[]
Jun 15 12:02:34 ip-0 [CLOUDINIT] cloud-init-cfg[ERROR]: errors running cloud_config [final]: ['scripts-user']
I have absolutely no idea what scripts-user means and Google is not helping much here either. I can, when I ssh into the server, see that it runs the userdata script since I can access cfn-init as a command whereas I cannot in the original AMI the instance is made from.
However I have a launchConfig:
"Comment" : "Install a simple PHP application",
"AWS::CloudFormation::Init" : {
"configSets" : {
"ALL" : ["WorkerRole"]
},
"WorkerRole" : {
"files" : {
"/etc/cron.d/worker.cron" : {
"content" : "*/1 * * * * ubuntu /home/ubuntu/worker_cron.php &> /home/ubuntu/worker.log\n",
"mode" : "000644",
"owner" : "root",
"group" : "root"
},
"/home/ubuntu/worker_cron.php" : {
"content" : { "Fn::Join" : ["", [
"#!/usr/bin/env php",
"<?php",
"define('ROOT', dirname(__FILE__));",
"const AWS_KEY = \"", { "Ref" : "WorkerKeys" }, "\";",
"const AWS_SECRET = \"", { "Fn::GetAtt": ["WorkerKeys", "SecretAccessKey"]}, "\";",
"const QUEUE = \"", { "Ref" : "InputQueue" }, "\";",
"exec('git clone x '.ROOT.'/worker');",
"if(!file_exists(ROOT.'/worker/worker_despatcher.php')){",
"echo 'git not downloaded right';",
"exit();",
"}",
"echo 'git downloaded';",
"include_once ROOT.'/worker/worker_despatcher.php';"
]]},
"mode" : "000755",
"owner" : "ubuntu",
"group" : "ubuntu"
}
}
}
}
Which does not seem to run at all. I have checked for the files existance in my home directory and it's not there. I have checked for the cronjob entry and it's not there either. I cannot, after reading through the documentation, seem to see what's potentially wrong with my code.
Any thoughts on why this is not working? Am I missing something blatant?
I am having real problems trying to wrap my head around my problem. I have decided on the middle and end solution to my problem but I can't seem to work out the the beginning of it.
So as a bit of a project I'm building a Youtube style site. My solution to the problem of uploading videos so far consists of:
The problem I got is how to handle the initial upload to server. So when a user goes onto the upload page how to initially get that video:
Now the main problem is space. I am trying to work out how to upload the file always knowing there will be enough space to initially boost that onto S3 (Thinking of many uploads per minute here). My worry is that I have to upload initially either to a super storage server like S3 but not CDN or I upload to the app server and hope for the best. Neither really seem super scalable in terms of both expansion and money so I am wondering how best is it to get the video uploaded initially?
If I must use my app server how best should I set it up to deal with these uploads? Should I just get a disk heavy app server?
If I were to do it another way outside of the app server how would I do it?
Sorry if my question seems a bit vague, this has given me a bit of a headache thinking about.
Thanks,
EIDT: I decided to use zencoder with S3. I think it's actually a really good idea I won't try and create this part myself as I don't really intend to properly host this site so it would just end up with me paying lots of money for something that might be better done third party.
Thanks for the help guys :D
I hope this is the right place for this. I have a Ubuntu 11.10 server with Apache on. Apaches doc root is on a shared folder sourced via cifs from a windows Machine.
All this is fine on every network I have set it up on until today when I came to set it up at home.
I am viewing my website, on my Apache server in a web browser (lets say Firefox) and every couple of refreshes I get a 403 response or no data at all back for certain files from the servers cif share in Firebug.
Now through the process of elimination I have been able to determine it is cifs. I came to this conclusion by copying my doc root to a local folder on the Ubuntu install and trying to get the same behaviour, which I did not.
Now I understand this is very hard to grasp through plain text so here are a couple of pictures to describe the problem.
This first picture shows everything on one refresh and it is all fine:
This picture shows the next refresh, as you can see Firebug responds that apache suddenly cannot access my files, not only that but what Firebug is not reporting that there are also images missing off of that page. I should mention the images actually come from DB, so not sure how cifs is interferring with that but the images work with the local version of the site so it must be.
This problem is intermitant.
Now I did try and Google this but then I realised, after a couple of tries, I wasn't actually quite sure what to Google for.
Can some one point me in the right direction to diagnose my problems? Maybe some one has had this problem before and can share their solution?
Thanks,
Update: Ok so I did a little searching around in the APache logs and it comes up with this:
[Thu Jan 19 20:59:03 2012] [error] [client 192.168.153.1] (12)Cannot allocate memory: file permissions deny server access: /server_ws/htdocs/css/960.css, referer: http://stagex.co.uk/stream
I have seen this error quite a lot and it always has a different meaning. So first I decided to check the actual file permissions in Linux before I did a refresh after this problem:
drwxrwxrwx 1 www-data sam 0 2012-01-17 18:33 .
drwxrwxrwx 1 www-data sam 4096 2012-01-17 21:40 ..
-rwxrwxrwx 1 www-data sam 10641 2012-01-17 18:33 960.css
-rwxrwxrwx 1 www-data sam 0 2012-01-20 18:58 c_css.css
-rwxrwxrwx 1 www-data sam 129488 2012-01-17 18:33 main.css
-rwxrwxrwx 1 www-data sam 1086 2012-01-17 18:33 reset.css
drwxrwxrwx 0 www-data sam 0 2012-01-17 18:33 .svn
Everything looks tip top there. So I checked Windows to make sure I will always have access to it. I have shared the server_ws folder from windows using the built in protocol and have assigned my Windows user "Sam" as the owner. I have told cifs to hook into this Windows like so:
//192.168.0.2/server_ws/the_stage /media/server_ws cifs username=Sam,password=****,rw,uid=www-data,gid=sam,iocharset=utf8,dir_mode=0777,file_mode=0777 0 0
Nothig immediately wrong there and if there was something terrible there why would it only happen every so oftern.
Though now that I've found the error in Apache Ima do more Google searching.
If anyone knows something please let me know :)
Up until recently I was using mongo to try and stem to tide of massive MySQL upgrades to system servers, however, after reading this:
http://www.mikealrogers.com/2010/07/mongodb-performance-durability/
I have halted production of a mongo version of my site until mongo would gain more stability.
Some have suggested I use the _change within couche DB to create a new mongo object which would solve the main issues within mongo db atm but I'm not sure about it.
I have looked into other DB's such as redis and Cassandra but I have since ditched Cassandra as you have to design for your queries and that's just too restricting for my site (bad modulisation). I'm not really looking for joins etc, I'm just looking for the ability to search within a row instead of just the columnfamily id's as it can make programming quite hard when trying to add new functionality. It would be great for a search engine or something but not so good for a real website like facebook in it's entirty (instead of just in its mail search).
I was wondering what experiences people on here have had with trying to get a workable solution to the SQL speed issues. Is there a silver bullet db for it all or is it just a case of gritting your teeth with MySQL or another SQL db if you want reliability?
Maybe a cross between a form of caching and SQL (I noticed facebook uses heavy caching on their wall pages etc...probably why).
Thanks,
I have been investigating the differences between Apache and Nginx recently and am confused about which I should choose.
I have done some searching but there is no definitive comparison between the two and I was wondering if someone here could give their views on the differences between the two.
My current knowledge allows me to understand that mod_php is faster and more secure than fastcgi however Apache is much worse when it comes to simultaneous connections and memory consumption.
My site is using a lot of long polling but has a non AJAX web base (i.e. Apache with long polling over the top).
My original solution to Apaches memory problems were to send the long polling through node.js and then get node.js to access Apache every 2 secs in which case Apache would not have an open connection but instead node.js would. I have come to the realisation this might not be good enough and am looking at different solutions. I am still interested as to whether my original idea would have worked.
So which is better for the modern web? Apache or Nginx?
Update: All the suggestions given were good and valid. I have gone with the original second idea which is to use a full Nginx server. I am satisfied that being a dedicated server I could not suffer security issues from fastcgi and since my long polling scripts need to be written in PHP I require a server that can deal with high load simultaneous connections and Apache just cannot do that no matter how much I change the structure it will still be memory hungry.
I have marked Martin F's answer since he gave such a clear and complete answer to my questions points that I feel he deserves the mark, however, all three answers were good and valid and will most definately look into using reverse proxy for another site I own since I just found something very very very kool that Nginx can do in proxying.
Thanks,