Tomas Morgan's questions

Hi everyone and thanks in advance.

So I got stuck with FTP response «550 Permission denied.»

System: Ubuntu 14.04 and vsftpd: version 3.0.2

I created a drupal user for working over FTP, homedir: /var/www/vhosts/somedomain.com

Part of /etc/passwd:

drupal:x:1005:65034::/var/www/vhosts/somedomain.com:/bin/false

VSFTPD conf /etc/vsftpd.conf:

listen=YES
listen_port=xxxx
anonymous_enable=NO
local_enable=YES
write_enable=YES
allow_writeable_chroot=YES
local_umask=022
dirmessage_enable=YES
use_localtime=YES
xferlog_enable=YES
connect_from_port_20=YES
chroot_local_user=YES
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd/nchroot
log_ftp_protocol=YES
secure_chroot_dir=/var/run/vsftpd/empty
rsa_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
rsa_private_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
tcp_wrappers=YES
pasv_enable=YES
pasv_min_port=1024
pasv_max_port=1048
pasv_address=x.x.x.x
user_config_dir=/etc/vsftpd/user_conf
guest_enable=YES
guest_username=vsftpd

All /etc/pam.d/vsftpd:

auth    sufficient  pam_pwdfile.so pwdfile /etc/vsftpd/ftpd.passwd
account required pam_permit.so

I tried to change permissions to 777 and the owner to drupal:drupal on /var/www/vhosts/somedomain.com/htdocs

Connection to FTP server seems to work and I can see a list of proper files and dirs, however I can't create a file.

In vsftpd logs I get: FTP response «550 Permission denied.»

Google keeps on telling me that it's either a permissions issue, or vsftpd.conf missconfiguration in

write_enable=YES
allow_writeable_chroot=YES

But in my case everything is set.

I'm thinking that I'm missing something in /etc/pam.d/vsftpd as it's customized by other guy.

I'm trying to block requests from bots by pattern glitter_calendar on apache.

Prehistory. My server is heavily accessed by bots. The request URI is ...glitter_calendar.. The wordpress returns 404. The server even goes down once a week. I tried blocking them with Fail2ban. Without success. The server load doesn't go down and I get new hosts appearing in my iptables block-rule constantly.

In my .htaccess I am trying to add a block rule by pattern glitter_calendar:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{THE_REQUEST} ^.*(wpdffcontent)|(glitter_calendar)|(mp4:).* [NC]
RewriteRule ^(.*)$ - [F,L]
</IfModule>

<Files xmlrpc.php>
Order Deny,Allow
Deny from all
</Files>

Found similar questions. Links:

• https://stackoverflow.com/questions/4640807/how-to-block-bot-requests-to-urls-that-match-a-common-pattern-in-apache/4640865#4640865
• https://perishablepress.com/eight-ways-to-blacklist-with-apaches-mod_rewrite/

My mod rewrite doesn't seem to work. I get Error 404 Not Found inside wordpress, not by means of apache.

Module mod_rewrite is turned on:

# httpd -M
...
Loaded Modules:
...
rewrite_module (shared)
...

The virtual host configuration is managed with virtualmin. How do I fix this?

Part of httpd.conf:

...
LoadModule rewrite_module modules/mod_rewrite.so
...

<VirtualHost x.x.x.x]:80>
SuexecUserGroup "#505" "#504"
ServerName example.com
ServerAlias www.example.com
ServerAlias webmail.example.com
ServerAlias admin.example.com
ServerAlias autoconfig.example.com
ServerAlias autodiscover.example.com
DocumentRoot /home/example/public_html
ErrorLog /var/log/virtualmin/example.com_error_log
CustomLog /var/log/virtualmin/example.com_access_log combined
ScriptAlias /cgi-bin/ /home/example/cgi-bin/
ScriptAlias /awstats/ /home/example/cgi-bin/
ScriptAlias /AutoDiscover/AutoDiscover.xml /home/example/cgi-bin/autoconfig.cgi
ScriptAlias /Autodiscover/Autodiscover.xml /home/example/cgi-bin/autoconfig.cgi
ScriptAlias /autodiscover/autodiscover.xml /home/example/cgi-bin/autoconfig.cgi
DirectoryIndex index.html index.htm index.php index.php4 index.php5
<Directory /home/example/public_html>
Options -Indexes +IncludesNOEXEC +SymLinksIfOwnerMatch +ExecCGI
allow from all
AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch
AddType application/x-httpd-php .php
AddHandler fcgid-script .php
AddHandler fcgid-script .php5
FCGIWrapper /home/example/fcgi-bin/php5.fcgi .php
FCGIWrapper /home/example/fcgi-bin/php5.fcgi .php5
</Directory>
<Directory /home/example/cgi-bin>
allow from all
AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch
</Directory>
RewriteEngine on
RewriteCond %{HTTP_HOST} =webmail.example.com
RewriteRule ^(.*) https://example.com:20000/ [R]
RewriteCond %{HTTP_HOST} =admin.example.com
RewriteRule ^(.*) https://example.com:10000/ [R]
RemoveHandler .php
RemoveHandler .php5
php_admin_value engine Off
FcgidMaxRequestLen 1073741824
Alias /dav /home/example/public_html
Alias /pipermail /var/lib/mailman/archives/public
<Location /dav>
DAV on
AuthType Basic
AuthName "example.com"
AuthUserFile /home/example/etc/dav.digest.passwd
Require valid-user
ForceType text/plain
Satisfy All
RemoveHandler .php
RemoveHandler .php5
RewriteEngine off
</Location>
<Files awstats.pl>
AuthName "example.com statistics"
AuthType Basic
AuthUserFile /home/example/.awstats-htpasswd
require valid-user
</Files>
RedirectMatch /cgi-bin/mailman/([^/\.]*)(.cgi)?(.*) https://example.com:10000/virtualmin-mailman/unauthenticated/$1.cgi$3
RedirectMatch /mailman/([^/\.]*)(.cgi)?(.*) https://example.com:10000/virtualmin-mailman/unauthenticated/$1.cgi$3
php_value memory_limit 32M
IPCCommTimeout 41
Redirect /mail/config-v1.1.xml /cgi-bin/autoconfig.cgi
Redirect /.well-known/autoconfig/mail/config-v1.1.xml /cgi-bin/autoconfig.cgi
</VirtualHost>

Is there a way I can conditionally log a web request that takes more than 3 seconds?

Thanks.

I've got an issue with uwsgi. When I start uwsgi from conole:

 uwsgi --socket 127.0.0.1:5555  --chdir /var/www/proj/smth/ --wsgi-file /var/www/.../rest_api/wsgi.py &

it shows web pages and everithing looks fine. But when I use uwsgi config file with something like this:

[uwsgi]
chdir = /var/www/proj
socket = :5555
wsgi-file = /var/www/proj/rest/rest_api/wsgi.py 
home = /var/www/proj
processes = 4
threads = 2
touch-reload=/var/www/proj/rest/rest_api/wsgi.py
daemonize=/var/log/uwsgi/rest.log
vacuum=true
; wtf we get errors w-out this and it won't start: 
no-site=true

I get an internal server error displayed in my web browser. I run nginx.

And some of my uwsgi log lines look like this:

ImportError: No module named ...
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
--- no python application found, check your startup logs for errors ---

I set up ghost blogging platform. It doesn't allow to insert «/» signs in article links, but I want them to appear in my links, so I want them to be fixed with nginx rewrites.

The link in a browser looks like this:

http://blog.site.com/stuff-life/

And I want it to be like this:

http://blog.site.com/stuff/life/

It goes without saying that it should be applied for a number of links starting from "stuff-*", not for one.

I found this, but it's way too complicated for me:

http://nginx.org/en/docs/http/ngx_http_rewrite_module.html

My config looks like this:

server {
listen 80;
server_name blog.site.com;
access_log /var/log/nginx/ghost.log;
error_log /var/log/nginx/ghost.error.log;
    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header HOST $http_host;
        proxy_set_header X-NginX-Proxy true;

        proxy_pass http://127.0.0.1:2368;
        proxy_redirect off;
        }
}

So how I do a rewrite in my case? Thanks.

Update.

Now I try to do it this way:

server {
listen 80;
server_name somesite.com;
access_log /var/log/nginx/ghost.log;
error_log /var/log/nginx/ghost.error.log;
location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header HOST $http_host;
    proxy_set_header X-NginX-Proxy true;

    proxy_pass http://127.0.0.1:2368;
    proxy_redirect off;
}


location /stuff- {
    rewrite ^/stuff-(.*) http://$server_name/stuff/$1 permanent;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header HOST $http_host;
    proxy_set_header X-NginX-Proxy true;

    proxy_pass http://127.0.0.1:2368;
    proxy_redirect off;
}


}

And it looks like nginx is passing rewrited links to ghost. And I get 404 page.

I set up Cisco Wccp + Squid on Suse Linux Enterprise 11 SP3.

Firefox opens our intranet portal pages with no problems. I can see requests to squid. I see cache hits and access in squid log. I see GRE traffic from Cisco on a tunnel in linux (for some reason only incoming RX – I see changing counters on the interface). I see squid returns elements from it’s cache by watching logs and visually. On a slow channel pictures appear immediately even if I clear Firefox cache.

Considering about a firewall in linux, I temporary deleted all rules on it. I’m not sure whether I cleared all of them? Can traffic drop somewhere?

That’s what basically I’m doing in console, when my server boots up:

modprobe ip_gre
ip tunnel add gre1 mode gre remote y.y.y.y local x.x.x.x dev eth0
ip addr add x.x.x.x/32 dev gre1
ip link set gre1 up
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 0 > /proc/sys/net/ipv4/conf/default/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth0/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/lo/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/gre1/rp_filter
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -t nat -A PREROUTING -i gre1 -p tcp -m tcp --dport 80 -j DNAT --to-destination x.x.x.x:3127
/etc/init.d/squid start

Note:

• x.x.x.x - ip-address of linux proxy with squid
• y.y.y.y - ip-address of my cisco router

I’ve got issues with Internet Explorer. I use IE 11. Occasionally, sometimes (usually) I have to hit Refresh Page button several times (like 5 or 6 times) to make page load from a remote server on a slow channel. The page usually doesn’t load from first time. I downloaded fiddler – http debugger. It shows ERROR 504 most of time.

I didn’t try other sites. Basically I’ve got one remote portal, and people work mostly with it.

What else did I try :

CISCO IOS

debug ip wccp events 
debug ip wccp packets

squid.conf

debug_options 80,3

I saw now errors in logs.

Question: how can I repair an issue with IE. I can’t get rid of IE, so I have to deal with it.

php-fpm is running:

# /etc/init.d/php5-fpm status
php5-fpm is running

php-fpm is listening:

# netstat -an | grep :9000
tcp        0      0 127.0.0.1:9000          0.0.0.0:*               LISTEN

Nginx config in /etc/nginx/nginx.conf:

# Nginx web server main configuration file: /etc/nginx/nginx.conf
#
user www-data;
worker_processes  8;
pid /var/run/nginx.pid;

events {
  multi_accept on;
}

http {
  default_type application/octet-stream;
  gzip on;
  gzip_disable "msie6";
  keepalive_timeout 65;
  sendfile on;
  tcp_nodelay on;
  tcp_nopush on;
  types_hash_max_size 8192;
  include /etc/nginx/mime.types;
  include /etc/nginx/conf.d/*.conf;
  include /etc/nginx/sites-enabled/*;
  access_log /var/log/nginx/access.log;
  error_log /var/log/nginx/error.log debug;
}

The second config is a shortcut in /etc/nginx/conf.d:

# Aegir web server main configuration file

#######################################################
###  nginx.conf main
#######################################################

 ## FastCGI params
 ## fastcgi_param  SCRIPT_FILENAME     $document_root$fastcgi_script_name;
  fastcgi_param  SCRIPT_FILENAME     $document_root;
  fastcgi_param  QUERY_STRING        $query_string;
  fastcgi_param  REQUEST_METHOD      $request_method;
  fastcgi_param  CONTENT_TYPE        $content_type;
  fastcgi_param  CONTENT_LENGTH      $content_length;
  fastcgi_param  SCRIPT_NAME         $fastcgi_script_name;
  fastcgi_param  REQUEST_URI         $request_uri;
  fastcgi_param  DOCUMENT_URI        $document_uri;
  fastcgi_param  DOCUMENT_ROOT       $document_root;
  fastcgi_param  SERVER_PROTOCOL     $server_protocol;
  fastcgi_param  GATEWAY_INTERFACE   CGI/1.1;
  fastcgi_param  SERVER_SOFTWARE     ApacheSolaris/$nginx_version;
  fastcgi_param  REMOTE_ADDR         $remote_addr;
  fastcgi_param  REMOTE_PORT         $remote_port;
  fastcgi_param  SERVER_ADDR         $server_addr;
  fastcgi_param  SERVER_PORT         $server_port;
  fastcgi_param  SERVER_NAME         $server_name;
  fastcgi_param  USER_DEVICE         $device;
  fastcgi_param  GEOIP_COUNTRY_CODE  $geoip_country_code;
  fastcgi_param  GEOIP_COUNTRY_CODE3 $geoip_country_code3;
  fastcgi_param  GEOIP_COUNTRY_NAME  $geoip_country_name;
  fastcgi_param  REDIRECT_STATUS     200;
  fastcgi_index  index.php;

 ## Default index files
  index         index.php index.html;

 ## Size Limits
  client_body_buffer_size        64k;
  client_header_buffer_size      32k;
  client_max_body_size          100m;
  large_client_header_buffers 32 32k;
  connection_pool_size           256;
  request_pool_size               4k;
  server_names_hash_bucket_size  512;
  server_names_hash_max_size    8192;
  types_hash_bucket_size         512;
  map_hash_bucket_size           192;
  fastcgi_buffer_size           128k;
  fastcgi_buffers             256 4k;
  fastcgi_busy_buffers_size     256k;
  fastcgi_temp_file_write_size  256k;

 ## Timeouts
  client_body_timeout             60;
  client_header_timeout           60;
  send_timeout                    60;
  lingering_time                  30;
  lingering_timeout                5;
  fastcgi_connect_timeout         60;
  fastcgi_send_timeout           300;
  fastcgi_read_timeout           300;

 ## Open File Performance
  open_file_cache max=8000 inactive=30s;
  open_file_cache_valid          60s;
  open_file_cache_min_uses         3;
  open_file_cache_errors          on;

 ## FastCGI Caching
  fastcgi_cache_path /var/lib/nginx/speed
                     levels=2:2:2
                     keys_zone=speed:10m
                     inactive=15m
                     max_size=3g;

 ## General Options
  ignore_invalid_headers          on;
  limit_conn_zone $binary_remote_addr zone=gulag:10m;
  recursive_error_pages           on;
  reset_timedout_connection       on;
  fastcgi_intercept_errors        on;
  server_tokens                  off;
  fastcgi_hide_header         'Link';
  fastcgi_hide_header  'X-Generator';
  fastcgi_hide_header 'X-Powered-By';
  fastcgi_hide_header 'X-Drupal-Cache';

 ## TCP options moved to /etc/nginx/nginx.conf

 ## SSL performance
  ssl_session_cache   shared:SSL:10m;
  ssl_session_timeout            10m;

 ## GeoIP support
  geoip_country /usr/share/GeoIP/GeoIP.dat;

 ## Compression
  gzip_buffers      16 8k;
  gzip_comp_level   5;
  gzip_http_version 1.0;
  gzip_min_length   10;
  gzip_types        text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;
  gzip_vary         on;
  gzip_proxied      any;
  add_header Vary "Accept-Encoding";
  gzip_static       on;
  upload_progress uploads 1m;

 ## Log Format
  log_format        main '"$proxy_add_x_forwarded_for" $host [$time_local] '
                         '"$request" $status $body_bytes_sent '
                         '$request_length $bytes_sent "$http_referer" '
                         '"$http_user_agent" $request_time "$gzip_ratio"';

  client_body_temp_path  /var/lib/nginx/body 1 2;
  access_log             /var/log/nginx/access.log main buffer=32k;
  error_log              /var/log/nginx/error.log crit;

# Extra configuration from modules:
#######################################################
###  nginx default maps
#######################################################

###
### Support separate Boost and Speed Booster caches for various mobile devices.
###
map $http_user_agent $device {
  default                                                                normal;
  ~*Nokia|BlackBerry.+MIDP|240x|320x|Palm|NetFront|Symbian|SonyEricsson  mobile-other;
  ~*iPhone|iPod|Android|BlackBerry.+AppleWebKit                          mobile-smart;
  ~*iPad|Tablet                                                          mobile-tablet;
}

###
### Set a cache_uid variable for authenticated users (by @brianmercer and @perusio, fixed by @omega8cc).
###
map $http_cookie $cache_uid {
  default  '';
  ~SESS[[:alnum:]]+=(?<session_id>[[:graph:]]+)  $session_id;
}

###
### Live switch of $key_uri for Speed Booster cache depending on $args.
###
map $request_uri $key_uri {
  default                                                                            $request_uri;
  ~(?<no_args_uri>[[:graph:]]+)\?(.*)(utm_|__utm|_campaign|gclid|source=|adv=|req=)  $no_args_uri;
}

###
### Set cache expiration depending on the Drupal core version.
###
map $sent_http_x_purge_level $will_expire_in {
  default   on-demand;
  ~*5|none  5m;
}

###
### Deny crawlers.
###
map $http_user_agent $is_crawler {
  default  '';
  ~*HTTrack|BrokenLinkCheck|2009042316.*Firefox.*3\.0\.10|MJ12|HTMLParser|libwww|PECL|Automatic|Click|SiteBot|BuzzTrack|Sistrix|Offline|Screaming|Nutch|Mireo|SWEB|Morfeus|GSLFbot  is_crawler;
}

###
### Deny all known bots on some URIs.
###
map $http_user_agent $is_bot {
  default  '';
  ~*crawl|goog|yahoo|yandex|spider|bot|tracker|click|parser  is_bot;
}

###
### Deny almost all crawlers under high load.
###
map $http_user_agent $deny_on_high_load {
  default  '';
  ~*crawl|goog|yahoo|yandex|baidu|bing|spider|tracker|click|parser  deny_on_high_load;
}

###
### Deny listed requests for security reasons.
###
map $args $is_denied {
  default  '';
  ~*delete.+from|insert.+into|select.+from|union.+select|onload|\.php.+src|system\(.+|document\.cookie|\;|\.\.  is_denied;
}

#######################################################
###  nginx default server
#######################################################

server {
  limit_conn   gulag 32; # like mod_evasive - this allows max 32 simultaneous connections from one IP address
  listen       *:80;
  server_name  _;
  location / {
     root   /var/www/nginx-default;
     index  index.html index.htm;
  }
}

#######################################################
###  nginx virtual domains
#######################################################

# virtual hosts
include /var/aegir/config/server_master/nginx/pre.d/*;
include /var/aegir/config/server_master/nginx/platform.d/*;
include /var/aegir/config/server_master/nginx/vhost.d/*;
include /var/aegir/config/server_master/nginx/post.d/*;

And my last config I added is located in /etc/nginx/conf.d/. Not a shortcut.

server {
listen   80; 


root /var/www/nginx-default/cacti;
index index.php index.html index.htm;


location / {
    try_files $uri $uri/ /index.html;
}

location /doc/ {
    alias /usr/share/doc/;
    autoindex on;
    allow 127.0.0.1;
    allow ::1;
    deny all;
}

error_page 500 502 503 504 /50x.html;
location = /50x.html {
    root /var/www/nginx-default/cacti;
}

location ~ \.php$ {
    try_files $uri =404;
    fastcgi_split_path_info ^(.+\.php)(/.+)$;

    fastcgi_pass 127.0.0.1:9000;

    fastcgi_index index.php;
    include fastcgi_params;
}

location ~ /\.ht {
    deny all;
    }
}