Neel's questions

I have a linux dedicated server that has 300GB of uploaded files that I need to transfer to AWS Storage S3 since I am now changing the uploads to be saved in S3 bucket instead of local disk. I read that I can do the transfer using aws cli command to copy the directory over to the S3 bucket. My questions are:

1. When I do the cp command from aws cli, how long roughly can it take for a dedicated server to transfer 300GB of data over to S3 bucket? Both S3 and the server are in the same region.

These are my server specs:

RAID Policy Raid 1
Operating System    Cloud Linux
HDD Bay 1   480GB SSD
HDD Bay 2   480GB SSD
Network Bandwidth   10TB
CPU 6 Core E5-2620v2 - 2.00Ghz x2
RAM 64 GB

I completely understand there are many variables, but want to get an idea on rough estimation from people who have migrated data from linux server to S3 storage.

2. When I use the aws cli cp command, does it show the progress during that time? What happens if I get disconnected from SSH when the command is still running?

3. Is it safer for me to run the aws cli cp command using screen command?

4. During the transfer, will the server performance take a hit? This server has couple of websites running so do I need to take the site offline when during the data transfer or can I safely run the transfer even when the sites are live?

I have installed a send only postfix. Postfix works when I send a mail to a full email address. However, if a mail to sent to a user, it adds the FQDN name as the extension instead of just the domain name I have set-up during installation.

For example, when I send a mail from ssh like this:

echo "This will go into the body of the mail." | mail -s "Hello world" root

I want the mail to go to [email protected]. But the email is sent to [email protected].

I have reviewed my host settings, mailname settings, postfix settings and I have no clue on why it keeps adding the full hostname to it which obviously is getting bounced. Here are my config files:

Postfix main.cf:

# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

myhostname = host.example.com.au
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination =
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = loopback-only

virtual_alias_maps = hash:/etc/postfix/virtual

Aliases:

mailer-daemon: postmaster
postmaster: root
nobody: root
hostmaster: root
usenet: root
news: root
webmaster: root
www: root
ftp: root
abuse: root
noc: root
security: root

/etc/postfix/virtual:

root            [email protected]

/etc/mailname:

example.com.au

/etc/hosts:

127.0.0.1       localhost
11x.0.0.xxx   host.example.com.au    host

/etc/hostname

host

I have reviewed everything and I dont have a clue on why mail send to @host.example.com.au. Can someone help me here please?

I am on debian 8.

EDIT: Just now, I tried adding one more setting to main.cf file: masquerade_domains = $mydomain. Adding this seems to fix the from address to show my domain name instead of hostname but yet, the to address is still having the FQDN name. I am receiving the bounced email "Mail Delivery System" notification on my email address since the origin email is now right after adding the masquerade_domains, but still cant get my head around why the user email is not respecting the same and the to address is still addressed as [email protected] instead of [email protected].

myhostname = host.example.com.au
mydomain = example.com.au
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = $mydomain
masquerade_domains = $mydomain
mydestination =
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128

I am using Mod_Pagespeed Filters for Nginx and the combine and minify javascipt or css is not working. I checked everything and could not find the reason:

1) There are no errors in the error log

2) Pagespeed works cuz its able to collapse_whitespace, remove_comments etc..

3) The cache folder permission seems to be fine with chown to www-data

4) Tried adding Domain and LoadFromFile to see if that makes a difference but no:

pagespeed Domain http://example.com;
pagespeed LoadFromFile "http://example.com/assets/"
                        "/var/www/example/public_html/assets";

5) I have authentication lock set in my nginx like this (see below). When I disable auth-lock, the minify seems to work sometimes. I didint get a steady results to come to that conclusion:

location / {
                        auth_basic "Login";
                        auth_basic_user_file /etc/nginx/login_pass;
}

So my question is, if nginx authentication is enabled, would that fail pagespeed from minifying and combining JS or CSS? Or could it be failing for me due to something else? I dont know where to look next and I am so lost...

This is my Pagespeed filters:

# PageSpeed
# Enable ngx_pagespeed
pagespeed on;
pagespeed FileCachePath /usr/share/nginx/pagespeed;

# Ensure requests for pagespeed optimized resources go to the pagespeed handler
# and no extraneous headers get set.
location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" {
  add_header "" "";
}
location ~ "^/ngx_pagespeed_static/" { }
location ~ "^/ngx_pagespeed_beacon$" { }
location /ngx_pagespeed_statistics { allow 127.0.0.1; deny all; }
location /ngx_pagespeed_global_statistics { allow 127.0.0.1; deny all; }
location /ngx_pagespeed_message { allow 127.0.0.1; deny all; }



pagespeed EnableFilters collapse_whitespace;
pagespeed EnableFilters remove_comments;

# Defer and minify Javascript
# pagespeed EnableFilters defer_javascript;
pagespeed EnableFilters rewrite_javascript;
pagespeed EnableFilters combine_javascript;
pagespeed EnableFilters canonicalize_javascript_libraries;

# Inline and minimize css
pagespeed EnableFilters rewrite_css;
pagespeed EnableFilters fallback_rewrite_css_urls;
pagespeed EnableFilters combine_css;

UPDATE: It appears the authentication lock is not the one stopping pagespeed from combining and minimizing. I am thinking if any of the included JS is causing Pagespeed to not minify all the files if thats possible. There are about 6 CSS and 6 JS files to minify and combine. Still not sure why and looking into it.

SOLUTION: Turns out Pagespeed was unable to locate my static assets and hence it wasnt able to minify and combine the css. Mapping the static assets worked. Have added my answer below.

Having a very hard to trying to figure this out. I have changed my website from another platform to Joomla and now Nginx is unable to handle old urls.

My Old Urls were like this:

example.com/home.php
example.com/contact-us.php

My New Joomla SEF Urls are like this:

example.com/home
example.com/contact-us

I have the following Nginx config as per Joomla guide:

 location / {             
              try_files     $uri $uri/ /index.php?$args;
        }

 # Process PHP
 location ~ \.php$ {
            try_files $uri =404;
            fastcgi_split_path_info ^(.+\.php)(/.+)$;

            fastcgi_pass   unix:/var/run/php5-fpm.sock;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
            include        fastcgi_params;
 }

I want Nginx to pass these old urls to Joomla to handle it. Now, what is happening is, Nginx is treating these old urls as php files and then showing me this No input file specified. error. I then changed the try_files inside the php block to try_files $uri /index.php?$args; so my Nginx config looks like this:

 location / {             
              try_files     $uri $uri/ /index.php?$args;
        }

 # Process PHP
 location ~ \.php$ {
            try_files       $uri /index.php?$args;
            fastcgi_split_path_info ^(.+\.php)(/.+)$;

            fastcgi_pass   unix:/var/run/php5-fpm.sock;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
            include        fastcgi_params;
 }

Is this valid? Would this cause an infinite loop issues in some cases? Is this is right way to do this? I didint find any solution similar to this. Can someone guide me please?

I have set-up the RDNS from my hosting provider for the ip address to resolve to "maximus.example.com". After that, do I also need to add the PTR records to the SPF Text details in my DNS zone file as well like this?

v=spf1 include:_spf.google.com ip4:111.11.11.111 ptr:maximus.example.com ~all

My question: Is the above addition of PTR details to SPF record necessary or is it not needed when the host already set-up the RDNS for my ip address?

I was going to add the record to my SPF and I then read this somewhere that this is strongly discouraged.

Note: The ptr sender mechanism is strongly discouraged by RFC 7208 which even goes so far as to suggest its immediate removal for performance reasons since it places a load on the IN-ADDR.ARPA (IPv4) or IP6.ARPA reverse-map domains which generally have less capacity than the gTLD and ccTLD domains.

REF: http://www.zytrax.com/books/dns/ch9/spf.html#ptr

I am not sure if I have misunderstood what it meant. So can someone please clarify this for me please since I am unsure whether I need to add PTR to my zone file or not..

I've got nginx on Debian 7 without cpanel.

I am seting-up my Crontab like this:

*/45 * * * * wget "http://example.com/cron-url.php" >/dev/null 2>&1

The above cron is being blocked with 403 forbidden:

--2014-12-10 05:40:01--  http://example.com/cron-url.php
Connecting to xyz.xx.xx.xxx:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2014-12-10 05:40:01 ERROR 403: Forbidden.

After searching here and googling, I understand that my server is probably blocking wget. I had a look at my nginx configuration file and I think it should be due to this:

if ($http_user_agent ~* LWP::Simple|BBBike|wget) {
    return 403;
}

For cron command, I have to use the url to it and absolute link to script path doesn't work. Now, what do I do if I need to allow cron to work without getting blocked? I was thinking maybe I need to allow wget from my own server, but don't know how to do that with nginx. Can someone help me resolve this cron issue?

I have already re-read the docs on this as well as other posts here and this is still very unclear to me. I have been testing various things to understand the difference between alias_maps and virtual_alias_maps and I don't see the use of these 2 separate settings in postfix. This is what I found so far (Note - I am using postfix in the same server as my web server as null client to send emails only):

1) /etc/aliases file:

root: [email protected]

When I add the above to the alias_maps, I noticed that some services like fail2ban are able to pick this and it sends root emails to the alias email addresses mentioned. However, I also noticed that some other services (like mail command) does not respect this and tries to send the email directly to [email protected] which does not exist (I think its the postfix myorigin setting that is adding the @mydomain.com). To fix this I then added the virtual_alias_maps

2) /etc/postfix/virtual

root     [email protected]

When the above is added, all services uses this virtual aliases email. I also noticed that once I add the above, even fail2ban begins to ignore my initial settings in /etc/aliases/ file and starts to follow the email address given in virtual file.

Now this has confused me even more -

1. Why do we need /etc/aliases/ when having the email inside virtual aliases map seems to override it?

2. What is the purpose of having these 2 separate aliases mapping and when do we decide when to use what?

3. Why did fail2ban (which is configured to email to root@localhost) first follow email address given in alias_maps (/etc/aliases/) and later decides to ignore that once virtual_alias_maps was added?

4. Why doesn't all services read email aliases mentioned in /etc/aliases and they only work when the email aliases are added in virtual alias map?

I have spend several hours since yesterday and still unsure. Can someone help me clear my confusion?

EDIT: This is the mail log when email is sent to root using mail root command. The aliases email for root is mentioned in /etc/aliases/. But mail does not work until I move this root aliases email from aliases_maps to virtual_aliases_maps

Log when root email alias is mentioned in /etc/aliases/:

Nov 14 16:39:27 Debian postfix/pickup[4339]: 0F12643432: uid=0 from=<root>

Nov 14 16:39:27 Debian postfix/cleanup[4495]: 0F12643432: message-id=<[email protected]>

Nov 14 16:39:27 Debian postfix/qmgr[4338]: 0F12643432: from=<[email protected]>, size=517, nrcpt=1 (queue active)

Nov 14 16:39:27 Debian postfix/error[4496]: 0F12643432: to=<[email protected]>, orig_to=<root>, relay=none, delay=0.04, delays=0.03/0/0/0.01, dsn=4.4.1, status=deferred (delivery temporarily suspended: connect to domainname.com[128.199.147.136]:25: Connection refused)

This is the log after the email aliases for root is moved from /etc/aliases/ to /etc/postfix/virtual where the email delivery is successful after the change:

Nov 14 16:44:58 Debian postfix/pickup[4545]: ADD9A43436: uid=0 from=<root>

Nov 14 16:44:58 Debian postfix/cleanup[4563]: ADD9A43436: message-id=<[email protected]>

Nov 14 16:44:58 Debian postfix/qmgr[4544]: ADD9A43436: from=<[email protected]>, size=453, nrcpt=1 (queue active)

Nov 14 16:45:00 Debian postfix/smtp[4551]: ADD9A43436: to=<[email protected]>, orig_to=<root>, relay=somesite.com[108.160.157.120]:25, delay=1.9, delays=0.03/0/0.97/0.88, dsn=2.0.0, status=sent (250 OK id=1XpEqC-0002ry-9s)

Nov 14 16:45:00 Debian postfix/qmgr[4544]: ADD9A43436: removed

This is driving me nuts and I dont understand how this works automatically. I have nginx as my web server and I have installed Joomla that has URL Rewriting which removes the index.php from the URL. Before, when I was using Apache, I have to enable the .htaccess with RewriteEngine On in order for this to work. But with Nginx, when I enable "Use URL Rewriting" it works automatically. I only use Nginx that passes the php files to php-fpm. Thats all. I havent added any special rewrite rule other than what was given in Joomla docs. I dont understand how 'Use URL Rewriting' just works automatically when I enable it since there is no .htaccess for Nginx.

The Joomla docs on this topic didint help neither. In second step it says

Enable the Use Apache mod_rewrite/URL rewriting option and Save: This option uses the Apache mod_rewrite function to eliminate the "index.php" portion of the URL.

.....

If this option causes errors, please see How to check if mod rewrite is enabled on your server. If it is not enabled and you have access to the file apache/conf/httpd.conf, open that file and check if the line LoadModule rewrite_module modules/mod_rewrite.so is uncommented. If necessary, uncomment the line and restart the Apache web server.

No idea why this is added in an Nginx configuration since there is no mod_rewrite in Nginx. The URL Rewriting in Joomla backend says this:

Use URL rewriting Select to use a server's rewrite engine to catch URLs that meet specific conditions and rewrite them as directed. Available for IIS 7 and Apache. Apache users only! Rename htaccess.txt to .htaccess before activating. Rename web.config.txt to web.config and install IIS URL Rewrite Module before activating.

It says nothing about Nginx but still it works. I am scratching my head here. Can someone tell me how the Joomla's index.php is removed so easily in Nginx? This is my Nginx Vhost configuration:

server {

        listen 80;
        server_name example.com;
        root /var/www/example/public_html;
        index  index.php index.html index.htm default.html default.htm;

        access_log /var/log/nginx/accn_access.log;
        error_log /var/log/nginx/accn_error.log;

        ##
        # JOOMLA SEF
        ##

        location / {
              try_files   $uri $uri/ /index.php?q=$request_uri;
        }

        ##
        # PHP scripts to FastCGI 
        ##
        location ~ \.php$ {
            try_files $uri =404;
            fastcgi_split_path_info ^(.+\.php)(/.+)$;

            fastcgi_pass   unix:/var/run/php5-fpm.sock;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
            include        fastcgi_params;

        }

}

See.. its a pretty straight forward configuration. Where is the magic happening?

I keep getting this error in nginx/error.log and its driving me nuts:

8096 worker_connections exceed open file resource limit: 1024

I've tried everything I can think of and cant figure out what is limiting nginx here. Can you tell what am I missing?

nginx.conf has this:

worker_processes 4;
events {
        worker_connections 8096;
        multi_accept on;
        use epoll;
}

I changed my system's Ulimit in security/limits.conf like this:

# This is added for Open File Limit Increase
*               hard    nofile          199680
*               soft    nofile          65535

root            hard    nofile          65536
root            soft    nofile          32768

# This is added for Nginx User
nginx           hard    nofile          199680
nginx           soft    nofile          65535

It was still showing the error. So I also tried editing /etc/default/nginx and added this line:

ULIMIT="-n 65535"

It is still showing the same error. Can't figure out what is limiting the nginx worker connection to only 1024. Can you point me out?

I've got Debian 7 + nginx

In my log, I am frequently seeing dropped ips like this:

> Oct 30 17:32:24 IPTables Dropped: IN=eth0 OUT=
> MAC=04:01:2b:bd:b0:01:4c:96:14:ff:df:f0:08:00 SRC=62.210.94.116
> DST=128.199.xxx.xxx LEN=40 TOS=0x00 PREC=0x00 TTL=244 ID=45212
> PROTO=TCP SPT=51266 DPT=5900 WINDOW=1024 RES=0x00 SYN URGP=0
> 
> Oct 30 17:29:57 Debian kernel: [231590.140175] IPTables Dropped:
> IN=eth0 OUT= MAC=04:01:2b:bd:b0:01:4c:96:14:ff:ff:f0:08:00
> SRC=69.30.240.90 DST=128.199.xxx.xxx LEN=40 TOS=0x00 PREC=0x00 TTL=245
> ID=12842 DF PROTO=TCP SPT=18534 DPT=8061 WINDOW=512 RES=0x00 SYN
> URGP=0

From the above, I am assuming these are the Syn flood that are being dropped by my IpTables rules. This is what I have in iptables for Syn (although not sure which one of these rules are dropping the ones above):

# Drop bogus TCP packets
iptables -A INPUT -p tcp -m tcp --tcp-flags SYN,FIN SYN,FIN -j DROP
iptables -A INPUT -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP

# --- Common Attacks: Null packets, XMAS Packets and Syn-Flood Attack ---
iptables -A INPUT -p tcp --tcp-flags ALL NONE -j DROP
iptables -A INPUT -p tcp --tcp-flags ALL ALL -j DROP
iptables -A INPUT -p tcp ! --syn -m state --state NEW -j DROP

In Fail2ban, I dont see any specific filter for Syn attacks in filter.d folder. My question for this are:

1) Do I just ignore the above logs and not worry about setting up a Fail2Ban filter for these since its internet and there is constantly going to be script kiddies doing these anyways?

2) Since Fail2ban work based on iptables log, is there a way to ban the above Syn attempts on my server?

This is my lame attempt on a filter and its not working. Not sure if its even valid:

[Definition]
failregex = ^<HOST> -.*IPTables Dropped:.*SYN URGP=0
ignoreregex =

I am using Debian + Nginx

I have a web server where I have installed Postfix for sending emails only. All incoming mails will be handled by google apps and not my web server. Using iptables rules, I have blocked all incoming ports (including Port 25) and opened only ssh and http ports. All Outbound traffic are open. From testing I can see that postfix is able to send emails even when port 25 is blocked so that is not an issue. My understanding was Port 25 is only needed to be opened if I am receiving emails on that server. But I read this in coding horror site that has confused me a bit:

Port25 offers a really nifty public service -- you can send email to [email protected] and it will reply to the from: address with an extensive diagnostic!

So my question is, does some mail server ping my Port 25 to check the authenticity of my server origin when Postfix sends email to them? If Postfix is used to send emails only, do I still need to open Port 25 so other mail servers can verify my server info? I am concerned if I block port 25, is that another reason for some email clients to mark my emails as Spam because they are unable to get info on my server by pinging to Port 25? I cant find info on this. I will be obliged if someone can clarify this plz...

In /etc/fail2ban/jail.local, when I have the MTA setting to Postfix, Fail2Ban is showing an error and is unable to start.

[DEFAULT]

ignoreip = 127.0.0.1/8
bantime  = 1800                         
maxretry =  4                           
destemail = [email protected]   
mta = postfix                           
action = %(action_mwl)s 

Error:

WARNING 'findtime' not defined in 'ssh'. Using default value
ERROR  /etc/fail2ban/action.d/postfix-whois-lines.conf and /etc/fail2ban/action.d/postfix-whois-lines.local do not exist
ERROR  Error in action definition postfix-whois-lines[name=ssh, dest="[email protected]", logpath=/var/log/auth.log, chain="INPUT"]
ERROR  Errors in jail 'ssh'. Skipping...
['set', 'loglevel', 3]
['set', 'logtarget', '/var/log/fail2ban.log']

The error is pointing somewhere else, but when I comment out mta=postfix it works.

I have Postfix installed as my MTA (in Debian 7) and I thought I am supposed to change default sendmail to postfix for this setting. Am I wrong?

What does Nginx Proxy Filter in Fail2ban really do?

I am currently setting up Fail2Ban for Nginx. Almost everywhere I've read, they all have included a setting for [nginx-proxy] proxy filter. From the examples, I can see how this filter bans the ipaddress that does things like these in the log:

58.218.204.110 - - [06/Mar/2011:08:04:42 -0800] "GET http://www.shopsline.com/proxyheader.php HTTP/1.1" 404 505 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [06/Mar/2011:11:01:33 -0800] "GET http://www.foodnese.com/indux.php HTTP/1.1" 404 498 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [06/Mar/2011:16:00:18 -0800] "GET http://98.126.64.106/judge123.php HTTP/1.1" 403 502 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.204.110 - - [06/Mar/2011:16:19:08 -0800] "GET http://98.126.64.106/judge123.php HTTP/1.1" 403 502 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [06/Mar/2011:18:30:13 -0800] "GET http://www.travelimgusa.com/ip.php HTTP/1.1" 404 499 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [06/Mar/2011:23:27:23 -0800] "GET http://www.seektwo.com/proxy-1.php HTTP/1.1" 404 499 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.204.110 - - [07/Mar/2011:00:36:45 -0800] "GET http://www.eduju.com/proxyheader.php HTTP/1.1" 404 501 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [07/Mar/2011:04:27:13 -0800] "GET http://www.eduju.com/proxyheader.php HTTP/1.1" 404 501 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [07/Mar/2011:09:24:13 -0800] "GET http://98.126.64.106/judge123.php HTTP/1.1" 403 502 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [07/Mar/2011:14:25:03 -0800] "GET http://www.foodnese.com/indux.php HTTP/1.1" 404 498 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [07/Mar/2011:16:54:09 -0800] "GET http://www.foodnese.com/indux.php HTTP/1.1" 404 498 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [07/Mar/2011:19:21:51 -0800] "GET http://www.racross.com/proxyheader.php HTTP/1.1" 404 503 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [08/Mar/2011:05:19:50 -0800] "GET http://piceducation.com/proxyheader.php HTTP/1.1" 301 605 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [08/Mar/2011:15:15:22 -0800] "GET http://www.piggmail.com/proxyheader.php HTTP/1.1" 404 504 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [08/Mar/2011:17:43:58 -0800] "GET http://www.cjpjp.com/proxyheader.php HTTP/1.1" 404 501 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
58.218.199.147 - - [08/Mar/2011:20:14:15 -0800] "GET http://98.126.64.106/judge123.php HTTP/1.1" 403 502 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

But what I dont understand is, what are these guys really trying to do? Are these ip addresses trying to use my server to fetch another site for them? How does these proxy attacks work? I cant seem to find much info on how my server can be used as proxy and why I need the proxy filter for.

Does jail.local file act as an override to jail.conf or as a replacement to jail.conf?

When I was learning about Fail2Ban from tutorials, most of them usually say either to copy jail.conf to jail.local and make the edits there and some of them says to create a new jail.local file and gives a bunch of settings to copy and paste. But what they don't address is how does jail.local work with jail.conf. These are the 2 scenarios:

1. Override: If jail.local acts as an override to jail.conf file then all I need to do is only add the necessary configs that I want to override to the default given in jail.conf. In this case I don't need to add SSH config etc.. since it is already included in jail.conf.

2. Replacement: If jail.conf becomes invalid when jail.local is present, then I need to add all the rules in jail.local and then edit the ones I want modified.

Can you confirm what happens to jail.conf when jail.local is present please? If jail.local acts like just an override on top of jail.conf file, then it is easier for me to just have just those few lines of rules that I want to add to it which also makes it easy for maintaining and readability. What is the best approach on this?

Can I use both Memcache and Memcached php extensions on the same server at the same time? Just to be clear so you dont misunderstand what I mean by this question, here is my scenario: I have both Joomla and phpbb for my website. Joomla provides support for both memcached and memcache where as phpbb only provides support for memcache. So I am planning to do this:

1) Install memcached server: apt-get install memcached

2) Install php-memcache to use for phpbb: apt-get install php5-memcache

3) Install php-memcached to use for Joomla: apt-get install php5-memcached

My questions are:

1. Is it okay to have both memcache and memcached extensions installed and enabled where each of these are used by different php applications at the same time? Would this conflict in any ways?

2. When I set the memory limit for the memcached server to 512M in /etc/memcached.conf, does it mean that this memory limit is going to be shared between both memcache and memcached extension? This shouldnt be a problem if say memcache uses more memory than memcached or vice versa? Do they both play nice to each other?

3. In the scenario like this, should I try to give preference to only 1 php extension (like either memcache only in my case) to keep things simple without conflicts between the two or it shouldnt matter. What I mean is, if you say it is okay to use both (for Q1), then out of professional practice, even if both can be used at the same time, should I still try to use the same php extension for both Joomla and phpbb if possible to minimize headaches? Would that make difference in any ways?

Most CMS (Joomla for example) comes with php mail() as default and has the option of changing it to SMTP if we want. I will be setting up both the webserver and mail server (Postfix) in the same VPS on Debian 7. What I am trying to understand is:

1. Delivery Reputation: Would using php mail() vs SMTP make any difference with the delivery rate? From what I understand, Postfix is going to be using SMTP protocol when communicating with other servers on the internet when delivering emails, so therefore is it safe to assume that it doesnt matter if the email was sent to postfix via php mail or smpt from localhost? Does any of these 2 methods affect the email reputation is any ways even in a small way?

2. Performance Load: Is there any performance difference between php mail and SMTP when sending emails? Say, if I am sending like 10,000 emails, which method would take the most resources (or time)? My assumption is that both can take a bit of time like: php mail for compiling the emails with headers, etc.. and SMTP for making connections each time. Which one consumes the most server resources?

3. Security Issue: When I searched for the difference between the two, many sites says about the security issue with php mail since a hacker can upload php script to send out spams. But I can also see another security issue with SMTP as well since the SMTP username and passwords are stored in configuration file in text which is not secure neither. Since there are security issues for both methods, does one weigh better than the other in any aspects?

4. Preference: If I am able to set up both phpmail and SMTP on my server, should I give preference to use one over the other for any reason? I mean, if I can use SMTP in my server, should I then try to use SMTP instead of phpmail due to various reasons covered in the above points?

Background: My machine is going to be a simple web server that uses postfix just for sending out emails from Joomla, newsletters and root email from various services.

This might sound stupid and basic to some of you, but I am still so confused about this even after reading numerous sources on the net. My questions still unclear are:

1. Is the Hostname in machine (for example when typing hostname in debian OS ssh) different from Hostname that you type in DNS server A Record?

2. Is it normal to have my Debian Machine hostname set as maximus and have the DNS A Record Hostname set as mail?

3. Are these 2 the same that I am confused thinking of them as separate or are they usually used in interchangeable term causing the confusion?

4. When it comes to FQDN, for the host part, does it use the hostname of the machine or the hostname given in the DNS A record?

My setup: 1 Debian 7 VPS with web server and mail server both on the same box.

I am trying to compile Nginx with Pagespeed from source and I am having a hard time trying to resolve the not found errors I am seeing when I add the configure settings. This is my configure settings:

I am using CentOS 7 - 64 bit

Configure Settings:

sudo ./configure --add-module=/nginx-compile/modules/ngx_pagespeed-release-1.9.32.1-beta \
--prefix=/etc/nginx \
--sbin-path=/usr/sbin/nginx \
--conf-path=/etc/nginx/nginx.conf \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--pid-path=/var/run/nginx.pid \
--lock-path=/var/run/nginx.lock \
--http-client-body-temp-path=/var/cache/nginx/client_temp \
--http-proxy-temp-path=/var/cache/nginx/proxy_temp \
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \
--http-scgi-temp-path=/var/cache/nginx/scgi_temp \
--with-http_ssl_module \
--with-http_realip_module \
--with-http_addition_module \
--with-http_sub_module \
--with-http_dav_module \
--with-http_flv_module \
--with-http_mp4_module \
--with-http_gunzip_module \
--with-http_gzip_static_module \
--with-http_random_index_module \
--with-http_secure_link_module \
--with-http_stub_status_module \
--with-http_auth_request_module \
--with-mail \
--with-mail_ssl_module \
--with-file-aio \
--with-ipv6 \
--with-http_spdy_module \
--with-cc-opt='-O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector \ 
--param=ssp-buffer-size=4 -m64 -mtune=generic' \
--user=nginx \
--group=nginx

When I add the above in ssh, I am getting the following not found errors and no supported file AIO was found error. Here is the list I get:

Errors after executing Configure:

checking for OS
 + Linux 3.10.0-123.el7.x86_64 x86_64
checking for C compiler ... found
 + using GNU C compiler
 + gcc version: 4.8.2 20140120 (Red Hat 4.8.2-16) (GCC)
checking for gcc -pipe switch ... found
checking for gcc builtin atomic operations ... not found
checking for C99 variadic macros ... not found
checking for gcc variadic macros ... not found
checking for unistd.h ... found
checking for inttypes.h ... found
checking for limits.h ... found
checking for sys/filio.h ... not found
checking for sys/param.h ... found
checking for sys/mount.h ... found
checking for sys/statvfs.h ... found
checking for crypt.h ... found
checking for Linux specific features
checking for epoll ... not found
checking for O_PATH ... not found
checking for sendfile() ... not found
checking for sendfile64() ... not found
checking for sys/prctl.h ... found
checking for prctl(PR_SET_DUMPABLE) ... not found
checking for sched_setaffinity() ... not found
checking for crypt_r() ... not found
checking for sys/vfs.h ... found
checking for poll() ... not found
checking for /dev/poll ... not found
checking for kqueue ... not found
checking for crypt() ... not found
checking for crypt() in libcrypt ... not found
checking for F_READAHEAD ... not found
checking for posix_fadvise() ... not found
checking for O_DIRECT ... not found
checking for F_NOCACHE ... not found
checking for directio() ... not found
checking for statfs() ... not found
checking for statvfs() ... not found
checking for dlopen() ... not found
checking for dlopen() in libdl ... not found
checking for sched_yield() ... not found
checking for sched_yield() in librt ... not found
checking for SO_SETFIB ... not found
checking for SO_ACCEPTFILTER ... not found
checking for TCP_DEFER_ACCEPT ... not found
checking for TCP_KEEPIDLE ... not found
checking for TCP_FASTOPEN ... not found
checking for TCP_INFO ... not found
checking for accept4() ... not found
checking for kqueue AIO support ... not found
checking for Linux AIO support ... not found

./configure: no supported file AIO was found
Currently file AIO is supported on FreeBSD 4.3+ and Linux 2.6.22+ only

I also installed the following dependencies before I added the configure settings:

sudo yum install gcc gcc-c++ pcre-devel zlib-devel make unzip openssl-devel

The configuration I used is similar to the configuration settings that comes when I install Nginx using centos inbuilt repo except it has the pagespeed module added.

UPDATE:

The actual problem was the way I've split the line when passing the --params to the configure call which is incorrect and was causing the errors. Thanks to @XavierLucas answer below, the error was resolved once I added --with-cc-opt and --params in 1 line.

EXTENDING:

While the above was the cause for error, I also learnt that its good to have the following libraries installed too in order for Nginx to make the most out of its core features and while having pagespeed module installed:

gcc 
gcc-c++ 
pcre-devel 
zlib-devel 
make 
unzip 
openssl-devel 
libaio-devel
glibc 
glibc-devel 
glibc-headers
libevent
linux-vdso.so.1
libpthread.so.0
libcrypt.so.1
libstdc++.so.6
librt.so.1
libm.so.6
libpcre.so.0
libssl.so.10
libcrypto.so.10
libdl.so.2
libz.so.1
libgcc_s.so.1
libc.so.6
/lib64/ld-linux-x86-64.so.2
libfreebl3.so
libgssapi_krb5.so.2
libkrb5.so.3
libcom_err.so.2
libk5crypto.so.3
libkrb5support.so.0
libkeyutils.so.1
libresolv.so.2
libselinux.so.1

And

yum groupinstall 'Development Tools'

I have a LEMP set-up in CentOS 7. I am planning to install the ngx_pagespeed module to it. When I was reading through the docs I came to realise that unlike Apache, I cannot install this module dynamically with Nginx. So from my understanding, I now need to uninstall the current Nginx, compile Nginx with ngx_pagespeed and re-install this compiled version. This comes to my next question: What happens if I have to upgrade Nginx or ngx_pagespeed module in the future and I already have a working production LEMP server with ngx_pagespeed? Do I then need to do the same i.e uninstall Nginx + ngx_pagespeed first, compile the latest version of the module & nginx and reinstall the newer version? It seems to be a big hassle. Isint there a better or simpler way to do this?

I am very new to to server administration and just discovered that I can use Google Compute Engine to host my website similar to way it works with Linode or Digital Ocean. I am unsure on the following questions related firewall set-up with Google Compute Engine:

1) I see that Google Compute Engine comes with a firewall setting we can use for each instance. So in this case, does it mean I dont need to open and close ports in iptable as well when I set up a CentOs with Nginx web server on GCE?

2) If Google compute engine does the same job as iptables, then do I need to setup any special firewall rules for Blocking Null packets, Reject Syn-Flood Attack, Reject XMAS Packets, etc.. to GCE Firewall or is that not needed?