petermolnar's questions

I got into an argument on the net.core.somaxconn parameter: I was told that it will not make any difference if we change the default 128.

I believed this might be enough proof:

"If the backlog argument is greater than the value in /proc/sys/net/core/somaxconn, then it is silently truncated to that value" http://linux.die.net/man/2/listen

but it's not.

Does anyone know a method to testify this with two machines, sitting on a Gbit network? The best would be against MySQL, LVS, apache2 ( 2.2 ), memcached.

According to Mongo documentation in order to safely deploy a replication set, you need at least two active and one arbiter because election of the a primary needs the majorities of the votes.

Let's say I'm able to have three machines, therefore deploy three full-blown mongo instances, no arbiter.

If the elected primary fails I end up with TWO nodes, where both of them has the same "power" level: for me this seems to be the must-aviod situation described at the deploy.

Can someone unfold why is it not a problem to elect a primary in this situation when it is if the initial setup is the same?

I'm trying to get postscreen work. I've followed the man page and it's already running correctly for smtp.

But it I want to use it for smtps as well (adding the same line as smtp in master.cf but with smtps) i receive failure messages in syslog like:

postfix/postscreen[8851]: fatal:
btree:/var/lib/postfix/postscreen_cache: unable to get exclusive lock:
Resource temporarily unavailable

Some say that postscreen can only run once; that's ok. But can I use the same postscreen session for both smtp and smtps? If not, how to enable postscreen for smtps as well?

Any help would be apprecieted!

The parts of the configs:

main.cf postscreen_access_list = permit_mynetworks, cidr:/etc/postfix/postscreen_access.cidr

postscreen_dnsbl_threshold = 8
postscreen_dnsbl_sites = dnsbl.ahbl.org*3 dnsbl.njabl.org*3 dnsbl.sorbs.net*3 pbl.spamhaus.org*3 cbl.abuseat.org*3 bl.spamcannibal.org*3 nsbl.inps.de*3 spamrbl.imp.ch*3
postscreen_dnsbl_action = enforce
postscreen_greet_action = enforce

master.cf (full)

smtpd     pass  -       -       n       -       -       smtpd
smtp      inet  n       -       n       -       1       postscreen
tlsproxy  unix  -       -       n       -       0       tlsproxy
dnsblog   unix  -       -       n       -       0       dnsblog

### the problematic line ###
smtps     inet  n       -       -       -       -       smtpd

pickup    fifo  n       -       -       60      1       pickup
cleanup   unix  n       -       -       -       0       cleanup
qmgr      fifo  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       -       1000?   1       tlsmgr
rewrite   unix  -       -       -       -       -       trivial-rewrite
bounce    unix  -       -       -       -       0       bounce
defer     unix  -       -       -       -       0       bounce
trace     unix  -       -       -       -       0       bounce
verify    unix  -       -       -       -       1       verify
flush     unix  n       -       -       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       -       -       -       smtp
relay     unix  -       -       -       -       -       smtp
showq     unix  n       -       -       -       -       showq
error     unix  -       -       -       -       -       error
retry     unix  -       -       -       -       -       error
discard   unix  -       -       -       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       -       -       -       lmtp
anvil     unix  -       -       -       -       1       anvil
scache    unix  -       -       -       -       1       scache

dovecot   unix  -       n       n        -      -       pipe
  flags=DRhu user=virtuser:virtuser argv=/usr/bin/spamc -e /usr/lib/dovecot/deliver -d ${recipient} -f {sender}

I have a VPS which is originally destined to be a webserver but some minimal mail capabilities are needed to be deployed as well, including sending and receiving as standalone server.

The current setup is the following:

• Postfix reveices the mail, the users are in virtual tables, stored in MySQL
• on connection all servers are tested with policyd-weight service against some DNSBLs
• all mail is runs through SpamAssassin spamd with the help of spamc client
• the mail is then delivered with Dovecot 2' LDA (local delivery agent), virtual users as well

As you saw...

• there's no virus scanner running, and that's for a reason: clamav eats all the memory possible and also, virus mails are all filtered out with this setup (I've tested the same with ClamAV enabled for 1,5 years, no virus mail ever got even to ClamAV)
• I don't use amavisd and I really don't want to. You only need that monster if you have plenty of memory and lots of simultaneous scanners. It's also a nightmare to fine tune by hand.
• I run policyd-weight instead of policyd and native DNSBLs in postfix. I don't like to send someone away because a single service listed them.

Important statement: everything works fine. I receive very small amount of spam, nearly never get a false positive and most of the bad mail is stopped by policyd-weight. The only "problem" that I feel the services at total uses a bit much memory alltogether.

I've already cut the modules of spamassassin (see below), but I'd really like to hear some advices how to cut the memory footprint as low as possible, mostly: what plugins SpamAssassin really needs and what are more or less useless, regarding to my current postfix & policyd-weight setup?

SpamAssassin rules are also compiled with sa-compile (sa-update runs once a week from cron, compile runs right after that)

These are some of the current configurations that may matter, please tell me if you need anything more.

postfix/master.cf (parts only)

dovecot   unix  -       n       n        -      -       pipe
  flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e /usr/lib/dovecot/deliver -d ${recipient} -f {sender}

postfix/main.cf (parts only)

smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks,
        reject_invalid_hostname,
        permit

smtpd_recipient_restrictions =  permit_mynetworks,
        permit_sasl_authenticated,
        reject_invalid_hostname,
        reject_non_fqdn_hostname,
        reject_non_fqdn_recipient,
        reject_unknown_recipient_domain,
        reject_unauth_pipelining,
        reject_unauth_destination,
        check_policy_service inet:127.0.0.1:12525,
        permit

policyd-weight.conf (parts only)

$REJECTMSG = "550 Mail appeared to be SPAM or forged. Ask your Mail/DNS-Administrator to correct HELO and DNS MX settings or to get removed from DNSBLs";

$REJECTLEVEL = 4;
$DEFER_STRING = 'IN_SPAMCOP= BOGUS_MX=';
$DEFER_ACTION = '450';
$DEFER_LEVEL  = 5;
$DNSERRMSG = '450 No DNS entries for your MTA, HELO and Domain. Contact YOUR administrator';

# 1: ON, 0: OFF (default)
# If ON request that ALL clients are only checked against RBLs
$dnsbl_checks_only = 0;

# 1: ON (default), 0: OFF
# When set to ON it logs only RBLs which affect scoring (positive or negative)
$LOG_BAD_RBL_ONLY  = 1;

## DNSBL settings
@dnsbl_score = (
        # host,                 hit,    miss,   log name
        'dnsbl.ahbl.org',       3,      -1,     'dnsbl.ahbl.org',
        'dnsbl.njabl.org',      3,      -1,     'dnsbl.njabl.org',
        'dnsbl.sorbs.net',      3,      -1,     'dnsbl.sorbs.net',
        'bl.spamcop.net',       3,      -1,     'bl.spamcop.net',
        'zen.spamhaus.org',     3,      -1,     'zen.spamhaus.org',
        'pbl.spamhaus.org',     3,      -1,     'pbl.spamhaus.org',
        'cbl.abuseat.org',      3,      -1,     'cbl.abuseat.org',
        'list.dsbl.org',        3,      -1,     'list.dsbl.org',
);

# If Client IP is listed in MORE DNSBLS than this var, it gets REJECTed immediately
$MAXDNSBLHITS  = 3;

# alternatively, if the score of DNSBLs is ABOVE this level, reject immediately
$MAXDNSBLSCORE = 9;

$MAXDNSBLMSG = '550 Az levelezoszerveruk IP cime tul sok spamlistan talahato, kerjuk ellenorizze! / Your MTA is listed in too many DNSBLs; please check.';

## RHSBL settings
@rhsbl_score = (
        'multi.surbl.org',              4,      0,      'multi.surbl.org',
        'rhsbl.ahbl.org',               4,      0,      'rhsbl.ahbl.org',
        'dsn.rfc-ignorant.org',         4,      0,      'dsn.rfc-ignorant.org',
#       'postmaster.rfc-ignorant.org',  0.1,    0,      'postmaster.rfc-ignorant.org',
#       'abuse.rfc-ignorant.org',       0.1,    0,      'abuse.rfc-ignorant.org'
);

# skip a RBL if this RBL had this many continuous errors
$BL_ERROR_SKIP = 2;

# skip a RBL for that many times
$BL_SKIP_RELEASE = 10;

## cache stuff
# must be a directory (add trailing slash)
$LOCKPATH = '/var/run/policyd-weight/';

# socket path for the cache daemon.
$SPATH = $LOCKPATH.'/polw.sock';

# how many seconds the cache may be idle before starting maintenance routines
#NOTE: standard maintenance jobs happen regardless of this setting.
$MAXIDLECACHE = 60;

# after this number of requests do following maintenance jobs: checking for config changes
$MAINTENANCE_LEVEL = 5;

# negative (i.e. SPAM) result cache settings ##################################

# set to 0 to disable caching for spam results. To this level the cache will be cleaned.
$CACHESIZE = 2000;

# at this number of entries cleanup takes place
$CACHEMAXSIZE = 4000;

$CACHEREJECTMSG  = '550 temporarily blocked because of previous errors';

# after NTTL retries the cache entry is deleted
$NTTL = 1;

# client MUST NOT retry within this seconds in order to decrease TTL counter
$NTIME = 30;

# positve (i.,e. HAM) result cache settings ###################################

# set to 0 to disable caching of HAM. To this number of entries the cache will be cleaned
$POSCACHESIZE = 1000;

# at this number of entries cleanup takes place
$POSCACHEMAXSIZE = 2000;

$POSCACHEMSG = 'using cached result';

#after PTTL requests the HAM entry must succeed one time the RBL checks again
$PTTL = 60;

# after $PTIME in HAM Cache the client must pass one time the RBL checks again.
#Values must be nonfractal. Accepted time-units: s, m, h, d
$PTIME = '3h';

# The client must pass this time the RBL checks in order to be listed as hard-HAM
# After this time the client will pass immediately for PTTL within PTIME
$TEMP_PTIME = '1d';


## DNS settings

# Retries for ONE DNS-Lookup
$DNS_RETRIES = 1;

# Retry-interval for ONE DNS-Lookup
$DNS_RETRY_IVAL  = 5;

# max error count for unresponded queries in a complete policy query
$MAXDNSERR = 3;

$MAXDNSERRMSG = 'passed - too many local DNS-errors';

# persistent udp connection for DNS queries.
#broken in Net::DNS version 0.51. Works with Net::DNS 0.53; DEFAULT: off
$PUDP= 0;

# Force the usage of Net::DNS for RBL lookups.
# Normally policyd-weight tries to use a faster RBL lookup routine instead of Net::DNS
$USE_NET_DNS  = 0;

# A list of space separated NS IPs
# This overrides resolv.conf settings
# Example: $NS = '1.2.3.4 1.2.3.5';
# DEFAULT: empty
$NS  = '';

# timeout for receiving from cache instance
$IPC_TIMEOUT  = 2;

# If set to 1 policyd-weight closes connections to smtpd clients in order to avoid too many
#established connections to one policyd-weight child
$TRY_BALANCE  = 0;

# scores for checks, WARNING: they may manipulate eachother
# or be factors for other scores.
#  HIT score, MISS Score
@client_ip_eq_helo_score = (1.5, -1.25 );
@helo_score  = (1.5, -2 );
@helo_score  = (0, -2 );
@helo_from_mx_eq_ip_score= (1.5, -3.1  );
@helo_numeric_score= (2.5,  0 );
@from_match_regex_verified_helo= (1,-2 );
@from_match_regex_unverified_helo = (1.6, -1.5  );
@from_match_regex_failed_helo  = (2.5,  0 );
@helo_seems_dialup = (1.5,  0 );
@failed_helo_seems_dialup= (2, 0 );
@helo_ip_in_client_subnet= (0,-1.2  );
@helo_ip_in_cl16_subnet  = (0,-0.41 );
#@client_seems_dialup_score  = (3.75, 0 );
@client_seems_dialup_score  = (0, 0 );
@from_multiparted  = (1.09, 0 );
@from_anon= (1.17, 0 );
@bogus_mx_score = (2.1,  0 );
@random_sender_score  = (0.25, 0 );
@rhsbl_penalty_score  = (3.1,  0 );
@enforce_dyndns_score = (3, 0 );

spamassassin/init.pre (I've put the .pre files together)

loadplugin Mail::SpamAssassin::Plugin::Hashcash
loadplugin Mail::SpamAssassin::Plugin::SPF
loadplugin Mail::SpamAssassin::Plugin::Pyzor
loadplugin Mail::SpamAssassin::Plugin::Razor2
loadplugin Mail::SpamAssassin::Plugin::AutoLearnThreshold
loadplugin Mail::SpamAssassin::Plugin::MIMEHeader
loadplugin Mail::SpamAssassin::Plugin::ReplaceTags
loadplugin Mail::SpamAssassin::Plugin::Check
loadplugin Mail::SpamAssassin::Plugin::HTTPSMismatch
loadplugin Mail::SpamAssassin::Plugin::URIDetail    
loadplugin Mail::SpamAssassin::Plugin::Bayes
loadplugin Mail::SpamAssassin::Plugin::BodyEval
loadplugin Mail::SpamAssassin::Plugin::DNSEval
loadplugin Mail::SpamAssassin::Plugin::HTMLEval
loadplugin Mail::SpamAssassin::Plugin::HeaderEval
loadplugin Mail::SpamAssassin::Plugin::MIMEEval
loadplugin Mail::SpamAssassin::Plugin::RelayEval
loadplugin Mail::SpamAssassin::Plugin::URIEval
loadplugin Mail::SpamAssassin::Plugin::WLBLEval
loadplugin Mail::SpamAssassin::Plugin::VBounce
loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody

spamassassin/local.cf (parts)

use_bayes                       1
bayes_auto_learn                1
bayes_store_module              Mail::SpamAssassin::BayesStore::MySQL
bayes_sql_dsn                   DBI:mysql:db:127.0.0.1:3306
bayes_sql_username              user
bayes_sql_password              pass
bayes_ignore_header             X-Bogosity
bayes_ignore_header             X-Spam-Flag
bayes_ignore_header             X-Spam-Status

### User settings
user_scores_dsn                 DBI:mysql:db:127.0.0.1:3306
user_scores_sql_password        user
user_scores_sql_username        pass
user_scores_sql_custom_query    SELECT preference, value FROM _TABLE_ WHERE username = _USERNAME_ OR username = '$GLOBAL' OR username = CONCAT('%',_DOMAIN_) ORDER BY username ASC

# for better speed
score DNS_FROM_AHBL_RHSBL       0
score __RFC_IGNORANT_ENVFROM    0
score DNS_FROM_RFC_DSN          0
score DNS_FROM_RFC_BOGUSMX      0
score __DNS_FROM_RFC_POST       0
score __DNS_FROM_RFC_ABUSE      0
score __DNS_FROM_RFC_WHOIS      0

UPDATE 01

As adaptr advised I remove policyd-weight and configured postfix postscreen, this resulted approximately -15-20 MB from RAM usage and a lot faster work. I'm not sure it's working at full capacity but it seems promising.

I have the following rewrite rule, used for WordPress Network to handle the file requests:

   location @rewrite {                                                     
            rewrite ^/(.*)$ /index.php?q=$uri&$args;                        
            rewrite ^.*/files/(.*)$ /wp-includes/ms-files.php?file=$1 last; 
    }

I seems that running owncloud 3 on the same domain the dav requests are caught by this rule. Therefore I'd like to extend the line to only be applied of no owncloud word was found in the uri.

Can someone help me, please?

After trying anything I could, the gzip compression in my nginx is still not working.

environment:

• Ubuntu 11.04 with nginx ppa, with the latest release (1.0.8)
• PHP5-FPM backend from package with proxy_pass.

main nginx config:

user www-data;
worker_processes 2;

error_log  /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;

events {
  worker_connections 1024;
  use epoll;
}


http {
  server_names_hash_bucket_size 64;
  sendfile on;
  tcp_nopush  on;
  tcp_nodelay off;

  client_max_body_size 64M;

  include /etc/nginx/mime.types;

  default_type text/html;

  types_hash_max_size 8192;

  log_format main '$remote_addr - $remote_user [$time_local] '
                '"$request" $status $body_bytes_sent "$http_referer" '
                '"$http_user_agent" "$http_x_forwarded_for"' ;

  access_log /var/log/nginx/access.log main;

  ## Compression
  gzip on;
  gzip_min_length 1000;
  gzip_http_version 1.1;
  gzip_comp_level 1;
  gzip_proxied any;
  gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;

  include /etc/nginx/sites-enabled/*;
}

I remove gzip_disable, I read bugs about it with Ubuntu.

nginx compile config:

apt-get source nginx

auto/options contains:

HTTP_GZIP=YES
HTTP_GZIP_STATIC=NO

Please someone say something what do I miss.