Vladislav Rastrusny's questions

Two of four of my servers currently have mismatch_cnt about 40000 and that worries me. We are using RAID10 setup. Manual states, that

However on RAID1 and RAID10 it is possible for software issues to cause a mismatch to be reported. This does not necessarily mean that the data on the array is corrupted. It could simply be that the system does not care what is stored on that part of the array - it is unused space.

We do not use any swap files on our servers. One of the server's HDDs has SMART self-check failing and Available_Reservd_Space is too low. Hosting provider says, that it replaces HDDs only when they are physically faulty.

I think I do not get the real meaning and the usefullness of this param. What could be other reasons for this parameter to have such a big value? How could that be that the system does not care about what is there on that part of array if that's a mirrored one? Due to security considerations a system should sync free space also I think and then - what's left?

Are there any reliable ways to estimate the risk of having a particular HDD in a server?

What is the algorithm used to generate etags in Nginx? They look something like "554b73dc-6f0d" now.

Are they generated from timestamp only?

Is there a way to delete all iptables rules, that relate to a given interface? I don't see any command line option for this, may be there is a shorcut bash script for this?

Good routers usually have a price higher than a price of a low-budget PC ($200) which I can make a router by installing Zentyal, pfSense, ClearOS or something like this.

For example, my friend uses Dlink DFL-860E in the office. This thing prices like $550. I can buy a descent PC for this money and make it fulful more job, than this Dlink can.

What is the advantage of using such devices instead of PCs? Is this really only electricity cost considerations?

I have a head office and a number of suboffices. Both head office and suboffices need to access some corporate server, which is physically in the head office. But suboffices should not be able to access head office or each other. Suboffices are at the substantial distance of each other and from the head office (many kms).

How do I design the network for this task?

I suppose, each suboffice should have internet access. Central Server LAN switch should have static IP and OpenVPN software installed. For example, SubOffice1 network is 10.0.1.x, SubOffice2 10.0.2.x etc, Head office 10.254.x.x, Central Server LAN 192.168.0.x.

When suboffice PC or head office PC needs to access Central Server, a PC starts VPN connection for this to the central server.

I should then use routers for each network and setup firewall so that it permits connections from inside network only except for the Central Server LAN router.

Is this correct? May be there are some important details I should keep in mind building this network? What hardware would you recommend for this (routers, supporting needed firewalling modes etc)?

ADDED 07/05/2012:

Our ISP can provide nothing except for Internet access. I cannot expect them to support anything like MPLS. ISPs are different in each suboffice and head office.

The number of suboffices are like 20.

Connections from suboffices into the head office needs to be encrypted because it will be routed through internet.

I want suboffices to be isolated from head office and from each other completely so that no packet can travel there and back.

I plan to have only Linux PCs in the offices, but there can be also some Windows machines. No Active Directory or something. Just a PC under Windows/Linux.

Any good books out there on the subject?

Which IP should hostname machine.example.com resolve to in /etc/hosts?

• To localhost (127.0.0.X),
• to a local network (10.X.X.X or 192.168.X.X),
• or to the external server IP visible from outside (X.X.X.X)?

My servers are currently on sendmail which is not very efficient. I'd like to migrate to PostFix.

I'm using Google Mail for Domain for handling incoming email so I need to setup PostFix to send emails only. What's the minimal PostFix configuration I must do to set up outgoing email delivery and prohibit receiving email? All localhost emails are trustable by default.

In new MySQL versions Barracuda file format was introduced. So I havew a few questions:

• Is it working only with file_per_table option?
• How to switch existing system to it (preferably the whole tablespace) other than by reimporting everything from mysqldump backup on empty server?

I have some files on a server with the date several months ago, but they are invisible to find -mtime 7 search.

When I list them with ls -l, they look perfectly normal:

-rw-r--r-- 1 root root    347253 Jun 12 16:26 pedia_main.2010-06-12-04-25-02.sql.gz
-rw-r--r-- 1 root root 490144578 Nov 24 16:26 gsmforum_main.2010-11-24-04-25-02.sql.gz

The top file is invisible to "find . -mtime 1", but the bottom one is visible.

I almost banged my head against the wall trying to understand why. I tried some random things and came across ls --full-time command. And it shows that these two are somehow a little bit different

-rw-r--r-- 1 root root    347253 2010-06-12 16:26:20.000000000 +0400 pedia_main.2010-06-12-04-25-02.sql.gz
-rw-r--r-- 1 root root 490144578 2010-11-24 16:26:12.000000000 +0300 gsmforum_main.2010-11-24-04-25-02.sql.gz

The date seems to be ok. Bit one has +0400 as a timezone, and another one is +0300. How come find fails to find the ones with +0400?

The OS is CentOS 5.5 Final with latest updates, and ls version is (GNU coreutils) 5.97.

Also, the thing is that I don't understand where this "timezone" of the file is stored. The inode does not have any extra attributes to store them it seems. The file system on the server is ext4.

What is the maximum number of VirtualHosts Apache can handle on a single machine (I don't mean anything related to load, let's suppose it's irrelevant for the question). And we take only Apache without any proxifying things like nginx.

I am asking because on one forum one guy reported that his Apache works unstable with the number of sites more than 400 on a single machine.

If you have a config, that handles more than 400, please tell me here.

Thanks.

What is exactly --enable-zend-multibyte option for when you ./configure PHP?

I have a forum script running on server and somehow small number of attachments begin to get lost. I want to find out what is deleting them and at what time. How can I setup Linux auditd (auditctl) to watch directory tree (attachments are stored inside multi-level directory tree) to watch for file deletions there?

May be I should use some other tool for this?

Tune2fs allows to change inode size from default (128 bytes on ext3, 256 bytes on ext4) to almost anything, but it should be power of two. What are the reasons for changing default inode size?

Here it's written that this can be done to be able to store ACL attributes inside inodes. What else can be stored inside an inodes?

Is there any reason to increase inode size on modern high-capacity drives (2TB and more)?

I've seen millions of websites using nginx as a proxifying webserver working together with Apache. But I've seen very few servers running nginx only as their default webserver. What are the main downsides of such config?

I can see some:

• Inability to use per-directory config files like .htaccess so every configuration change should be done to main server config file and requires server reload. But pecl htscanner can compensate them for php settings
• Unavailability of mod_php for nginx, which can be compensated by php-fpm for example.

What are others? Why don't people just drop Apache and move to nginx or any other lightweight solution? May be, there are some special reasons?

EDIT: this question is primarily about working with LAMP stack.

What are the main differences between desktop-series hard disks and server-series?

The obvious things I can see are: durability (server hardware mostly more qualitative and have more warranty) and power consumption (server hardware more focused on performance, than on power economy). Also server disks are usually a little faster, but it seems, that it is not always the case.

May be there are some other reasons, that make you choose server-oriented series (Seagate ES drives, for example) over desktop-oriented ones (Seagate Barracuda series)? What are they?

I wanted to ask you about PHP/Apache configuration methods you know, their pros and cons. I will start myself:

---------------- PHP as Apache module----------------

Pros: good speed since you don't need to start exe every time especially in mpm-worker mode. You can also use various PHP accelerators in this mode like APC or eAccelerator.

Cons: if you are running apache in mpm-worker mode, you may face stability issues because every glitch in any php script will lead to unstability to the whole thread pool of that apache process. Also in this mode all scripts are executed on behalf of apache user. This is bad for security. mpm-worker configuration requires PHP compiled in thread-safe mode. At least CentOS and RedHat default repositories doesn't have thread-safe PHP version so on these OSes you need to compile at least PHP yourself (there is a way to activate worker mpm on Apache). The use of thread-safe PHP binaries is considered experimental and unstable. Plus, many PHP extensions does not support thread-safe mode or were not well-tested in thread-safe mode.

---------------- PHP as CGI ----------------

This seems to be the slowest default configuration which seems to be a "con" itself ;)

---------------- PHP as CGI via mod_suphp ----------------

Pros: suphp allows you to execute php scipts on behalf of the script file owner. This way you can securely separate different sites on the same machine. Also, suphp allows to use different php.ini files per virtual host.

Cons: PHP in CGI mode means less performance. In this mode you can't use php accelerators like APC because each time new process is spawned to handle script rendering the cache of previous process useless. BTW, do you know the way to apply some accelerator in this config? I heard something about using shm for php bytecode cache. Also, you cannot configure PHP via .htaccess files in this mode. You will need to install PECL htscanner for this if you need to set various per-script options via .htaccess (php_value / php_flag directives)

---------------- PHP as CGI via suexec ----------------

This configuration looks the same as with suphp, but I heard, that it's slower and less safe. Almost same pros and cons apply.

---------------- PHP as FastCGI ----------------

Pros: FastCGI standard allows single php process to handle several scripts before php process is killed. This way you gain performance since no need to spin up new php process for each script. You can also use PHP accelerators in this configuration (see cons section for comment). Also, FCGI almost like suphp also allows php processes to be executed on behalf of some user. mod_fcgid seems to have the most complete fcgi support and flexibility for apache.

Cons: The use of php accelerator in fastcgi mode will lead to high memory consumption because each PHP process will have his own bytecode cache (unless there is some accelerator that can use shared memory for bytecode cache. Is there such?). FastCGI is also a little bit complex to configure. You need to create various configuration files and make some configuration modifications.

It seems, that fastcgi is the most stable, secure, fast and flexible PHP configuration, however, a bit difficult to be configured. But, may be, I missed something?

Comments are welcome!

I am upgrading LAMP stack on customer's server and need to ./configure mysql and apache with exact last settings they were compiled with last time. Where do I get these? PHP configure string can be got by php -i. What about others?

How do I tell lsof I need to list only physical files (not sockets, not TCP/IP connections, only physical files)?

I have the exact problem, like in this post: Sendmail to local domain ignoring MX records . I am also using an email provider like GMail For Your Domain (which stores your mail and manages it). I am sending mail from my server directly, but receiving mail is done via Yandex (email provider). Since the server hosts a forum, I prefer to send mail directly from it because using another mail provider can slow things down. Also, when I send 300.000 email to my subscribers, another email provider will surely block me thinking I am sending spam.

My DNS zone now is:

;
; GSMFORUM.RU
;

$TTL 1H

gsmforum.ru. SOA ns1.hc.ru. support.hc.ru. (
               2009122268 ; Serial
               1H         ; Refresh
               30M        ; Retry
               1W         ; Expire
               1H       ) ; Minimum

gsmforum.ru.          NS      ns1.hc.ru.
gsmforum.ru.          NS      ns2.hc.ru.
@                     A       79.174.68.223
*.gsmforum.ru.        CNAME   @
ns1                   A       79.174.68.223
ns2                   A       79.174.68.224
@                     MX      10 mx.yandex.ru.
mail                  CNAME   domain.mail.yandex.net.
yamail-xxxxxxxxx      CNAME   mail.yandex.ru.

Server hostname is server.gsmforum.ru. May be this is the cause? Can someone explain the reason of the matter (the rules that make sendmail consider domain to be local)? Can I easily change

*.gsmforum.ru.        CNAME   @

into

*.gsmforum.ru.        A   79.174.68.224

to solve this problem?

[root@server ~]# cat /etc/mail/local-host-names
localhost
localhost.localdomain

This server hosts gsmforum.ru so I cannot put it into another domain like David Mackintosh suggests. Putting the domain in mailertable doesn't solve the problem either: sendmail -bt still shows the address is local. DontProbeInterfaces is also set to true in the sendmail config.

M4 file follows:

divert(-1)dnl
dnl #
dnl # This is the sendmail macro config file for m4. If you make changes to
dnl # /etc/mail/sendmail.mc, you will need to regenerate the
dnl # /etc/mail/sendmail.cf file by confirming that the sendmail-cf package is
dnl # installed and then performing a
dnl #
dnl #     make -C /etc/mail
dnl #
include(`/usr/share/sendmail-cf/m4/cf.m4')dnl
VERSIONID(`setup for linux')dnl
OSTYPE(`linux')dnl
dnl #
dnl # Do not advertize sendmail version.
dnl #
dnl define(`confSMTP_LOGIN_MSG', `$j Sendmail; $b')dnl
dnl #
dnl # default logging level is 9, you might want to set it higher to
dnl # debug the configuration
dnl #
dnl define(`confLOG_LEVEL', `9')dnl
dnl #
dnl # Uncomment and edit the following line if your outgoing mail needs to
dnl # be sent out through an external mail server:
dnl #
dnl define(`SMART_HOST', `smtp.your.provider')dnl
dnl #
define(`confDEF_USER_ID', ``8:12'')dnl
dnl define(`confAUTO_REBUILD')dnl
define(`confTO_CONNECT', `1m')dnl
define(`confTRY_NULL_MX_LIST', `True')dnl
define(`confDONT_PROBE_INTERFACES',`True')
define(`PROCMAIL_MAILER_PATH', `/usr/bin/procmail')dnl
define(`ALIAS_FILE', `/etc/aliases')dnl
define(`STATUS_FILE', `/var/log/mail/statistics')dnl
define(`UUCP_MAILER_MAX', `2000000')dnl
define(`confUSERDB_SPEC', `/etc/mail/userdb.db')dnl
define(`confPRIVACY_FLAGS', `authwarnings,novrfy,noexpn,restrictqrun')dnl
define(`confAUTH_OPTIONS', `A')dnl
dnl #
dnl # The following allows relaying if the user authenticates, and disallows
dnl # plaintext authentication (PLAIN/LOGIN) on non-TLS links
dnl #
dnl define(`confAUTH_OPTIONS', `A p')dnl
dnl # 
dnl # PLAIN is the preferred plaintext authentication method and used by
dnl # Mozilla Mail and Evolution, though Outlook Express and other MUAs do
dnl # use LOGIN. Other mechanisms should be used if the connection is not
dnl # guaranteed secure.
dnl # Please remember that saslauthd needs to be running for AUTH. 
dnl #
dnl TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
dnl define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
dnl #
dnl # Rudimentary information on creating certificates for sendmail TLS:
dnl #     cd /usr/share/ssl/certs; make sendmail.pem
dnl # Complete usage:
dnl #     make -C /usr/share/ssl/certs usage
dnl #
dnl define(`confCACERT_PATH', `/etc/pki/tls/certs')dnl
dnl define(`confCACERT', `/etc/pki/tls/certs/ca-bundle.crt')dnl
dnl define(`confSERVER_CERT', `/etc/pki/tls/certs/sendmail.pem')dnl
dnl define(`confSERVER_KEY', `/etc/pki/tls/certs/sendmail.pem')dnl
dnl #
dnl # This allows sendmail to use a keyfile that is shared with OpenLDAP's
dnl # slapd, which requires the file to be readble by group ldap
dnl #
dnl define(`confDONT_BLAME_SENDMAIL', `groupreadablekeyfile')dnl
dnl #
dnl define(`confTO_QUEUEWARN', `4h')dnl
dnl define(`confTO_QUEUERETURN', `5d')dnl
dnl define(`confQUEUE_LA', `12')dnl
dnl define(`confREFUSE_LA', `18')dnl
define(`confTO_IDENT', `0')dnl
dnl FEATURE(delay_checks)dnl
FEATURE(`no_default_msa', `dnl')dnl
FEATURE(`smrsh', `/usr/sbin/smrsh')dnl
FEATURE(`mailertable', `hash -o /etc/mail/mailertable.db')dnl
FEATURE(`virtusertable', `hash -o /etc/mail/virtusertable.db')dnl
FEATURE(redirect)dnl
FEATURE(always_add_domain)dnl
FEATURE(use_cw_file)dnl
FEATURE(use_ct_file)dnl
dnl #
dnl # The following limits the number of processes sendmail can fork to accept 
dnl # incoming messages or process its message queues to 20.) sendmail refuses 
dnl # to accept connections once it has reached its quota of child processes.
dnl #
dnl define(`confMAX_DAEMON_CHILDREN', `20')dnl
dnl #
dnl # Limits the number of new connections per second. This caps the overhead 
dnl # incurred due to forking new sendmail processes. May be useful against 
dnl # DoS attacks or barrages of spam. (As mentioned below, a per-IP address 
dnl # limit would be useful but is not available as an option at this writing.)
dnl #
dnl define(`confCONNECTION_RATE_THROTTLE', `3')dnl
dnl #
dnl # The -t option will retry delivery if e.g. the user runs over his quota.
dnl #
FEATURE(local_procmail, `', `procmail -t -Y -a $h -d $u')dnl
FEATURE(`access_db', `hash -T<TMPF> -o /etc/mail/access.db')dnl
FEATURE(`blacklist_recipients')dnl
EXPOSED_USER(`root')dnl
dnl #
dnl # For using Cyrus-IMAPd as POP3/IMAP server through LMTP delivery uncomment
dnl # the following 2 definitions and activate below in the MAILER section the
dnl # cyrusv2 mailer.
dnl #
dnl define(`confLOCAL_MAILER', `cyrusv2')dnl
dnl define(`CYRUSV2_MAILER_ARGS', `FILE /var/lib/imap/socket/lmtp')dnl
dnl #
dnl # The following causes sendmail to only listen on the IPv4 loopback address
dnl # 127.0.0.1 and not on any other network devices. Remove the loopback
dnl # address restriction to accept email from the internet or intranet.
dnl #
DAEMON_OPTIONS(`Name=MTA,Port=smtp')
dnl #
dnl # The following causes sendmail to additionally listen to port 587 for
dnl # mail from MUAs that authenticate. Roaming users who can't reach their
dnl # preferred sendmail daemon due to port 25 being blocked or redirected find
dnl # this useful.
dnl #
dnl DAEMON_OPTIONS(`Port=submission, Name=MSA, M=Ea')dnl
dnl #
dnl # The following causes sendmail to additionally listen to port 465, but
dnl # starting immediately in TLS mode upon connecting. Port 25 or 587 followed
dnl # by STARTTLS is preferred, but roaming clients using Outlook Express can't
dnl # do STARTTLS on ports other than 25. Mozilla Mail can ONLY use STARTTLS
dnl # and doesn't support the deprecated smtps; Evolution <1.1.1 uses smtps
dnl # when SSL is enabled-- STARTTLS support is available in version 1.1.1.
dnl #
dnl # For this to work your OpenSSL certificates must be configured.
dnl #
dnl DAEMON_OPTIONS(`Port=smtps, Name=TLSMTA, M=s')dnl
dnl #
dnl # The following causes sendmail to additionally listen on the IPv6 loopback
dnl # device. Remove the loopback address restriction listen to the network.
dnl #
dnl DAEMON_OPTIONS(`port=smtp,Addr=::1, Name=MTA-v6, Family=inet6')dnl
dnl #
dnl # enable both ipv6 and ipv4 in sendmail:
dnl #
dnl DAEMON_OPTIONS(`Name=MTA-v4, Family=inet, Name=MTA-v6, Family=inet6')
dnl #
dnl # We strongly recommend not accepting unresolvable domains if you want to
dnl # protect yourself from spam. However, the laptop and users on computers
dnl # that do not have 24x7 DNS do need this.
dnl #
FEATURE(`accept_unresolvable_domains')dnl
dnl #
dnl FEATURE(`relay_based_on_MX')dnl
dnl # 
dnl # Also accept email sent to "localhost.localdomain" as local email.
dnl # 
LOCAL_DOMAIN(`localhost.localdomain')dnl
dnl #
dnl # The following example makes mail from this host and any additional
dnl # specified domains appear to be sent from mydomain.com
dnl #
dnl MASQUERADE_AS(`mydomain.com')dnl
dnl #
dnl # masquerade not just the headers, but the envelope as well
dnl #
dnl FEATURE(masquerade_envelope)dnl
dnl #
dnl # masquerade not just @mydomainalias.com, but @*.mydomainalias.com as well
dnl #
dnl FEATURE(masquerade_entire_domain)dnl
dnl #
dnl MASQUERADE_DOMAIN(localhost)dnl
dnl MASQUERADE_DOMAIN(localhost.localdomain)dnl
dnl MASQUERADE_DOMAIN(mydomainalias.com)dnl
dnl MASQUERADE_DOMAIN(mydomain.lan)dnl
MAILER(smtp)dnl
MAILER(procmail)dnl
dnl MAILER(cyrusv2)dnl
FEATURE(`dnsbl',`zen.spamhaus.org',`Rejected - your IP is blacklisted by http://www.spamhaus.org')

How can I setup iptables in a way I know what packet is blocked by which rule? The one solution I know of is to -j LOG --log-prefix. Is there any other way?