coding_hero's questions

This may be a stupid question, but after googling a while I can't find the answer or just don't know how to ask it.

I have a web app running on a server named 'myserver1'. I've brought up 'myserver2' with an identical instance of the web app, and set up replication between the two databases on the two boxes. Now, I'd like to employ nginx to do some load balancing, plus make one server take over if the other keels over.

Most of the nginx documentation is written around a simple scenario like this, but it seems to indicate that you put an nginx server in front of the web servers. That would seem to be another single point of failure. How do you make nginx itself redundant? Can you just run nginx on both web server boxes? If so, where do you point the DNS entry of myapp.mydomain.com?

EDIT: I guess I should add that this is for an internal app with a relatively small user base. My primary concern is that our internal users can still get to it if we lose a server or connectivity to one of the data centers. I just can't see how to do that on nginx without introducing another single point of failure.

I need to recompile php on a Fedora server because I need to use the --enable-zip flag. When trying to install the devel package, I get the following message. This is after a 'yum clean all':

yum install php53-devel.x86_64
Loaded plugins: rhnplugin, security
rhel-x86_64-server-5                                                                  | 1.4    kB     00:00     
rhel-x86_64-server-5/primary                                                          | 4.9 MB     00:00     
rhel-x86_64-server-5                                                                             14161/14161
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package php53-devel.x86_64 0:5.3.3-13.el5_8 set to be updated
--> Processing Dependency: php53 = 5.3.3-13.el5_8 for package: php53-devel
--> Finished Dependency Resolution
php53-devel-5.3.3-13.el5_8.x86_64 from rhel-x86_64-server-5 has depsolving problems
  --> Missing Dependency: php53 = 5.3.3-13.el5_8 is needed by package php53-devel-5.3.3-13.el5_8.x86_64 (rhel-x86_64-server-5)
Error: Missing Dependency: php53 = 5.3.3-13.el5_8 is needed by package php53-devel-5.3.3-13.el5_8.x86_64 (rhel-x86_64-server-5)
 You could try using --skip-broken to work around the problem
 You could try running: package-cleanup --problems
                        package-cleanup --dupes
                        rpm -Va --nofiles --nodigest

Output of 'yum repolist':

# yum repolist
Loaded plugins: rhnplugin, security
repo id                                               repo name                                                                           status
rhel-x86_64-server-5                                  Red Hat Enterprise Linux (v. 5 for 64-bit x86_64)                                   enabled: 14,161
repolist: 14,161

This is a follow-up to this question. mghocke was kind enough to help me out with that question, but this is really a separate question, therefor a new post.

I really don't know a lot about syslog, and am trying to move away from syslogd to syslog-ng.

In Solaris' syslogd, there is an 'audit' facility that you can see configured below:

local7.debug                                    /var/log/ncolog
audit.debug                                     /var/log/ncolog
local7.debug                                    @nimitz
audit.debug                                     @nimitz

In my new syslog-ng.conf, it is configured like this:

filter f_local7 { facility (local7); };
filter f_audit { facility (audit); };

...

log { source (s_sys); filter (f_local7); destination (d_ncolog); destination (d_nimitz); };
log { source (s_sys); filter (f_audit); destination (d_ncolog); destination (d_nimitz); };

But syslog-ng doesn't recognize the 'audit' facility. I've looked around some, and it seems that the 'audit' facility is facility 13, but 'local13' doesn't work. What is this facility called in syslog-ng?

I'm moving a Solaris box from syslogd to syslog-ng, because the Solaris version of syslogd obliterates the original source hostname on the logs. I'm looking through the syslogng.conf documentation, but am not sure I understand it all fully. We have a relatively simple syslog.conf, I was hoping a syslog-ng expert out there could tell me how to 'convert' it to a workable syslogng.conf?

#ident  "@(#)syslog.conf        1.5     98/12/14 SMI"   /* SunOS 5.0 */
#
# Copyright (c) 1991-1998 by Sun Microsystems, Inc.
# All rights reserved.
#
# syslog configuration file.
#
# This file is processed by m4 so be careful to quote (`') names
# that match m4 reserved words.  Also, within ifdef's, arguments
# containing commas must be quoted.
#
*.err;kern.notice;auth.notice                   /dev/sysmsg
*.err;kern.debug;daemon.notice;mail.crit        /var/adm/messages

#*.alert;kern.err;daemon.err                    operator
#*.alert                                                root

*.emerg                                         *
local7.debug                                    /var/log/ncolog
audit.debug                                     /var/log/ncolog
local7.debug                                    @nimitz
audit.debug                                     @nimitz

# if a non-loghost machine chooses to have authentication messages
# sent to the loghost machine, un-comment out the following line:
#auth.notice                    ifdef(`LOGHOST', /var/log/authlog, @loghost)

mail.debug                      ifdef(`LOGHOST', /var/log/syslog, @loghost)

#
# non-loghost machines will use the following lines to cause "user"
# log messages to be logged locally.
#
ifdef(`LOGHOST', ,
user.err                                        /dev/sysmsg
user.err                                        /var/adm/messages
#user.alert                                     `root, operator'
user.emerg                                      *
)

This is a continuation of my previous question here. The troubleshooting has gotten me a little further, and I think I'm facing a new set of problems now. Originally, I found out that sendmail wasn't listening on port 25 anymore, but I fixed this by increasing the values of the QueueLA and RefuseLA. I can telnet to port 25, and send email manually using SMTP commands. However, when I try sendmail:

[netcool1 (root)/]$ sendmail -v [email protected] < ~/testmessage
[email protected]... queued
[netcool1 (root)/]$ mailq
/var/spool/mqueue is empty
                Total requests: 0
[netcool1 (root)/]$ cd /var/spool/mqueue/
[netcool1 (root)/var/spool/mqueue]$ ls -la
total 6
drwxr-x---   2 root     bin         2048 Jan 11 11:54 .
drwxr-xr-x  13 root     bin          512 Jul 10  2009 ..
[netcool1 (root)/var/spool/mqueue]$ cd ../clientmqueue/
[netcool1 (root)/var/spool/clientmqueue]$ ls -la
total 76
drwxrwx---   2 smmsp    smmsp      22016 Jan 11 12:19 .
drwxr-xr-x  13 root     bin          512 Jul 10  2009 ..
-rw-rw----   1 smmsp    smmsp        100 Jan 11 07:03 dfq0BC3b8H007834
-rw-rw----   1 smmsp    smmsp         99 Jan 11 07:33 dfq0BCXba1008078
-rw-rw----   1 smmsp    smmsp         15 Jan 11 12:02 dfq0BH29Zu010211
-rw-rw----   1 smmsp    smmsp         15 Jan 11 12:13 dfq0BHDeEA010348
-rw-rw----   1 smmsp    smmsp        117 Jan 11 12:16 dfq0BHGuda010384
-rw-rw----   1 smmsp    smmsp        117 Jan 11 12:18 dfq0BHI9Q4010389
-rw-rw----   1 smmsp    smmsp         15 Jan 11 12:19 dfq0BHJjEv010410
-rw-rw----   1 smmsp    smmsp        871 Jan 11 07:03 qfq0BC3b8H007834
-rw-rw----   1 smmsp    smmsp        870 Jan 11 07:33 qfq0BCXba1008078
-rw-rw----   1 smmsp    smmsp        542 Jan 11 12:02 qfq0BH29Zu010211
-rw-rw----   1 smmsp    smmsp        562 Jan 11 12:13 qfq0BHDeEA010348
-rw-rw----   1 smmsp    smmsp        583 Jan 11 12:16 qfq0BHGuda010384
-rw-rw----   1 smmsp    smmsp        583 Jan 11 12:18 qfq0BHI9Q4010389
-rw-rw----   1 smmsp    smmsp        562 Jan 11 12:19 qfq0BHJjEv010410
-rw-------   1 smmsp    smmsp         34 Jan 11 12:12 sm-client.pid
[netcool1 (root)/var/spool/clientmqueue]$ sendmail -q -v
[netcool1 (root)/var/spool/clientmqueue]$ ls -la
total 76
drwxrwx---   2 smmsp    smmsp      22016 Jan 11 12:19 .
drwxr-xr-x  13 root     bin          512 Jul 10  2009 ..
-rw-rw----   1 smmsp    smmsp        100 Jan 11 07:03 dfq0BC3b8H007834
-rw-rw----   1 smmsp    smmsp         99 Jan 11 07:33 dfq0BCXba1008078
-rw-rw----   1 smmsp    smmsp         15 Jan 11 12:02 dfq0BH29Zu010211
-rw-rw----   1 smmsp    smmsp         15 Jan 11 12:13 dfq0BHDeEA010348
-rw-rw----   1 smmsp    smmsp        117 Jan 11 12:16 dfq0BHGuda010384
-rw-rw----   1 smmsp    smmsp        117 Jan 11 12:18 dfq0BHI9Q4010389
-rw-rw----   1 smmsp    smmsp         15 Jan 11 12:19 dfq0BHJjEv010410
-rw-rw----   1 smmsp    smmsp        871 Jan 11 07:03 qfq0BC3b8H007834
-rw-rw----   1 smmsp    smmsp        870 Jan 11 07:33 qfq0BCXba1008078
-rw-rw----   1 smmsp    smmsp        542 Jan 11 12:02 qfq0BH29Zu010211
-rw-rw----   1 smmsp    smmsp        562 Jan 11 12:13 qfq0BHDeEA010348
-rw-rw----   1 smmsp    smmsp        583 Jan 11 12:16 qfq0BHGuda010384
-rw-rw----   1 smmsp    smmsp        583 Jan 11 12:18 qfq0BHI9Q4010389
-rw-rw----   1 smmsp    smmsp        562 Jan 11 12:19 qfq0BHJjEv010410
-rw-------   1 smmsp    smmsp         34 Jan 11 12:12 sm-client.pid
[netcool1 (root)/var/spool/clientmqueue]$ cat sm-client.pid 
10344
/usr/lib/sendmail -Ac -q15m
[netcool1 (root)/var/spool/clientmqueue]$ 

I've been working on this for three days now, any help would be appreciated.

Edit: Contents of submit.mc (the only version I could find under /etc/mail was under /etc/mail/cf/cf):

$ more submit.mc 
divert(-1)
#
# Copyright (c) 2001, 2002 Sendmail, Inc. and its suppliers.
#       All rights reserved.
#
# By using this file, you agree to the terms and conditions set
# forth in the LICENSE file which can be found at the top level of
# the sendmail distribution.
#
#ident  "@(#)submit.mc  1.3     02/12/29 SMI"
#

#
#  This is the prototype file for a set-group-ID sm-msp sendmail that
#  acts as a initial mail submission program.
#

divert(0)dnl
VERSIONID(`$Id: submit.mc,v 8.6.2.4 2002/12/29 03:54:34 ca Exp $')
define(`confCF_VERSION', `Submit')dnl
define(`__OSTYPE__',`')dnl dirty hack to keep proto.m4 from complaining
define(`_USE_DECNET_SYNTAX_', `1')dnl support DECnet
define(`confTIME_ZONE', `USE_TZ')dnl
define(`confDONT_INIT_GROUPS', `True')dnl
dnl
dnl If you use IPv6 only, change [127.0.0.1] to [IPv6:::1]
FEATURE(`msp', `[127.0.0.1]')dnl

I'm wondering if this still has something to do with the RefuseLA and QueueLA values. I've set them impossibly high (QueueLA = 50, RefuseLA = 60) because this server really only sends emails from scripts and has a high LA due to many multi-threaded listeners. The increased RefuseLA got sendmail to listen on port 25 again (see previous question), but QueueLA doesn't seem to have any effect on the queuing. However, suddenly last night around 1900, the server sent through all the test emails that I had sent during the day.

I understand load averages on Linux, but am a bit mystified by the load averages on a legacy Solaris 10 machine my app runs on. The load averages seem impossibly high. Here's the output.

[netcool1 (root)/]$ uptime
 11:49am  up 580 day(s), 10:51,  3 users,  load average: 35.50, 38.54, 39.03
[netcool1 (root)/]$ uname -a
SunOS netcool1 5.10 Generic_139555-08 sun4u sparc SUNW,Sun-Fire-V245
[netcool1 (root)/]$ psrinfo -v
Status of virtual processor 0 as of: 01/11/2012 11:52:52
  on-line since 06/10/2010 01:58:29.
  The sparcv9 processor operates at 1504 MHz,
        and has a sparcv9 floating point processor.
Status of virtual processor 1 as of: 01/11/2012 11:52:52
  on-line since 06/10/2010 01:58:27.
  The sparcv9 processor operates at 1504 MHz,
        and has a sparcv9 floating point processor.
[netcool1 (root)/]$ 

I don't see how you can have a load average of 35 on a two-processor system. That seems incredibly high to me. When I view the processes with top, the system is about 60-70% idle. Could someone help explain this?

vmstat 10 6

kthr      memory            page            disk          faults      cpu
r b w   swap  free  re  mf pi po fr de sr rm s0 s2 --   in   sy   cs us sy id
3 0 0 8747008 5562704 865 1866 188 63 63 0 0 -0 9 40 0 762 8588 1495 26  8 66
0 0 0 7715256 5068016 73 23 5 17 17  0  0  0 110 66 0 1135 3888 9855 59 12 30
0 0 0 7717936 5069128 0  5  0  6  6  0  0  0 100 4  0 1071 3273 4191 62  6 32
0 0 0 7717952 5027912 0 11649 0 5 5  0  0  0 115 21 0 1017 26370 3260 32 15 53
102 1 0 7717952 4979088 0 1 0  0  0  0  0  0 112 4  0  900 3464 7683 15  9 76
0 0 0 7717952 4978936 0  1  0  0  0  0  0  0 105 4  0  886 3379 8698 19  9 72

I have a sendmail server on a Solaris 10 server that's stopped working. I can't seem to find anything about failed messages in /var/adm/sendmail. Here's what I'm seeing when I try to debug:

[(root)/var/adm]$ sendmail -v [email protected] < testmessage 
[email protected]... queued
[(root)/var/adm]$ /usr/ucb/mail -v [email protected]
Subject: Test message
Here is a test message
.
EOT
[(root)/var/adm]$ [email protected]... queued
[(root)/var/adm]$ mailq
/var/spool/mqueue is empty
    Total requests: 0

So, it doesn't look like it's doing anything. Sometimes when I try the debug, I get a '127.0.0.1 Connection Refused' back, but it doesn't do it constantly. Note: I've substituted my personal gmail address with '[email protected]' above. There's no procmail or anything set up for this address, and I am continually checking the Spam folder.

Any other things I can try to figure out why this isn't working?

Edit: Output of mailq -C includes lines and lines of this:

q0AFMJwG026997X    1629 Tue Jan 10 10:22 MAILER-DAEMON
             (Deferred: Connection refused by [127.0.0.1])

Could have sworn I tried this yesterday, but neither telnetting to localhost 25 from the machine nor telnetting to the machine from my desktop works:

[(root)/]$ telnet 127.0.0.1 25
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
[(root)/]$ 

and

[~]$ telnet 192.168.129.50 25
Trying 192.168.129.50...
telnet: connect to address 192.168.129.50: Connection refused
telnet: Unable to connect to remote host

Sendmail is running:

[(root)/]$ ps -ef |grep sendmail
smmsp 18616     1   0 16:22:18 ?           0:00 /usr/lib/sendmail -Ac -q15m
root 18614     1   0 16:22:18 ?           0:03 /usr/lib/sendmail -bd -q15m

But I don't see it listening on port 25 with 'netstat -an'.

EDIT: Thank you all for sticking around and helping. I have sendmail listening now on port 25 (it was refusing due to the high load average and the 'QueueLA' and 'RefuseLA' values). I can telnet to port 25 now and send email using SMTP commands. But, any mail I try to send with sendmail or /usr/ucb/mail still gets queued and disappears. What else do I need to check?

EDIT: I'm going to go ahead and close this, because I've moved on to a new set of problems now. The QueueLA and RefuseLA settings helped sendmail get up and running on port 25, but now I'm having a queueing issue. I've opened a new question here.