Home / user-11786

grufftech's questions

Martin Hope
grufftech
Asked: 2014-05-07 10:24:31 +0800 CST
  • 2

So a month or so ago, I installed Windows Server 2012 R2 Standard Eval. The server was just being used as a quickbooks server (remote desktop only used for administration, only runs Quickbooks DB software & file sharing)

The finance team likes the new setup, so it's time to buy a license, but I realised now that Standard, isn't the "base level" package, it's "Essentials", and from what I can tell, there's no REAL need for standard.

Please tell me there's some way I can just buy Essentials, and it'll downgrade Windows 2012 Server Standard to Essentials, without me having to rebuild the entire machine, migrate quickbooks, etc.

windows-server-2012-r2
Martin Hope
grufftech
Asked: 2011-10-18 13:37:13 +0800 CST
  • 4

So my HAProxy configuration is similar to this. 

listen webaustin 0.0.0.0:80
    mode http
    timeout connect 12000
    timeout server 60000
    timeout queue 120000
    balance roundrobin
    option httpchk GET /index.html
    log global
    option httplog
    option dontlog-normal
    server web1 10.10.10.101:80 maxconn 600 check fall 10
    server web2 10.10.10.102:80 maxconn 600 check fall 10
    server web3 10.10.10.103:80 maxconn 600 check fall 10
    server web4 10.10.10.104:80 maxconn 600 check fall 10

Which works pretty well for our systems, Four backend webservers, one frontend webserver, timeouts set to be reasonable.

My problem is, I have one URL, which i need to be vastly longer timeout. Its a very heavy task that takes a long time on the backend.

Is there anyway to change the timeout variables for a single URL?

url timeout haproxy 504
Martin Hope
grufftech
Asked: 2011-08-03 14:37:54 +0800 CST
  • 2

In my configuration, alerts are enabled.

yet somehow; every time i reboot my nagios cluster, it re-disables-notifications on a handful of services.

Oh nagios gods

Oh Nagios Gods, what have I done to offend thee?? I disable these via the web interface, but low and behold every time i change config and reload nagios, bloop! back to disabled they come.

Getting angry. Please help; i wise ones of Serverfault. I'm out of ideas. >.<

  • Ubuntu 10.04.2 LTS

    Nagios Core 3.2.0

    Apache/2.2.14 (Ubuntu)

    NagiosQL 3.1.1

ubuntu apache-2.2 nagios
Martin Hope
grufftech
Asked: 2011-07-16 07:36:14 +0800 CST
  • 1

So I have 2 configuration files.

mysite.conf

<VirtualHost *:80>
        ServerName      www.mysite.com
        ServerAlias     mysite.com *.mysite.com
        DocumentRoot    /var/www/mysite/www
        ErrorDocument 404 /redirect.php
    ErrorLog    logs/application/mysite.error_log
    CustomLog   logs/application/mysite.access_log Combined
</VirtualHost>

videoupload.conf

<VirtualHost *:80>
        ServerName      upload.mysite.com
        DocumentRoot    /var/www/videouploader/www
    ErrorLog logs/application/videoupload.error_log
    CustomLog logs/application/videoupload.access_log Combined
</VirtualHost>

See, we have about 50 possible something.mysite.com subdomains, all that do different things, but they're all served off the same document root.

I'd like to keep *.mysite.com in there, simply because its cleaner then having the worlds longest serveralias line, plus it protects from retards trying bananananan.mysite.com or something that wouldn't work.

I've read through the apache documentation and it didn't really have anything for this senario. Anyone know how to do this?

linux ubuntu apache-2.2 virtualhost httpd
Martin Hope
grufftech
Asked: 2011-07-01 08:29:28 +0800 CST
  • 6

My HAProxy Configuration.

#HA-Proxy version 1.3.22 2009/10/14  Copyright 2000-2009 Willy Tarreau <[email protected]>
global
    maxconn 10000
    spread-checks 50
    user haproxy
    group haproxy
    daemon
    stats socket /tmp/haproxy
    log localhost   local0
    log localhost   local1 notice

defaults
    mode    http
    maxconn 50000
    timeout client 10000
    option forwardfor except 127.0.0.1
    option httpclose
    option httplog

listen dcaustin 0.0.0.0:80
    mode http
    timeout connect 12000
    timeout server 60000
    timeout queue 120000
    balance roundrobin
    option httpchk GET /index.html
    log global
    option httplog
    option dontlog-normal
    server web1 10.10.10.101:80 maxconn 300 check fall 1
    server web2 10.10.10.102:80 maxconn 300 check fall 1
    server web3 10.10.10.103:80 maxconn 300 check fall 1
    server web4 10.10.10.104:80 maxconn 300 check fall 1

listen stats 0.0.0.0:9000
    mode http
    balance
    log global
    timeout client 5000
    timeout connect 4000
    timeout server 30000
    stats uri /haproxy

HAProxy is running, and the socket is working...

adam@dcaustin:/etc/haproxy# echo "show info" | socat stdio /tmp/haproxy 
Name: HAProxy
Version: 1.3.22
Release_date: 2009/10/14
Nbproc: 1
Process_num: 1
Pid: 6320
Uptime: 0d 0h14m58s
Uptime_sec: 898
Memmax_MB: 0
Ulimit-n: 20017
Maxsock: 20017
Maxconn: 10000
Maxpipes: 0
CurrConns: 47
PipesUsed: 0
PipesFree: 0
Tasks: 51
Run_queue: 1
node: dcaustin
desiption:

Errors show nothing from socket...

adam@dcaustin:/etc/haproxy# echo "show errors" | socat stdio /tmp/haproxy 
adam@dcaustin:/etc/haproxy#

However...

My Error log is exploding with "badrequests" with the Error code cR. cR (according to 1.3 documentation) is The "timeout http-request" stroke before the client sent a full HTTP request. This is sometimes caused by too large TCP MSS values on the client side for PPPoE networks which cannot transport full-sized packets, or by clients sending requests by hand and not typing fast enough, or forgetting to enter the empty line at the end of the request. The HTTP status code is likely a 408 here.

Correct on the 408, but we're getting literally thousands of these requests every hour. (This log snippet is an clip for about 10 seconds of time...)

Jun 30 11:08:52 localhost haproxy[6320]: 92.22.213.32:26448 [30/Jun/2011:11:08:42.384] dcaustin dcaustin/<NOSRV> -1/-1/-1/-1/10002 408 212 - - cR-- 35/35/18/0/0 0/0 "<BADREQ>"
Jun 30 11:08:54 localhost haproxy[6320]: 71.62.130.24:62818 [30/Jun/2011:11:08:44.457] dcaustin dcaustin/<NOSRV> -1/-1/-1/-1/10001 408 212 - - cR-- 39/39/16/0/0 0/0 "<BADREQ>"
Jun 30 11:08:55 localhost haproxy[6320]: 84.73.75.236:3589 [30/Jun/2011:11:08:45.021] dcaustin dcaustin/<NOSRV> -1/-1/-1/-1/10008 408 212 - - cR-- 35/35/15/0/0 0/0 "<BADREQ>"
Jun 30 11:08:55 localhost haproxy[6320]: 69.39.20.190:49969 [30/Jun/2011:11:08:45.709] dcaustin dcaustin/<NOSRV> -1/-1/-1/-1/10000 408 212 - - cR-- 37/37/16/0/0 0/0 "<BADREQ>"
Jun 30 11:08:56 localhost haproxy[6320]: 2.29.0.9:58772 [30/Jun/2011:11:08:46.846] dcaustin dcaustin/<NOSRV> -1/-1/-1/-1/10001 408 212 - - cR-- 43/43/22/0/0 0/0 "<BADREQ>"
Jun 30 11:08:57 localhost haproxy[6320]: 212.139.250.242:57537 [30/Jun/2011:11:08:47.568] dcaustin dcaustin/<NOSRV> -1/-1/-1/-1/10000 408 212 - - cR-- 42/42/21/0/0 0/0 "<BADREQ>"
Jun 30 11:08:58 localhost haproxy[6320]: 74.79.195.75:55046 [30/Jun/2011:11:08:48.559] dcaustin dcaustin/<NOSRV> -1/-1/-1/-1/10000 408 212 - - cR-- 46/46/24/0/0 0/0 "<BADREQ>"
Jun 30 11:08:58 localhost haproxy[6320]: 74.79.195.75:55044 [30/Jun/2011:11:08:48.554] dcaustin dcaustin/<NOSRV> -1/-1/-1/-1/10004 408 212 - - cR-- 45/45/24/0/0 0/0 "<BADREQ>"
Jun 30 11:08:58 localhost haproxy[6320]: 74.79.195.75:55045 [30/Jun/2011:11:08:48.554] dcaustin dcaustin/<NOSRV> -1/-1/-1/-1/10005 408 212 - - cR-- 44/44/24/0/0 0/0 "<BADREQ>"
Jun 30 11:09:00 localhost haproxy[6320]: 68.197.56.2:52781 [30/Jun/2011:11:08:50.975] dcaustin dcaustin/<NOSRV> -1/-1/-1/-1/10000 408 212 - - cR-- 49/49/28/0/0 0/0 "<BADREQ>"

From what I read on google, if i wanted to see what the bad requests are, I can show errors to the socket and it will spit them out. We do run a pretty heavily trafficed website and the percentage of "BADREQS" to normal requests is quite low, but I'd like to be able to get ahold of what that request WAS so I can debug it.

stats

# pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,
dcaustin,FRONTEND,,,64,120,50000,88433,105889100,2553809875,0,0,4641,,,,,OPEN,,,,,,,,,1,1,0,,,,0,45,0,128,
dcaustin,web1,0,0,10,28,300,20941,25402112,633143416,,0,,0,3,0,0,UP,1,1,0,0,0,2208,0,,1,1,1,,20941,,2,11,,30,
dcaustin,web2,0,0,9,30,300,20941,25026691,641475169,,0,,0,3,0,0,UP,1,1,0,0,0,2208,0,,1,1,2,,20941,,2,11,,30,
dcaustin,web3,0,0,10,27,300,20940,30116527,635015040,,0,,0,9,0,0,UP,1,1,0,0,0,2208,0,,1,1,3,,20940,,2,10,,31,
dcaustin,web4,0,0,5,28,300,20940,25343770,643209546,,0,,0,8,0,0,UP,1,1,0,0,0,2208,0,,1,1,4,,20940,,2,11,,31,
dcaustin,BACKEND,0,0,34,95,50000,83762,105889100,2553809875,0,0,,0,34,0,0,UP,4,4,0,,0,2208,0,,1,1,0,,83762,,1,43,,122,

88500 "Sessions" and 4500 errors. in the last 20 minutes.

load-balancing reverse-proxy haproxy ubuntu-10.04
Martin Hope
grufftech
Asked: 2011-06-14 08:37:12 +0800 CST
  • 2

So -- I've used mod rewrite a few thousand times to do things like changing url.tld/show/season/1 to load video.php?show=someshow&season=1.

However -- Now i need to do the opposite, show a temporary HTML page instead of serving PHP on those pages.

myshows.com/seasons/?sid=1

I've tried doing a rewrite cond including the ?, but it never seems to work.

RewriteRule ^/seasons/\?sid=1 migrating.html

EDIT I need to make sure the only page effected is the SID=1, not any of the other possible season ID's

Any ideas?

apache-2.2 mod-rewrite
Martin Hope
grufftech
Asked: 2011-05-11 12:10:42 +0800 CST
  • 3

Alright. So we have a lighttpd server pumping out the loading of our images.

It logs on a daily basis, and we're considering moving all those images to our S3 account for better load times, but before we can do that i need to get atleast a "feel" for what our transfer will look like.

So right now we have the standard lighttp access log

accesslog.filename = "/var/log/lighttpd/images.access_log" accesslog.format = "%h %V %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""

so want to dump all the "bytes transferred" from that log file, nice and easy.

cat images.access_log | awk '{print $10}'

which kicks out a output similar to this.

19547
6138
17782
8044
345
0
2727
2125
1838
1649
2127
3275
3653
0
16688

Now -- I've done a bit of googling and maybe i'm forgetting something, is there a command hidden away in linux somewhere that will take all that output, and just add it all together for me? so i can run this command and just have it spit out an absurdly large number every day until i get a baseline of our bandwidth per day?

--- EDIT ---

i Found https://stackoverflow.com/questions/450799/linux-command-to-sum-integers-one-per-line

IS there anyway to get awk to return the full number, instead of doing what is shown below?

cat images.access_log | awk '{print $10}' | awk '{s+=$1} END {print s}' 
9.48886e+10
linux bash command-line-interface
Martin Hope
grufftech
Asked: 2011-03-12 10:03:47 +0800 CST
  • 7

Alright. First and foremost, Warning. This is a bigger-then-normal question. I like to be thorough and try to eliminate all possible "easymode" answers, as well as give everyone a feel of what i've tried. I've included several images of our setup and the problem it is having..

TLDR Version: So I've followed the guides located here: ESX Deployment Guide V1 this is the guide Dell has sent me to setup two ESX3.5 servers mounting a Dell MD3000i. It doesn't work. Both servers can't use the same storage partition on the MD3000. Both servers see it, but only one server can actually use it. (that server being whatever server created the partition on the target.) Both ESX servers are members of the Host Group.

Full Version

I have 2 ESX3.5 Servers (10.0.7.102, also called EPI2, and 10.0.7.103, also called EPI3.) connected to a iSCSI SAN Device (Dell MD3000i). Both ESX servers can "scan" the SAN and see the LUNS.

Part One: MD3000i Storage

On the MD3000i, Both servers are in my host group. enter image description here

I have two partitions, VM1 and VM2, both 1.6TB (vmware doesn't like anything past 2tb.) enter image description here

And you can even see that the ESX servers are targetting the MD3000 just fine.enter image description here

Part Two: The ESX Servers

scanning the md3000 Figure 1.

So as you can see above, Both ESX Servers (10.0.7.102 and 10.0.7.103) are able to see and scan the MD3000i SAN.
Storage on Vmware Figure 2.

Above is the storage both servers see. I created the storage partition on EPI2 (102). I then Extended the partition to include the second LUN for a grand total of 3.27 TB of storage.

vmwarepartition

When i "rescan" on 103 (the server not mounting the partition), I get the below log in log/messages. Mar 11 10:41:18 epi3 kernel: scsi1: remove-single-device 0 0 0 failed, device busy(4). being the only line that grabs my attentions. (EPI3 is the server name)

Mar 11 10:41:04 epi3 vmkiscsid[5436]: Connected to Discovery Address 192.168.130.101 
Mar 11 10:41:04 epi3 vmkiscsid[5437]: Connected to Discovery Address 192.168.130.102 
Mar 11 10:41:04 epi3 vmkiscsid[5438]: Connected to Discovery Address 192.168.131.101 
Mar 11 10:41:04 epi3 vmkiscsid[5439]: Connected to Discovery Address 192.168.131.102 
Mar 11 10:41:17 epi3 kernel: scsi singledevice 2 0 0 0
Mar 11 10:41:17 epi3 kernel:   Vendor: DELL      Model: MD3000i           Rev: 0735
Mar 11 10:41:17 epi3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Mar 11 10:41:17 epi3 kernel: VMWARE SCSI Id: Supported VPD pages for sdb : 0x0 0x80 0x83 0x85 0x86 0x87 0xc0 0xc1 0xc2 0xc3 0xc4 0xc8 0xc9 0xca 0xd0 
Mar 11 10:41:17 epi3 kernel: VMWARE SCSI Id: Device id info for sdb: 0x1 0x3 0x0 0x10 0x60 0x1 0xe4 0xf0 0x0 0x1a 0x1a 0xa2 0x0 0x0 0x15 0xe2 0x4d 0x75 0xf6 0x99 0x53 0x98 0x0 0x54 0x69 0x71 0x6e 0x2e 0x31 0x39 0x38 0x34 0x2d 0x30 0x35 0x2e 0x63 0x6f 0x6d 0x2e 0x64 0x65 0x6c 0x6c 0x3a 0x70 0x6f 0x77 0x65 0x72 0x76 0x61 0x75 0x6c 0x74 0x2e 0x36 0x30 0x30 0x31 0x65 0x34 0x66 0x30 0x30 0x30 0x31 0x61 0x31 0x61 0x61 0x32 0x30 0x30 0x30 0x30 0x30 0x30 0x30 0x30 0x34 0x37 0x39 0x30 0x36 0x32 0x32 0x65 0x2c 0x74 0x2c 0x30 0x78 0x30 0x30 0x30 0x31 0x30 0x30 0x30 0x30 0x30 0x30 0x30 0x32 0x0 0x0 0x0 0x51 0x94 0x0 0x4 0x0 0x0 0x80 0x1 0x53 0xa8 0x0 0x44 0x69 0x71 0x6e 0x2e 0x31 0x39 0x38 0x34 0x2d 0x30 0x35 0x2e 0x63 0x6f 0x6d 0x2e 0x64 0x65 0x6c 0x6c 0x3a 0x70 0x6f 0x77 0x65 0x72 0x76 0x61 0x75 0x6c 0x74 0x2e 0x36 0x30 0x30 0x31 0x65 0x34 0x66 0x30 0x30 0x30 0x31 0x61 0x31 0x61 0x61 0x32 0x30 0x30 0x30 0x30 0x30 0x30 0x30 0x30 0x34 0x37 0x39 0x30 0x36 0x32 0x32 0x65 0x0 0x0 0x0 0x0 
Mar 11 10:41:17 epi3 kernel: VMWARE SCSI Id: Id for sdb 0x60 0x01 0xe4 0xf0 0x00 0x1a 0x1a 0xa2 0x00 0x00 0x15 0xe2 0x4d 0x75 0xf6 0x99 0x4d 0x44 0x33 0x30 0x30 0x30 
Mar 11 10:41:17 epi3 kernel: VMWARE: Unique Device attached as scsi disk sdb at scsi2, channel 0, id 0, lun 0
Mar 11 10:41:17 epi3 kernel: Attached scsi disk sdb at scsi2, channel 0, id 0, lun 0
Mar 11 10:41:17 epi3 kernel: scan_scsis starting finish
Mar 11 10:41:17 epi3 kernel: SCSI device sdb: 3509329920 512-byte hdwr sectors (1797751 MB)
Mar 11 10:41:17 epi3 kernel:  sdb: sdb1
Mar 11 10:41:17 epi3 kernel: scan_scsis done with finish
Mar 11 10:41:17 epi3 kernel: scsi singledevice 2 0 0 1
Mar 11 10:41:17 epi3 kernel:   Vendor: DELL      Model: MD3000i           Rev: 0735
Mar 11 10:41:17 epi3 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Mar 11 10:41:18 epi3 kernel: VMWARE SCSI Id: Supported VPD pages for sdc : 0x0 0x80 0x83 0x85 0x86 0x87 0xc0 0xc1 0xc2 0xc3 0xc4 0xc8 0xc9 0xca 0xd0 
Mar 11 10:41:18 epi3 kernel: VMWARE SCSI Id: Device id info for sdc: 0x1 0x3 0x0 0x10 0x60 0x1 0xe4 0xf0 0x0 0x1a 0x1a 0x86 0x0 0x0 0xd 0xb7 0x4d 0x75 0xf2 0x77 0x53 0x98 0x0 0x54 0x69 0x71 0x6e 0x2e 0x31 0x39 0x38 0x34 0x2d 0x30 0x35 0x2e 0x63 0x6f 0x6d 0x2e 0x64 0x65 0x6c 0x6c 0x3a 0x70 0x6f 0x77 0x65 0x72 0x76 0x61 0x75 0x6c 0x74 0x2e 0x36 0x30 0x30 0x31 0x65 0x34 0x66 0x30 0x30 0x30 0x31 0x61 0x31 0x61 0x61 0x32 0x30 0x30 0x30 0x30 0x30 0x30 0x30 0x30 0x34 0x37 0x39 0x30 0x36 0x32 0x32 0x65 0x2c 0x74 0x2c 0x30 0x78 0x30 0x30 0x30 0x31 0x30 0x30 0x30 0x30 0x30 0x30 0x30 0x32 0x0 0x0 0x0 0x51 0x94 0x0 0x4 0x0 0x0 0x80 0x1 0x53 0xa8 0x0 0x44 0x69 0x71 0x6e 0x2e 0x31 0x39 0x38 0x34 0x2d 0x30 0x35 0x2e 0x63 0x6f 0x6d 0x2e 0x64 0x65 0x6c 0x6c 0x3a 0x70 0x6f 0x77 0x65 0x72 0x76 0x61 0x75 0x6c 0x74 0x2e 0x36 0x30 0x30 0x31 0x65 0x34 0x66 0x30 0x30 0x30 0x31 0x61 0x31 0x61 0x61 0x32 0x30 0x30 0x30 0x30 0x30 0x30 0x30 0x30 0x34 0x37 0x39 0x30 0x36 0x32 0x32 0x65 0x0 0x0 0x0 0x0 
Mar 11 10:41:18 epi3 kernel: VMWARE SCSI Id: Id for sdc 0x60 0x01 0xe4 0xf0 0x00 0x1a 0x1a 0x86 0x00 0x00 0x0d 0xb7 0x4d 0x75 0xf2 0x77 0x4d 0x44 0x33 0x30 0x30 0x30 
Mar 11 10:41:18 epi3 kernel: VMWARE: Unique Device attached as scsi disk sdc at scsi2, channel 0, id 0, lun 1
Mar 11 10:41:18 epi3 kernel: Attached scsi disk sdc at scsi2, channel 0, id 0, lun 1
Mar 11 10:41:18 epi3 kernel: scan_scsis starting finish
Mar 11 10:41:18 epi3 kernel: SCSI device sdc: 3509329920 512-byte hdwr sectors (1797751 MB)
Mar 11 10:41:18 epi3 kernel:  sdc: sdc1
Mar 11 10:41:18 epi3 kernel: scan_scsis done with finish
Mar 11 10:41:18 epi3 kernel: scsi1: remove-single-device 0 0 0 failed, device busy(4).
Mar 11 10:41:18 epi3 kernel: scsi singledevice 1 0 0 0

Things I've Tried:

  1. Removing iSCSI targets from only 103, disabling iSCSI, rebooting, enabled iSCSI, re-adding targets, rescan. Same result.
  2. Removing partition on 102, Formatted partition on 103 instead. Same result, except flipped. 103 can use storage, 102 can not.
  3. Starting Over. Removing all iSCSI Targets on both ESX Boxes, disabling iSCSI, turning off the firewall for iSCSI, rebooting ESX. Then on the MD3000, Removed the Host Group, Removed the Host-to-Virtual Mappings, Restarted the SAN. Followed the Documentation again, same result. Both servers see the storage, but only one server can use it.
  4. Disabling and Re-enabling VMware DRS and HA. Same result.
  5. Flat-out turning off VMware DRS and HA, and doing the "start over" step to see if maybe that borked it. Same Result.

I'm kinda loosing my mind here, Everything i read online says "just partition it and if the ESX boxes can see the targets, it just works".... well crap.

Any ideas, any other things to try? Can anyone atleast point me in the right direction? I'm really tired of working from 1am til 4am (our maintenance hours)

dell iscsi vmware-esx dell-powervault
Martin Hope
grufftech
Asked: 2011-01-10 01:31:09 +0800 CST
  • 2

So here's the situation.

I've got about 20,000 maildir email accounts chewing up a several hundred GB of space on our email server. Maildir by nature keeps thousands of tiny a****** little files, instead of one .mbox file or the like...

So i need to migrate all of these several millions of files from one server to the other, for both space and life-cycle reasons.

the conventional methods i would use all work just fine. rsync is the option that comes immediately to mind, however i wanted to see if there are any other "better" options out there.

Rsync not handling multi-threaded transfers in this situation sucks because it never actually gets up to speed and saturates my network connection, because of this the transfer from one server to another will take hours beyond hours, when it shouldn't really take more then one or two.

I know this is highly opinionated and subjective and will therefore be marked community wiki.

linux email maildir
Martin Hope
grufftech
Asked: 2010-08-12 08:39:08 +0800 CST
  • 2

I built the server based on this guide, but i skipped the samba part. (CentOS 5.x Samba Domain Controller With LDAP Backend)

I added in sudoers support based on this guide. (Sudoers LDAP Manual)

I've rebuilt the system 7 or 8 times with different settings, tweaks, all that good stuff so i have a pretty solid understanding of LDAP/Slapd and its configuration.

We're integrating our new LDAP server with our systems, however some of those servers that can't be quite updated to the latest version of CentOS yet, and I've hit a bit of a roadblock. And I've written "launch" scripts for our nodes to speed up the turn up process a bit. Below is a script that autoconfigures my CentOS machines to be a member of the ldap node.

CentOS5 Script

#!/bin/sh
#
authconfig --enableldap --enableldapauth --enablemkhomedir --ldapserver=10.0.32.5 --ldapbasedn="dc=MYDOMAIN" --update
echo 'sudoers:    files ldap' >> /etc/nsswitch.conf
echo 'base dc=MYDOMAIN
timelimit 120
bind_policy soft
bind_timelimit 120
idle_timelimit 3600
uri ldap://10.0.32.5/
ssl no
tls_cacertdir /etc/openldap/cacerts
pam_password md5
sudoers_base    ou=SUDOers,dc=MYDOMAIN
' > /etc/ldap.conf

Now, running this doesn't work on CentOS 4 servers, I had to make a few changes.

CentOS4 Script

authconfig --enableldap --enableldapauth --ldapserver=10.0.32.5 --ldapbasedn="dc=MYDOMAIN"
echo 'sudoers:    files ldap' >> /etc/nsswitch.conf
echo 'base dc=MYDOMAIN
timelimit 120
bind_timelimit 120
bind_policy soft
idle_timelimit 3600
uri ldap://10.0.32.5/
ssl no
tls_cacertdir /etc/openldap/cacerts
pam_password md5
sudoers_base    ou=SUDOers,dc=MYDOMAIN
' > /etc/ldap.conf

Now, this almost works. It joins the LDAP server and my user logins work. I had to disable enablemkhomedir which made me sad, but I'll survive.

The primary difference is for some reason my CentOS4 servers don't work with my SUDOers setup, however those configs work fine on CentOS5 servers, that doesn't really surprise me, but I do need to figure out some kind of work around.

Does anyone know the solution or can point me in the right direction to make get ldap-based-sudoers to work with CentOS4 servers?

centos ldap sudo
Martin Hope
grufftech
Asked: 2010-07-24 14:15:44 +0800 CST
  • 7

So, as we've seen all over server fault, and over the years I've built several of each system,

However at the end of the day, I sometimes feel like I've somewhat "cobbled together" what seems like it could be one or two softwares, at most.

I'm kinda looking for an "All-in-one solution thats easier to manage then having to deal with 7 different softwares. Something that just works.

Anyone know of any? I've seen tons of monitoring apps and all that, but they always are missing the wiki and ticketing portions. That never made much sense to me, in my opinion if a alert goes off that should be a ticket for an engineer to address. Same goes for the IDS going off, and maybe trending warnings for future notice.

monitoring wiki ids
Martin Hope
GruffTech
Asked: 2010-07-02 15:25:13 +0800 CST
  • 2

So, few months ago my boss throws at me "what happens we needed to scale to X clients, in a reasonably short period of time." sort of scenarios, somewhat of an engineering challenge if anything else, but of course more clients means more income.

So, right now we have a setup similar to below. Redundant Load Balancers, serving data from two webservers (sessions shared via memcache), and two backend fully redundant File Servers. Each file server has the whole shabang, RAID6 Array across the drives, hot-space drive, dual raid controllers, dual nics, multipathing yada yada yada.

simple setup

The design is one that I feel is solid, High Availability through no single-point-of failure. High Performance by splitting the load to multiple web servers, high scalability in the sense that we should be able to just keeping adding more and more machines horizontally to scale to more and more clients. The primary bottleneck being how much storage each file server could hold, and how much space is allotted to each client. That part of this would scale out faster then the rest of the system. FileServer/Client "Pools" route was chosen because it scales horizontally and is cheaper then saying "okay we need to buy an even BIGGER SAN."

alt text

So, this is all very easy, more and more online storage, means more and more NFS mounts. And that's where my second-guessing comes into play. I want to make sure I've ironed out all the potential issues in this design before putting anything in place. As long as each piece of this puzzle is properly monitored, I think its controllable, but I wanted to get some other opinions first, maybe from people who've been down this road before.

I know of a few key issues that will have to be watched for.

  • Hotspots on the file servers, or particular machines working harder then the rest of the pool.
  • Bandwidth & Backend Switching. being there will be a lot of talking between webservers to NFS devices, A high quality switch will have to be in place with a high switch fabric capacity.

Unknown issues...

  • NFS Mounts on Webservers, Will there be any issues with having each webserver have 2... 5... 10... 100 NFS Mounts open at any given time? Is there any way to make this easier or more friendly? Maybe a NFS Proxy of some kind? (that'd create an internal bottle neck, which makes me frown). I thought about a Distributed file system, and have the webservers just mount that, However it seems that Non-Proprietary, POSIX Compliant, Expandable without downtime, Internally Redundant file systems are either too immature for production work, are outrageously expensive, or are really good at hiding from Google.

Let me know what you guys think, and if you see any suggestions & optimizations, it would be greatly appreciated.

((Due it it being an open-ended question where there is no one specific "right answer," question is a community Wiki))

backup nfs load-balancing web-server centos5
Martin Hope
grufftech
Asked: 2010-04-27 11:51:54 +0800 CST
  • 3

Our Current DHCPD.conf looks like the following. 

subnet 10.0.32.0 netmask 255.255.255.0
{
        range 10.0.32.100 10.0.32.254;
        option subnet-mask 255.255.255.0;
        option broadcast-address 10.0.32.255;
        option domain-name-servers 208.67.222.222,208.67.220.220;
        option routers 10.0.32.5;

        host Dev-ABaird-W {
                hardware ethernet 00:1D:09:3E:49:13;
                fixed-address 10.0.32.94;
    }

    ... more static hosts ....

}

About as basic as it gets. The old router is 10.0.32.1, our company wanted to implement a squid proxy to better monitor web traffic while at work, and if necessary block large time-wasters, IE Facebook.com.

However, we've quickly realized that this change has played a mean prank on our Polycom SIP Phones. Occasionally our phones will not ring, the end recipient hears ringing (this is artificially created by our PBX) however the handset never rings. The ONLY thing that has changed in our network is the option routers line.

So, Since all Polycom MAC addresses begin with 00:04:F2 would it be possible in DHCP to say any 00:04:F2:::* MAC addresses get option routers 10.0.32.1, and anything else must talk with our Gateway?

centos dhcp asterisk polycom
Martin Hope
grufftech
Asked: 2010-04-16 07:29:30 +0800 CST
  • 6

Alright. Some background. We have an anti-spam cluster trucking about 2-3 million emails per day, blocking somewhere in the range of 99% of spam email from our end users.

The underlying SMTP server is Postfix 2.2.10. The "Frontline defense" before mail gets carted off to SpamAssassin/ClamAV/ ect ect, is attached below.

...basic config....    
smtpd_recipient_restrictions =
            reject_unauth_destination,
            reject_rbl_client b.barracudacentral.org,
            reject_rbl_client cbl.abuseat.org,
            reject_rbl_client bl.mailspike.net,
            check_policy_service unix:postgrey/socket
...more basic config....

As you can see, standard RBL services from various companies, as well as a Postgrey service.

The problem is, I have one client (out of thousands) who is very upset that we blocked an important email of theirs. It was sent through a russian freemailer who was currently blocked in two of our three RBL servers. I explained the situation to them, however they are insisting we do not block any of their emails.

So i need a method of whitelisting ANY email that comes to domain.com, however i need it to take place before any of the recipient restrictions, they want no RBL or postgrey blocking at all.

I've done a bit of research myself, http://www.howtoforge.com/how-to-whitelist-hosts-ip-addresses-in-postfix seemed to be a good guide at first, almost fixing my problem, But i want it to accept based on TO address, not originating server.

email spam postfix rbl
Martin Hope
grufftech
Asked: 2010-03-12 11:53:33 +0800 CST
  • 4

So, I've been reading several articles around ServerFault as well as google. (For Example, this link)

My Requirements are very similar to the link above, however i'd like to also have dynamic or at least resizeable file volumes, so if necessary i can add 4-5 servers to the pool, and then expand the volume.

Any Distributed File systems that support that, to save me some time? Thanks!

LustreFS will be my next test cluster to build.

GlusterFS

I've build a 3-machine test GlusterFS cluster, However i quickly became aware of several of its limitations that it doesn't seem to make clearly public.

One, i can't seem to resize a volume. Once a volume is created, its done. Which seems retarded, why have a fully scalable file system if i can't scale a volume? So maybe i'm doing something wrong. I'm not sure.

AmazonS3 while gives the cheapest startup adds too much cost when broken down to per client per month, so its out. Building my own system when prorated over several years with no bandwidth costs makes it significantly cheaper.

MogileFS isn't an option as we'd like this server to be a SAN-Replacement, for storing tons of media from a multitude of systems, which for us means it needs to be POSIX compliant so it can be remotely mounted via NFS or CIFS.

cluster distributed-filesystems glusterfs
Martin Hope
grufftech
Asked: 2010-01-20 15:07:48 +0800 CST
  • 12

Okay, our datacenter doesn't charge us based on our "useage" just on how many Amps we have delivered to our rack. I'm planning on putting in a new rack this upcoming year for some server expansion, based on This article. Each server has a crapload of drives, and in turn, uses a crapload of power.

This rack will include.

1 Cisco Router.

1 Fiber Switch

1 1U Storage Controller, standard Dell 850 server.

11 3U Storage Servers, each with dual 750-watt power supplies.

So... I'm trying to calculate how many amps i should theoretically need. I dont have an example server yet to see just how much a 3U - 45-drive server pulls. So i figure it needs two power supplies, so lets go with 1000 watts of pull. Plenty for the hard drives, plenty for the onboard stuff. 11 x 1000 watts is obviously 11000 watts of power.

A=W/V. W= 11000. V= 120v. A= 91.

This is where my brain kinda tripped out. 91 amps of power is sheer insanity. (the most i've had piped to a rack atm is 40, and its only sitting at 60% capacity.

And thats with only 1000w used per server -- if the PSU is pulling full current at 1500watts for the system, thats something to the tune of 140amps.

.... This sound right or i'm I just crazy? Surely i did something wrong, 140amps of power is insanity.

****Edit: Box is actually 4U, so numbers are different then displayed here, but not by much.

storage electrical-power
Martin Hope
grufftech
Asked: 2009-08-21 08:27:50 +0800 CST
  • 2

We have a server that sends our weekly newsletter to subscribers, To prevent people like Hotmail or Yahoo from blocking us due to sending too many simultaneous emails to them, Is there a way we can stagger email, or rate-limit outbound emails from postfix?

Keep in mind, I dont want the mailserver to stop queueing mail or accepting new messages, Simply defer delivery if there are more then 3-4 messages per destination domain/ipaddress, or something similar.

Note: I dont want a Sender Throttle, as described in a similar question, here. I'm looking more for a recipient throttle but haven't had any luck finding out how to do so with PolicyD or Anvil services, and was wondering if anyone else has accomplished such a task.

centos email postfix
Martin Hope
grufftech
Asked: 2009-07-24 11:09:45 +0800 CST
  • 2

This is borderline programming, but i'm posting here first.

Output of PHP info

apache2handler
Apache Version  Apache/2.2.3 (CentOS)
Apache API Version  20051115
Server Administrator    root@localhost
Hostname:Port   localdomain.com:0
User/Group  apacheuser(500)/500

I'm trying to find a way for PHP to retrieve the "Hostname:port" field in this area.

We have apache that dumps all our virtual host information into a single file, and then that single file is broken up into several files & relevant information (total hits, ect) is logged to a database. One of the problems we're having at the moment is if a domain comes in on a ServerAlias, our php software isn't able to find the database table containing their logs (because the domain is technically not correct.) However, the "Hostname:port" field in php / phpinfo(); Always retrieves the information i want, which is the primary domain name.

If there is any other method PHP can gain the "ServerName" field from apache that i'm unaware of it, please let me know as that would be a lot easier.

An Alternative approach is having a SetEnv command in every virtual host, however instead of having to go through and add a

SetEnv primarydomain localdomain.com

to every VirtualHost, is there a way to use a variable?

In Example:

SetEnv primarydomain %v

Thanks!!

---------------------update---------------------------

We solved the problem with a regular expression search & replace.

Search: ^ServerName\s\b((?=[a-z0-9-]{1,63}.)[a-z0-9]+(-[a-z0-9]+)*.)+([a-z]{2,63})\b

Replace: $&\nSetEnv\tprimarydomain\t$1$3

php apache-2.2
Martin Hope
grufftech
Asked: 2009-07-18 12:58:22 +0800 CST
  • 2

Our frontend webservers are now logging all our weblogs into mySQL using mod_log_sql, freeing up thousands of "AccessLog" directives in our apache config (we're running between 600-900 virtual hosts on our servers now)

That being said, I'm trying to find a reasonable weblog analyzer that works with mod_log_sql. I've used webalizer & awstats for years and i really like them, however neither tool supports sql-based logging.

It doesn't have to be live-time, but it does atleast have to be able to grab data from a database table.

Anyone have any suggestions?

linux centos apache-2.2 httpd
Martin Hope
grufftech
Asked: 2009-07-07 14:53:55 +0800 CST
  • 2

After a bit of googling / research on my own i haven't found a definitive answer,

I'm looking to search the entire /var/www/html/ folder for occurances of "127.0.0.1" and replace them with a server IP address (we're moving the database off of the local box).

This goes for Localhost & 127.0.0.1. While i could just run

find * | xargs sed -i a/127.0.0.1/ser.ver.ip.addr/g

twice, once replacing 127.0.0.1 and localhost, i'd rather not have to run two commands.

Am i able to do a OR statement without it breaking?

Something like: 

find * | xargs sed -i a/127.0.0.1|localhost/ser.ver.ip.addr/g

----------------------------edit----------------------

I found a slightly more reliable method of doing a search & replace, 

find * -type f -exec sed -i 's/127.0.0.1/ser.ver.ip.addr/g' {} \;

However i can't seem to get the search statement to match 127.0.0.1 OR localhost.

linux centos scripting command-line-interface shell