Sandman's questions

So ImageMagick has the "convert" command which I use for my (Linux) web platform to deal with image resizing and such.

Sometimes this command gets "stuck", meaning it starts eating machine resources until the entire machine is unavailable. Logging in to the machine can take minutes when this happens.

SO, I'm looking for one of two solutions:

1. Upon using this command - is there a way to determine just how much max resources it may use?

2. Can I run a cron script that identifies these "stuck" processes and kill them? I.e. filter on CPU time, or CPU usage and kill when over a specific threshold.

I would probably prefer solution number 2, since solution number 1 could still yield several such processes that each is limited, and stuck, and together they would still eat up my resources.

I can't identify when and why this is happening, the system is slugish and stuck until I "killall convert", then all is well. This command is run thousands of times each hour so unless I would log each and every usage in some way, I can't say which one got stuck, unfortunately.

So, basically, a shell script that identifies, kills and logs when convert gets stuck.

Any ideas?

I am currently being flooded by requests to /wpad.dat - in a manner that makes it seem to be an attack.

Now, what I would like to do is to trigger iptables for every IP that requests /wpad.dat and then add it to a blacklist. Is there a way to do that? That means that the IP will be able to send one flood request and then none.

Any neat way to do this efficiently?

So, my apache server was slow, and I looked in the log files. Turned out they had grown to 12GB of accesses from tons and tons of different hosts trying to access /wpad.dat on one of my Vhosts.

Now, the virtual host in question is the "catch-all" vhost that is invoked when a browser doesn't supply a known hostname.

I am currently getting thousands of requests per minute to "/wpad.dat" and as far as Google can tell me, this is something that has something to do with proxy servers? But I don't use proxy servers, so why am I being literally bombarded by these requests.

I am getting more requests per minute for this non-existant file than I am getting normal requests. So my assumption is that I am under some form of attack. Funny thing is that it generally only occurs at night (here in Sweden) and not during the day.

A sample size of the latest 500 requests (i.e. half a minute) shows that it consists of 200 different hosts, and a small sample of those shows that they are all valid hosts (not TOR proxies) so is this some DNS servers being incorrectly configured? I do run a DNS server on the machine.

Please help! :)

EDIT The host they are accessing is "cluster.atlascms.se" so what they do is access http://cluster.atlascms.se/wpad.dat thousands of times per minute.

Now, cluster.atlascms.se is my DNS failover host. So all my clients point their subdomains to cluster.atlascms.se, which in turn points them to the current IP (master server of failover server).

As it seems - this means I am getting tons and tons of requests to cluster.arlascms.se - could that mean that my DNS is misconfigured?

I asked this question on Stack Overflow, then I released that serverfault would be more appropriate, I apologize if anyone stumbles upon these two issues.

So, I am currently rebuilding my web platform from a single-machine to a cluster of machines, and I will be using Apache load balancing to do this., but I have two questions that I need a good answer to before proceeding. I have Googled and searched here in SF, but didn't find anything good.

My setup will be one Debian machine running the Apache load balancing server (i.e. Apache with mod_proxy) and then any number of "slave" machines, that are balancing members. All of these are VPS inside a VMWare machine, so setting up new slaves as needed will be trivial.

Log Files The first question is that of log files. In order to troubleshoot my platform, I sometimes need to analyze log files, both access logs and error logs, from Apache. When the load is evenly distributed (i.e. I don't know if I'll even use sticky balancing, any host could probably handle any request at any time), so would the log files for each slave Apache instance. Is there a way to consolidate these live, meaning that my live log analyzer could see the log files from all hosts? I certainly understand that doing so while the files being on several hosts would be difficult, so is there a way to make sure that all log files are kept on one server?

I'm thinking about two things myself, but I would greatly appreciate your input.

syslogd The first is syslogd, where in it would be possible for several hosts to write to one logging host. The problem with this is that in my current setup, each virtual host in apache, has its own log file. That could probably be fixed in some manner though. My main usage for this is for troubleshooting, not keeping separate logs for each host (albeit if both goals could be met, that would certainly be a bonus).

NFS My next thought was about NFS, i.e. having a NFS share on the LAN where each slave can write to the same log file. I'm going to go ahead and assume that this will be difficult since slave 1 would open the log file and then slave 2 wouldn't be able to write to it.

As I said, your input is greatly appreciated since I feel I'm stuck in how to solve this.

Configuration files This is another thing altogether. Each slave will respond to each request as if acting as one single server. That is the entire idea. But what about making changes to the apache configuration files, adding virtual hosts, setting up other parameters? What if I have ten slaves, or fifty? Is there a way to make sure that all these slaves are always in sync? I am already using a NFS export to make sure they all have the same files, but should I use the same approach with the configuration files? Or should I have these as some form of repository and then use rsync to copy them out to the slaves? One problem is that I have built an interface in my web platform that edits these configuration files (namely the file with the virtual hosts), and since that action would take place on one of the slaves, the most current copy of this file could potentially be on one slave.

I realize that this was a long and wieldy post, and I apologize. I just wanted to make sure that all the parameters of my problem were expressed.

I hope someone out there can help me, as you have before! Thank you in advance!

I made some searches here and didn't find anything relevant enough for me. I'll try to be as brief as I can.

I currently have OSX servers, three of them. I am moving to a virtualisation enviroment for redundancy and since Apple has cancelled the Xserve. I have no prior experience with virtualisation, but I'm a quick learner and I have administered servers since 1997.

So, new dedicated server in new data center. Kickass hardware, installed VMware ESXi and inside that, I've created two networks, "VM Network" and "Switch". "VM Network" is connected to the physical NIC and "Switch" is a virtual switch.

Then I created a VPS named "FW" that has two virtual NIC's, one to each network in VMware, installed pfSense on this virtual machine and bridged the "real" NIC with the internal Switch. Done.

Then I intend to create my resource pool on the "Switch" network and let pfSense load balance it.

My Web application is currently contained in a single directory that weighs in at approx 300GB of data, my database is about 50GB more. They are currently on Xserve 1 and Xserve 2.

My current thought process has me at these VPS machines:

1. Main server (10.0.0.10) This contains two virtual hard drives, one small for Linux and one large for my web application. The larger is mounted inside Linux as "/Atlas" (name of my CMS) and is then NFS exported. VPS has 2GB of RAM and 2 CPU's

2. Database server (10.0.0.11) This is the master database server

3. Web servers (10.0.0.20 - 10.0.0.30) Ten Linux servers, each with 5GB of HD, 5GB of RAM and 4 CPU's. Each mount the NFS export from #1 and can read and write to my Atlas.

My questions are:

1. Should I run MySQL slaves on each Web server in 3? I only have about ~900GB in the server, so space is a concern.

2. If not, should I have seperate MySQL slave failover servers in the same VMware, in case something happens with #2?

3. I will rsync-backup the Main server to an offsite location, but is there a better way than NFS to let every web server read and write to this data. Using a cluster filesystem would lead to space concerns, rigth?

4. Should I use a VPS for load balancing and use Apache's own LB? Is that better for a web application than using pfSense's LB?

5. Any other weak points in my thoughts here? I will use this as my main server, but I will have a failover server set up identically in another location and use DNS servers to reroute the traffic when needed (either automatically or manually).

I'm not building the Fort Knox here, but I need better redundancy than I have today, and I want to do it as "correct" as possible. Any help/comments are greatly appriciated!

So, I've been using cronolog for years with this directive:

<VirtualHost *:80>
  SetEnv origin SND
  ServerName sandman.net
  CustomLog "|/usr/bin/cronolog [..]/logs/SND/log_%Y-%m.txt" combined
</VirtualHost>

Problem is that with a lot of virtual hosts, I suddenly reach my maximum of open file descriptors when apache is spawning cronolog.

With some googling (and searching here) I've found no solution for this, so is there none? As you can see, I set an ENV variable in Apache to "SND" for that vhost, what I would LIKE to do is something like this:

CustomLog "|/usr/bin/cronolog [..]/logs/${origin}/log_%Y-%m.txt" combined
<VirtualHost *:80>
   SetEnv origin SND
   ServerName sandman.net
</VirtualHost>

Which, theoretically, would spawn just one cronolog process that writes to the correct vhost log file.

But it seems that the pipe commend isn't parsed for Apache variables in any stage of the process, which means that the only solution that I can see is that I add ${origin} to the LogFormat and then write my own log rotate script that will parse out (and ignore) the vhost variable and then write the rest of the log line to the appropriate file. But will this be less overhead?

Any other suggestions?

So, whenever a user tries to fetch a file on my Apache server that doesn't exist I get a 404 line in the Access log:

[29/Sep/2010:12:14:45 +0200] "GET /asdf HTTP/1.1" 404

And I also get this in the Error log:

[Wed Sep 29 12:14:45 2010] [error] [client] File does not exist: /www/site/asdf

Is there a way to suppress 404 erros from showing up in the error log- only in the access log?

So, when I want to link my users to a specific page I always use (in PHP):

"http://" . $_SERVER["HTTP_HOST"] . "/page.php"

to be sure that the link points to the page they're currently surfing (and not one of the server aliases).

But with IDN names, HTTP_HOST is set to xn--hemmabst-5za.net (for example) - which of course works but doesn't look very nice. Is there a way to have HTTP_HOST set to the correct IDN name in these cases (in this case - hemmabäst.net)?

I rather do it in Apache before it comes to PHP because otherwise I'd have to replace all my usage of $_SERVER["HTTP_HOST"].

Any ideas?

So I want the user to be able to run write "foo.phps" for a "foo.php" script in order to present the source of that file.

I only want to do this in one directory (since it's a security risk). Can I use .htaccess?

I added a .htaccess file to the directory and wrote this in it:

AddType application/x-httpd-php-source .phps

And then restarted Apache, but I can't add "s" to the file to get the source. Do I need to do something with PHP as well?

So I have a virtual server set up for www.company.com:

<VirtualHost *:80>
    ServerName www.company.com
</VirtualHost>

And then I would like to direct *.company.com to another site. How do I do it? I could think of www. and inside. being directed at specific vhosts but "the rest" directed to a generic vhost.

Apache version is 2.2.4