Alex Lehmann's questions

I am using a system with two machines running mysql in master/master replication.

I would like to move the systems to new machines who are also doing master/master replication. Is it possible to do that without removing one node from the old cluster and replacing it with a new node and creating a new replication from the first new node to the second?

Let's say we have two nodes node1 and node2 that are active and doing master/master, how do I "magically" move the cluster to new nodes called new1 and new2 without too much downtime.

Ok, first of all let me describe my working situation so that is is clear why this is a question pertaining to system administration. I work at a larger IT solution company that is running operations for a large scale email service (5 million users) for a major telco company. One of the major issues with this portal is users being compromised by bot nets so that they can be used to send spam mails in the order of probably 100 000s per day. The complaint email address for the portal receives ARF reports at a rate of maybe 1 per minute amounting to maybe 1000-2000 per day. Any of these reports would have to be looked at manually, analyzed processed and archived. Since there is no man-power assigned to this tasks, the reports are basically ignored resulting in increase in spam to a level that the portals email server are all blacklisted on time or another. The reason why I would like to have a framework to automate this would be to identify the legit reports to be processed automatically while some reports still have to be analyzed by a human which could be delegated to off-shore people if the tasks are sufficiently easy to describe and to execute. I should stress that I do not need help implementing the steps, most of these are 1 line shells scripts plus a few ones accessing external tools, I am searching for a tool to string together the different steps.

I was thinking of something along the lines of Gearman meets procmail meets Jenkins meets RequestTracker meets Ansible (meets Actor model if you like), but I assume that such a tool is not available. There have to be some tools to automate at least some parts of this, since the complaint processing has to be an issue on any medium to major scale email portal where more than one person is running the email system.

(I hope this explanation is sufficient to motivate the relevance of the question, if not, I really do not what is. Or maybe this question is place wrong in this SO portal, but if so which one is applicable?)

I would to automate a few tasks that process files in some way, do some decisions, call some scripts and in some cases have some tasks that require human intervention.

My example use case is the following (when I do the whole thing by hand)

• receive an email spam complaint, which I open in editor to be able to search through
• decide if the email fits some criteria (e.g. it is really sent by a user in our systems, isn't older than 5 days etc)
• extract the username of the user that has sent the mail
• search the customer database if the user is already disabled
• disable the user if still active
• search mail queues for other offending emails by the same user
• move the mails to a holding area to analyze later
• send the customer service people an email that a user was disabled (probably with a collect list of disabled users once a day)

Most of these steps could be easily automated with e.g. perl scripts, but I have no idea how to automate the consecutive steps, especially if there are conditions and preconditions and some tasks could be collected together and run if there are a few todos already pending (e.g. search through the queues only if either 10 users have been found or the task wasn't run in the last hour or so).

Some decisions may require human intervention (e.g. looking at the mails that could not be identified automatically) or tuning of the decision process so that it would be helpful to be able to run some steps again with a changed script.

I assume that it would be possible to have some kind of job scheduler, but I wonder how to define the necessary steps and conditions and preconditions.

(Please note that I do not search for a tool that can do the described tasks, most of these are easily implemented as scripts, but a tool to organize the sequence of tasks that have to be done after each other for a given task data)

I would prefer a FOSS solution for this running on Linux.

When building our web pages from different content sources, it may be necessary to get some images from external servers (e.g. when incorporating a rss feed), which may not be as fast or as well connected as our own data center. I would like a have a means to copy respectively proxy the files into a server address that is running at our site to keep the load off the external servers possible changing the filename to hide that fact that the images were dynamically generated.

e.g. turn the following url

http://domain.de/content/query?file=foo/nr_1.gif

into something like this:

mydomain.net/static/domain.de/query_3fresource_3dfoo_2fnr_5f1.gif

This should honor etags, if-modified-since change expires headers to make the files static and cacheable, regardless of what the originating server says.

I think I could build something like this using varnish and another web server, but maybe there is a solution already available.

This could be part of a CDN, however I do not anticipate the necessity of a real CDN since we do not have many visitors from other countries