Matt White's questions

The use cases for the AWS Storage Gateway always seem to assume that the virtual storage gateway would be installed on-premises (i.e. local to the customer) and would then be synced to AWS. I am wondering about placing the storage gateway on EC2 instead.

My use-case is this:

I have a large amount of customer data that must be accessed via the filesystem (i.e. it cannot be simply stored in S3). A large percentage of this data is accessed very infrequently. I use a number of EBS volumes mounted to EC2 to accomplish this, and snapshot them to S3. However, the storage is expensive, and there is a single point of failure since EBS drives can only be attached to a single instance.

However, it sounds like I could install the virtual storage gateway on my EC2 server instead, and use the new Gateway-Caching feature to only keep the frequently-accessed data locally (using the free instance storage for the caching).

Additionally, I've read that iSCSI drives can be mounted on multiple servers as long as they are formatted with a cluster-aware filesystem.

My questions:

Is this a reasonable use-case for the Storage Gateway?

Can the Storage Gateway iSCSI drive really be mounted from multiple EC2 instances, eliminating the single point of failure?

Here's the scenario: the user runs "ssh -i sshkey user@server1". I want the SSH connection to be tunneled through server1 to server2. Normally, the user could do this himself using:

ssh -i sshkey user@server1 -o 'ProxyCommand /bin/nc server2 22'

However, I don't want the user to set up the proxy themselves, using ssh command-line arguments or even ssh_config changes. As sysadmin, I want to be able to redirect a user's SSH session to a different server transparently. All solutions I've found so far require ProxyCommand. Is there a way for me to accomplish this?

Note that the user is using SSH keys, not username/password, so those credentials need to be passed on to server2.

I am currently using Monit to monitor Apache and restart it if its memory usage is too high. However, I'd also like to be able to monitor the individual apache2 subprocesses that are spawned, and kill any subprocess whose memory usage is too high over a period of a few minutes. How can I do that?

I would like to create a mod_security collection that is based on something other than the user's IP, Session ID, or username. Specifically, based on the REQUEST_URI. However, mod_security indicates that this is not possible. My question is whether there might be some workaround.

I tried the following, but couldn't get it to work:

SetEnvIf Request_URI "^\/([a-z0-9]+)\/" account="$1"
SecAction setsid:%{account},pass,nolog
SecAction "phase:2,pass,setvar:session.count=+1,nolog"
SecRule SESSION:COUNT "@gt 60" "phase:2,deny,status:503,skip:1,nolog"
SecAction "phase:5,deprecatevar:session.count=1/1,pass,nolog"

The above should extract the account name from the URL (line 1), and use that as the session id (line 2; this is what doesn't work). It then increases the request count on each request that includes that account name (line 3). If the count goes over 60 (line 4), a 503 error is returned. The count is decreased by 1 every 1 seconds (line 5).

I am new to using tc and netem. I want to delay packets being sent to a specific IP address. However, the commands below cause all packets on the system to be delayed, instead of just to the IP address 1.2.3.4:

tc qdisc del dev eth0 root
tc qdisc add dev eth0 root handle 1: prio
tc qdisc add dev eth0 parent 1:1 handle 2: netem delay 500ms
tc filter add dev eth0 parent 1:0 protocol ip pref 55 handle ::55 u32 match ip dst 1.2.3.4 flowid 2:1

My guess is that I need some kind of catch-all filter at the end to specify that all remaining traffic should not go through netem. But I can't get anything to work. How would I get this to work?

In Apache 2.2, is there a way to forward certain subdomains to another server (in another datacenter) based on a lookup table of subdomain/IP address mappings (provided in a file or database)?

The reason I need this is that I have a SaaS web app where each customer gets their own subdomain, i.e. customer1.mycompany.com, customer2.mycompany.com, etc. If Apache receives a request for a customer whose data is not stored on the current server, I want Apache to simply forward (proxy?) the request on to the correct server. The number of customers will be continually increasing (hopefully!) and I cannot reload Apache each time that happens.

UPDATE:

Our current idea is to use GeoIP DNS to send customers to the correct (nearest) server, and if a customer is accidentally sent to the wrong server (e.g. they are traveling), then we would need to do the proxying. However, even if we added a DNS entry for each subdomain, there could still be requests that end up at the wrong server when an account is first created, or when it is moved from one server to another. For various reasons, we don't want to set a really low TTL on the DNS entries.

There are many rate limiting and QoS tools out there, but so far I have not been able to find one that meets my specific needs, as follows:

I have a web app running on Apache 2.2 and I want each of my customers to be limited to, say, 10000 requests per day. The limiting should not be based on the customer's IP address but on the customer's account, which can be extracted from the URL; e.g.: http://mywebsite.com/customer1/page1. Once they are over their limit, I would like requests from them to be "slowed down" to some predefined value, such as 15 requests per minute, perhaps by introducing a delay. If that's not possible, then returning an error like 503 Service Unavailable would be ok.

How can I do this with Apache? If there is no way to do it with Apache, then I'd be interested in suggestions for other tools, reverse proxies, etc.

UPDATE: The rate limiting needs to happen across a number of services (mod_dav_svn, mod_passenger, mod_wsgi) and must be high-performance.