Josh Johnson's questions

I recently made the move to The Cloud from a bare metal server for personal use and I couldn't be happier. Except for one thing: My server crashes about once a week.

Rackspace has been really great and extremely helpful and I wish to stay with them and recommend them to others. But with this issue I have been told that the server is just running out of memory and I need to look into it. While I'm fine with that and this does seem like my issue, I've never heard of a Linux machine crashing from running out of memory. In my experience it would, worst case scenario, run really slowly or the kernel would start killing processes.

Some Details:

• Running Gentoo Linux (up to date)
• 512MB RAM, 1G Swap
• Services installed & running:
  • 2 Apache 2 procs (1 minimal, for serving static/cached resources and proxying; 2nd has cgi, mod_perl and mod_jk)
  • 2 Tomcat Instances (1 has 2 apps I made, the other is just for Nexus)
  • Mysql
• When the crash happens:
  • server can be pinged
  • ssh connection hangs indefinitely
  • console will allow username but password prompt never comes up. after 60 seconds it times out and I'm presented with another login prompt. repeat.
  • services (http, tomcat, mysql) cannot be contacted; The connection will hang and not report 404, server not found, etc.
  • Softboot will not work, hard boot needs to be performed often more than once (even with a 30 min wait between reboot attempts)

I've run this same setup on servers throughout the years but with only 256MB RAM. The only difference here is I'm on a virtual machine.

My question could take one of two forms: Has anyone had a similar problem with Rackspace cloud or other cloud hosts? (and if so, what was the solution?) or; What's a good way to track down my issue? I setup a cron that outputs uptime and free to a file every minute so I can examine it after the next crash but that seems hacky.

Full Disclosure: I'm a software developer by trade so that's where most of my experience is, but I have about 15 years experience using Linux for desktop and servers both for personal and professional use.

I have Apache setup to authenticate with active directory through ldap for my users. There are a few "system" users (for automated build tests) that are manually setup and authenticate through file.

AuthBasicProvider ldap file

The issue I'm having is that the Active Directory box is wont to go down once in a while for maintenance or other reasons and I don't want my "system" users to be denied access during that time. I'd like Apache to attempt authentication with file and THEN ldap but this doesn't seem to be possible. According to http://httpd.apache.org/docs/2.2/mod/mod_auth_basic.html, "the order of processing is determined in the modules' source code and is not configurable."

This just doesn't seem right. I would think that one would (often) want to specify the order of the auth providers (if not for the reasons I want to then for other reasons). Is there no way to do this without hacking source code?

p.s. Changing the config to AuthBasicProvider file ldap makes no difference.