Patrick R's questions

I've seen plenty of posts outlining the process of converting VMware Linux serves to RHEL Xen (not xenserver) but nothing much has helped me go the other way. I've tried a number of things including using mkinitrd (works great for LVM) but just can't get it right in this situation. Truth be told, I'm a newb at xm commands but I've gained a lot of ground.

Does anyone care to share the process they've followed to successfully complete this conversion? I have access to two environments currently:

1. the original xm environment that has two Doms in it
2. a successful vmware build after running vmware converter that I can boot in rescue mode (but not boot otherwise).

I realize also that XM images aren't bootable and need to overcome this (again I've tried but must be missing a detail).

I'd like to experiment with setting up xm for the conversion before running vmware converter since I feel like I've exhausted the process from the standpoint of linux rescue mode after successfully bringing all data over via vmware converter. (but I'll go either way if I can just get a couple of helpful hints...)

Forgive me for the length of this question... it is mostly details... only attempt to follow if you also enjoy reading log files... or drinking coffee.

I'll state the questions first:

1) how the heck did a nano process fire off based on what I've stated below

2) how did nano manage to take so much resource

3) working with ossec restarts surely isn't a coincidence so is that related?

This is a Red Hat 4.1.2-46 XEN environment, three cluster members. We updated our Hurricane monitoring code manually on Jan17 at 11:34am. Two files were changed (using nano) while ossec was running:

preloaded-vars.conf
ossec.conf

ossec was then restarted and then the root user logged off.

Unfortunately the three servers went offline (still had ssh) because a nano process ran away (I imagine that this would have happened had I used VI - so the editor type is not in question). Oddly, no crons started the nano service and no one was logged into the server at the time, and I'm sure that I properly closed out of nano. Before I killed the PID, top provided me with the following insight:

Mem:  28359680k total, 28325064k used,    34616k free,     3424k buffers
Swap:  4194296k total,  4194296k used,        0k free,    70208k cached
PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
26351 root      18   0 29.7g  25g  784 R 100.1 95.6   4424:38 nano

Note: the nano editor took up ~28GB of ram.

It took just over three days for this to take our servers down. I found something else before I killed the process. Notice that the nano process began two hours later after the file was first edited and root logged off. Notice that the tty = ?.

# ps -ef | grep nano
root      7836  7689  0 13:19 pts/5    00:00:00 grep nano
root     26351     1 99 Jan17 ?        3-01:44:46 nano /opt/ossec/etc/ossec.conf

Thankfully after I killed the PID I had:

Mem:  28359680k total,  1189924k used, 27169756k free,     4584k buffers
Swap:  4194296k total,   260284k used,  3934012k free,   104352k cached

I first expected to find that the process status would be stopped or traced but it was running (see the R before the %CPU usage stat)

Additional Notes. The preloaded-vars.conf file was created from a .tar file (therefore the 1000:1000). It was edited by root. The .sav file was created when I killed nano (and it's smaller than the main file). On two of the Xen servers nano was stuck editing the preloaded-vars.conf and on the third nano was stuck editing the ossec.conf file. No ossec.conf.save was create when nano was killed.

-rwxr-xr-x  1 1000 1000  2918 Jan 17 11:04 preloaded-vars.conf
-rw-------  1 root root  2909 Jan 20 13:13 preloaded-vars.conf.save

Further Findings: I've discovered that if I open the preloaded-vars.conf file and then from another terminal kill the pid, the default behavior of nano is to create a preloaded-vars.conf.save file when it receives a SIGHUP or SIGTERM message. Still don't understand what caused it to go off the rails to begin with.

Here's the set up:

• Two Ubuntu 9.10 servers - LAMP environment (not really an option to switch OS)
• One MD3000i with iscsi connection

Here's what I'd like to do:

• Have both server see the files that the other one writes to the MD3000i without unmounting the drive. My connections are fine to the MD3000i (using open-iscsi). The MD3000i is great in almost every way I need except for one listed below.

Here's what I don't want to do (for reasons I'm not going to go into - been down this road already):

• Use NFS mounts

Example:

• Server1.com and Server2.com write to the same drive on the MD3000i. The folder that they are both mounted to is labeled /mnt/files/. They both recognize the partition size correctly.

  Then Server1.com writes test1.1.txt to /mnt/files and Server2.com writes test1.2.txt to /mnt/files. Files are written during ssh connections directly on server in this situation. Server1.com does not see test1.2.txt without remount. Server2.com does not see test1.1.txt without remount.

  Oddly, if I write the files from an apache server via PHP and the file has a unique name, both server's see it! But if the name is not unique I then have an I/O error (no surprise).

  I'm familiar with gfs, gfs2 and ocfs2 but far from competent. I've tried various methods for formatting the drive and mounting it. It seems as though I just can not get the recipe correct.

Question:

What are your suggestions? I've tried many different ways to set up what I'm been lead to believe is the holy grail - clustering - with no real success. I know what doesn't work. Could really really use a boost from some helpful tips.

Edit: I'm thinking about marking this question as answered since I've realized that I was pretty confused by the fact that file sizes were zero. This caused me to have a brain fart and panic. After reading the answers below I realized that I've often read these files before but it never occurred to me to check their filesizes. Therefore my question is misleading. I should have only asked why the file sizes were 0 only.

This is not a question about missing ram - but it may be a question about file sizes of 0. Am I looking at files that have zero file size because they are aliases to data in ram rather than on disk?

I understand that procinfo gathers information from the /proc directory on a linux box. But I've taken a look in that directory and can't figure out what files it actually queries. It appears as though most files are 0 bytes. What's really odd is that a particular file, kcore is listed as 128T on my system.

I've read that kcore represents the amount of ram on my system but I am sure I don't have 128T of ram (I have 12GB). I took a look at a second server that I rebooted and the file was 884M (that server has 1GB of ram).

I believe that, whether or not kcore is being used, procinfo is providing statistics based on data in ram. And I believe that is verified by procinfo's inability to keep data after a reboot.

I have two questions:

1. Would someone care to clarify these assumptions and add some helpful observations about this performance tool?

2. What is kcore if it's not a representation of the ram on my system (as I've read)?

Experienced DNS Admins please help me understand...

I run a small DNS server in a development environment that runs resolution for at most 5 domain names at any one time. My question stems from a discussion here: What's the difference...

My question is this:

• What should I use when I have a domain name and at least one subdomain pointing to that domain?

One Answer is this:

• I argue that the appropriate solution to this is to use Cnames. This will allow me to change an IP address once and affect multiple hosts names (the cnames) at the same time. This will help me keep things uncomplicated. This will also make managing TTL records much more straight forward and reliable. I am prone to making mistakes so this will also limit my chances of making mistakes.

Another Answer is this:

• Others argue that queries will be speed up by using A Name records (I don't necessarily argue against that fact).

I know that either will work but the adopted standards based on the RFCs for DNS lead me to believe that we should use Cnames and A Names correctly and were appropriate. Rules can be broken, but what's the point if you can't rely on them.

Please only respond to this question if you run a large DNS server or have experience doing so. Otherwise don't expect your opinion to change my opinion. I've been told that it's all a matter of personal preference and I'd like for this to be otherwise.

Forgive me if you believe I'm belaboring a very old point.

My question is this:

Can anyone suggest a linux server that runs mysql really well with really low overhead (other than Ubuntu)?

And here's why I have this question:

I have a PE1950 III that has given me no end of trouble. This isn't my first PE1950 but it is my newest. I've attempted to install at least 6 different versions of Ubuntu on this server (including 9.10) but I always get the error after the installation completes and I reboot the system.

usb... device not accepting address2... BusyBox

I've tried troubleshooting the issue with limited success. Oddly, I can run any distro of Ubuntu on this box as long as I have ESXI 4 installed first.

But the error message and troubleshooting it are really beside the point now since I'm thinking I'd like to try another distro like Damn Small Linux or something like that.

The only other item that I'll be running on the box is sendmail. I'll probably config iptables as well.

Before you read this, please note that I understand the benefits of caching. I'm familiar with the dogma that unused ram is wasted ram.

This question is one that I've adapted from a previous question:

deleting linux cached ram

In that question I was curious about adjusting how my server uses and caches ram. The system is fairly dynamic so I believe that the cached files doesn't really afford me much gain. Additionally, we have code on the server that has to quickly access large amounts of ram in short periods of time to process video files and I believe that I'll see a performance benefit from directly handing of ram rather then clearing it from cache and then handing it off.

I'd like to find out if any of you have experience with adjusting the default value of 60 in the following file (this happens to be on an Ubuntu server):

/proc/sys/vm/swappiness  

And if so, what affects did you see. If I replace the default value of 60 with 30 will I see less swapping and a lot more reuse of stale cache? Do I approach 0 or 100 to decrease swapiness and increase reuse of cache?

Finally, anyone know why the default is set to 60?

NOTE: If it's close to 0, Linux will prefer to keep applications in RAM and not grow the caches. If it's close to 100, Linux will prefer to swap applications out, and enlarge the caches as much as possible. The default is a healthy 60. - Thanks for the link below, 3dInfluence.

Here's the environment:

1. Website that hosts a forum/journal/bboard/email/socialmedia application in walled garden (ie you pay to get to use it or are invited to do so

2. Many Clients pay to use the site during specific chunks of time (ie they lease access to site) in order to interact with their clients. There are dozens of clients in a broad range of fields.

3. There is a very broad service level agreement. Meaning that it's not stated that the website can't go down for more than ten minutes but there's a gentleman's agreement that it won't. They don't pay for the 24/7 support be we give it to them because we love what we do.

4. Site runs in 7 different languages throughout multiple time zones.

Here's the situation:

The site goes down at 5:30EST and stays "offline" for approximately two hours due to DDOS attack. The clients reactions vary from annoyed to livid. The clients are also not very tech savvy. The clients are accustomed to 24/7 support and typically receive great support.

Here's the question:

How much to you divulge to the client about the DDOS attack? They want a reason as to why the site went down.

I have a webserver that has 8GB of ram and is running a pretty intensive php site (1 site) that does file manipulation, graphing, emailing, forums, you name it. The environment is far from static which leads me to believe that very little could be gained from caching anything in ram since almost every request to the server creates new or updated pages. And a lot of caching is done client side so we have a ton of 304 requests when it comes to images, javascript, css.

Additionally I do have language files that are written to flat files on the server where cached ram definitely is good rather than reading from disk. But there are only a handful of files like this.

In about a two weeks I've gone from having 98% free ram to 4% free ram. This has occurred during a time in which we also push several large svn updates onto the server.

My question is whether my server will be better tuned if I periodically clear my cache (I'm aware of Linus Torvalds' feeling about cache) using the following command:

sync; echo 3 > /proc/sys/vm/drop_caches

Or would I be better off editing the following file:

/proc/sys/vm/swappiness  

If I replace the default value of 60 with 30 I should have much less swapping going on and a lot more reuse of stale cache.

It sure feels good to see all that cache freed up using the first command but I'd be lying to you if I told you this was good for the desktop environment. But what about a web server like I've described above? Thoughts?

EDIT: I'm aware that the system will acquire memory as it needs it from the cache memory but thanks for pointing that our for clarity. Am I imagining things when Apache slows down when most of the server memory is stored in cache? Is that a different issue altogether?