Robert's questions

Is there a way to use hostgroups within a check_cluster --service check?

I've only seen examples that list out each host in the cluster individually like this:

define service{
    ...

    check_command   check_service_cluster!"Foo Cluster"!0!1!$SERVICESTATEID:host1:DNS Service$,$SERVICESTATEID:host2:DNS Service$,$SERVICESTATEID:host3:DNS Service$
    ...

}

But this seems very unwieldy for large clusters. Is there a way to pass it a whole hostgroup directly, $SERVICESTATEID:dns_hostgroup?

I'm making a backup script for ldap. I want the errors to go to a file in /var/log and the output to go to another file in the backup folder. Currently I'm redirecting to a temp file and then sending the temp file to the log. I'd rather do this as a 1 liner though...

/usr/bin/ldapsearch -x -LLL -b "dc=contoso,dc=com" "(objectclass=*)" -h ldap.server -v 2>>/tmp/ldaptmp.err |
  gzip -c > /mnt/backups/ldap/`date +\%Y\%m\%d`.ldif.gz || 
  logger -t ldapbackup -p local6.err error exit $?

cat /tmp/ldaptmp.err | grep -v "ldap_initialize( ldap://ldap.server )" | 
  grep -v "filter: (objectclass=\*)" |
  grep -v "requesting: All userApplication attributes" >$ERR_LOG
rm -f /tmp/ldaptmp.err

Any ideas on how to redirect stderr and stdout to different pipes to condense this command into 1 line? Or is there a better way?

As title says, the ganglia page is defaulting to cluster report, which is quite ugly as it is attempting to mesh the 2 clusters together. Both ganglia/?c=kvmsrvGrid and ganglia/?c=unspecified work fine.

I'd like it to look like this: http://monitor.millennium.berkeley.edu/?m=&r=hour&s=descending&hc=4

Instead it looks more like this (with kvmsrvGrid): http://ganglia.millennium.berkeley.edu/infrastructure/?c=Management%20Servers

gmetad.conf:

data_source "unspecified" localhost:8649
data_source "kvmsrvGrid" kvmsrv5:8650 kvmsrv4:8650

gridname "unspecified"

edit: Ganglia version 3.1.2

What is recommended storage solution as VM count grows? We currently have 5 VM servers (8 core, 32GB) and one storage server. Currently we have ~80 virtual machines running and the storage server's disk IO is getting overwhelmed and VM performance is getting degraded.

We're thinking about moving to sheepdog for both redundancy and ability to scale. We don't need super fast virtual machines as most are used for development instances of production machines or intranet sites. Some info on sheepdog scaling and performance in this slide deck: http://www.osrg.net/sheepdog/_static/kvmforum2010.pdf

Other option is increase the number of disks in our storage server, add a physical raid device, and add a read/write cache. (although then i'd have to power down our entire virtual infrastructure to do a hardware upgrade this server...).

What is best practice to allow scaling up to say ~150 virtual machines?

Edit

After Chris's comment, I started pouring over the sheepdog mailing lists, and they also don't recommend for production. Shows how I need to go over fine print more...

Looks like I'll need to do more reading on storage solutions and make some purchases to upgrade ours.

Currently we have Dual Quad-core AMD Opterons 2 disk mirrored set for OS 6x2TB disk software raid for data (I think Raid 10)