Pavel A's questions

I'm configuring ssh server, but have faced an issue.

First I generate the private/public rsa keys with:
ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key and set password on prompt.

Then try to start sshd:
/etc/init.d/ssh start
but get errors:

Could not load host key: /etc/ssh/ssh_host_rsa_key
Disabling protocol version 2. Could not load host key
sshd: no hostkeys available -- exiting.

Key point: Everything works if a leave password for the private key empty when generating keys, but the above errors occur otherwise. Still though, leaving that password empty is not recommended.

Any ideas? Thanks.

I'm trying to configure a 2 node cluster with shared storage, but the configuration is probably little unusual. It is described below:

1. Both nodes have access to a shared storage that has a number of drives. To cluster nodes they appear as local (SAN type of connection).
2. Node A is going to use half of those drives to assemble a s/w RAID (let's name it SW_RAID1). On top of this s/w RAID there going to be LVM (say VG-1).
   Same for Node B except for that Node B creates its own s/w RAID (SW-RAID2) using other hard drives than node A.
3. In case of a failure, the remaining node should take over all resources (both SW-RAID1 and 2 with VG-1 and VG-2 respectively).

The picture below illustrates the setup.

The question is: can I use plain LVM for this kind of setup or do I have to go with clustered LVM?

To my thinking I can use plain LVM because each volume group reseeds on an entity (/dev/md*) that is never going to be available on both nodes at a time (ensured by the cluster as it manages the above s/w RAIDs), so from LVM's perspective it is not even A/P.

Am I right? When answering, please, provide some reasoning if possible. Thanks!

I'm building up a 2-node cluster based on CMAN + Pacemaker cluster stack, but I have no hardware STONITH devices. However both nodes are connected to a shared storage via iSCSI and I would like to use this for fencing with SBD.

SBD is supported by openAIS and Heartbeat out of the box and I've already used SBD with Heartbeat + Pacemaker cluster stack, but now I need to get it working with CMAN. Therefore I have added little code to CMAN's init script to start/stop SBD and it seems to work okay.

Regarding the watchdog: it is highly recommended to run SBD with watchdog. SBD is a critical service in such a cluster and must be running all the time the cluster software is up. The watchdog helps to ensure that - if by some reason SBD process is unexpectedly terminated, the dog is not fed anymore and it reboots the node. So I get watchdog kernel module softdog to be loaded right before starting SBD (as it is in Heartbeat for example).

Briefly speaking I do the following in /etc/init.d/cman:

1. load the kernel module - modprobe softdog
2. start SBD - sbd -d <device> -D -W watch

And here is the problem:
when I start cman by hand

service cman start

everything is okay, but when cman starts automatically during boot, the node is rebooted.

It seems like the watchdog is not fed, because the node is fenced exactly <watchdog timeout> seconds (5s) after it (the watchdog) is initialized by SBD (14:21:29), however the logs (/var/log/syslog) are controversial, saying SBD is running:

Jan 15 14:21:28 cs-node1 kernel: [ 12.341755] softdog: Software Watchdog Timer: 0.08 initialized. soft_noboot=0 soft_margin=60 sec soft_panic=0 (nowayout=0)
...
Jan 15 14:21:29 cs-node1 sbd: [1200]: notice: Using watchdog device:/dev/watchdog
Jan 15 14:21:29 cs-node1 sbd: [1200]: info: Set watchdog timeout to 5 seconds.
...
Jan 15 14:21:30 cs-node1 sbd: [1202]: info: Latency: 1 on disk /dev/iscsi/disk2/part1
Jan 15 14:21:34 sbd: last message repeated 3 times
...
Jan 15 14:21:34 cs-node1 sbd: [1202]: info: Latency: 1 on disk /dev/iscsi/disk2/part1

Any ideas? Thanks!

p.s. Anyone with reputation over 300 points, please, consider creating the following tags: sbd or storage-based-death, stonith and cman.

Edit 1:

Now I have create a separate init script to manage SBD, that starts just before CMAN, but everything stays the same - it only works, when started manually after login. What is so special about the boot process that I don't know?

Edit 2:

Recently I have noticed that there can actually be no such messages in the logs:

Jan 15 14:21:30 cs-node1 sbd: [1202]: info: Latency: 1 on disk /dev/iscsi/disk2/part1
Jan 15 14:21:34 sbd: last message repeated 3 times
...
Jan 15 14:21:34 cs-node1 sbd: [1202]: info: Latency: 1 on disk /dev/iscsi/disk2/part1

and the node may be rebooted not exactly after 5 seconds after initializing the watchdog, but more often after 12, but every time exactly as the login prompt appears. Still though SBD is running even when there are no messages in syslog (I have added a background process that starts together with SBD and monitors its process).

I'm configuring a two node A/A cluster with a common storage attached via iSCSI, which uses GFS2 on top of clustered LVM. So far I have prepared a simple configuration, but am not sure which is the right way to configure gfs resource.

Here is the rm section of /etc/cluster/cluster.conf:

<rm>
    <failoverdomains>
        <failoverdomain name="node1" nofailback="0" ordered="0" restricted="1">
            <failoverdomainnode name="rhc-n1"/>
        </failoverdomain>
        <failoverdomain name="node2" nofailback="0" ordered="0" restricted="1">
            <failoverdomainnode name="rhc-n2"/>
        </failoverdomain>
    </failoverdomains>
    <resources>
        <script file="/etc/init.d/clvm" name="clvmd"/>
        <clusterfs name="gfs" fstype="gfs2" mountpoint="/mnt/gfs"  device="/dev/vg-cs/lv-gfs"/>
    </resources>
    <service name="shared-storage-inst1" autostart="0" domain="node1" exclusive="0" recovery="restart">
        <script ref="clvmd">
            <clusterfs ref="gfs"/>
        </script>
    </service>
    <service name="shared-storage-inst2" autostart="0" domain="node2" exclusive="0" recovery="restart">
        <script ref="clvmd">
            <clusterfs ref="gfs"/>
        </script>
    </service>
</rm>

This is what I mean: when using clusterfs resource agent to handle GFS partition, it is not unmounted by default (unless force_unmount option is given). This way when I issue

clusvcadm -s shared-storage-inst1

clvm is stopped, but GFS is not unmounted, so a node cannot alter LVM structure on shared storage anymore, but can still access data. And even though a node can do it quite safely (dlm is still running), this seems to be rather inappropriate to me, since clustat reports that the service on a particular node is stopped. Moreover if I later try to stop cman on that node, it will find a dlm locking, produced by GFS, and fail to stop.

I could have simply added force_unmount="1", but I would like to know what is the reason behind the default behavior. Why is it not unmounted? Most of the examples out there silently use force_unmount="0", some don't, but none of them give any clue on how the decision was made.

Apart from that I have found sample configurations, where people manage GFS partitions with gfs2 init script - https://alteeve.ca/w/2-Node_Red_Hat_KVM_Cluster_Tutorial#Defining_The_Resources

or even as simply as just enabling services such as clvm and gfs2 to start automatically at boot (http://pbraun.nethence.com/doc/filesystems/gfs2.html), like:

chkconfig gfs2 on

If I understand the latest approach correctly, such cluster only controls whether nodes are still alive and can fence errant ones, but such cluster has no control over the status of its resources.

I have some experience with Pacemaker and I'm used to that all resources are controlled by a cluster and an action can be taken when not only there are connectivity issues, but any of the resources misbehave.

So, which is the right way for me to go:

1. leave GFS partition mounted (any reasons to do so?)
2. set force_unmount="1". Won't this break anything? Why this is not the default?
3. use script resource <script file="/etc/init.d/gfs2" name="gfs"/> to manage GFS partition.
4. start it at boot and don't include in cluster.conf (any reasons to do so?)

This may be a sort of question that cannot be answered unambiguously, so it would be also of much value for me if you shared your experience or expressed your thoughts on the issue. How does for example /etc/cluster/cluster.conf look like when configuring gfs with Conga or ccs (they are not available to me since for now I have to use Ubuntu for the cluster)?

Thanks you very much!