Last modified 3 years ago Last modified on 05/21/11 01:54:57

How to set up multihoming on a RHEL5 (cluster2) cluster

Multihoming is not well tested on cluster2 but has been reported to work in small testing environments. It uses the multi-ring feature of openais. If you try it and encounter bugs please feel free to report them, but don't expect a priority response. It is NOT supported by Red Hat.

The first thing you will need, rather obviously, is two network interfaces (only two are allowed at the moment). These should be connected to different physical networks or VLANs and configured on different subnets.

The second IP addresses should then be added to cluster.conf as "altname"s to the existing nodes. cman likes to have names for these that map onto the IP addresses, but you can use the IP address instead if you prefer. Here's an example:

<clusternode name="node1" votes="1" nodeid="1"> 
    <altname name="node1a"/>
        <method name="single"> 
            <device name="apc" port="1"/> 

Note that you can optionally change the port and/or IP address used on the second interface:

<clusternode name="node1" votes="1" nodeid="1"> 
    <altname name="node1a" port="6899" mcast=""/>
        <method name="single"> 
            <device name="apc" port="1"/> 

It's vital to make sure that these are the same on all nodes.

If you are using the DLM in your cluster you will also need to tell it to use sctp as its transport rather than the default of tcp. To do this, add the following line to cluster.conf inside the <cluster> section:

<dlm protocol="sctp"/>

You will also need to increase the receive buffer space available to sctp by adding the following lines to a startup script. They don't have to be before the cman/dlm startup but should be before the first filesystem is mounted if you are using GFS.

echo 4194304 > /proc/sys/net/core/rmem_default 
echo 4194304 > /proc/sys/net/core/rmem_max     

Once the cluster is started, cman_tool status will show the extra configured interfaces for the local node:

Version: 6.1.0
Config Version: 5
Cluster Name: test-cluster
Cluster Id: 27310
Cluster Member: Yes
Cluster Generation: 1608
Membership state: Cluster-Member
Nodes: 8
Expected votes: 1
Total votes: 8
Quorum: 5  
Active subsystems: 7
Flags: Dirty 
Ports Bound: 0 11
Node name: node1
Node ID: 1
Multicast addresses: 
Node addresses: 

and cman_tool nodes -a will show all the addresses of all the nodes in the cluster.

Note that currently openais does not restart failed rings. so if a ring does go down you need to manually restart it (when it's fixed) using the command

openais-cfgtool -r