wiki:ClusterSSH
Last modified 3 years ago Last modified on 05/26/11 14:58:27

This page describes how to set up ssh so that it can fail over as part of a service without pesky "man in the middle" notifications or host key change warnings. (Note: This could be better done with a resource agent.)

Before you Begin

  • Gather information about your cluster service:
    • Service Name (we will use "MyService" as the example service name)
    • Service IP address(es) (we will use 172.31.31.1 as the example service IP address)
    • Device (e.g. eth0, eth1) which normally is used by the cluster for the given IP address (we will use eth0 as the example device)
  • Make a backup of /etc/ssh and /etc/init.d/sshd
  • Disable the clustered service to which you are adding SSH support
  • Grab a copy of my sshd script. This script makes use of the <script> resource's inheritance of the parent service name - that is, all scripts called from the <script> resource have access to the environment variable $OCF_RESKEY_service_name, thus allowing them to perform per-service operations.

Create a service-specific directory

  • Make a directory to hold your cluster-specific ssh configuration file and sshd script.
    mkdir -p /etc/cluster/ssh/MyService
    
  • Copy the global configuration into the the specific service directory{{{

cp /etc/ssh/sshd_config /etc/cluster/ssh/MyService/}}}

System-wide sshd configuration

  • Check to see if your configuration will work with a second ssh instance.
    [root@molly ~]# netstat -l | grep ssh
    tcp        0      0 *:ssh                       *:*                         LISTEN
    

If you see *:ssh, you need to edit your system-wide configuration.

  • Change ListenAddress to the IP address matching the hostname for the system, for example:
    ...
    Protocol 2
    #AddressFamily any
    ListenAddress 192.168.10.101  # was 0.0.0.0
    #ListenAddress ::
    ...
    
  • Restart sshd
    /sbin/service sshd restart
    
  • Verify that sshd is now binding to a specific IP/hostname
    [root@molly ~]# netstat -l | grep ssh
    tcp        0      0 molly:ssh                   *:*                         LISTEN
    
  • From another terminal, ensure you can still log in to the host via its hostname/IP without errors.

Per-service sshd instance configuration

  • Grab a copy of my sshd script, which is a drop-in replacement for /etc/init.d/sshd providing per-service configuration file loading/PID files.
  • Edit your private sshd config file (/etc/cluster/ssh/MyService/sshd_config) and change:
    • ListenAddress - IP address to listen on. This should be your service IP address:
      ListenAddress 172.31.31.1
      
    • HostKey - Key file locations. These should be something like:
      HostKey /etc/cluster/ssh/MyService/ssh_host_key
      HostKey /etc/cluster/ssh/MyService/ssh_host_rsa_key
      HostKey /etc/cluster/ssh/MyService/ssh_host_dsa_key
      
  • Manually add the cluster service IP address to the system:
    [root@molly ~]# ip addr add 172.31.31.1/24 dev eth0
    
  • Start the cluster-specific sshd instance:
    [root@molly ~]# OCF_RESKEY_service_name=MyService /etc/init.d/sshd start
    Generating SSH1 RSA host key:                              [  OK  ]
    Generating SSH2 RSA host key:                              [  OK  ]
    Generating SSH2 DSA host key:                              [  OK  ]
    Starting sshd (MyService):                                 [  OK  ]
    

(This should generate host keys for you the first time)

  • Netstat should now show two things listening on the ssh port:
    [root@molly ~]# netstat -l | grep ssh
    tcp        0      0 172.31.31.1:ssh             *:*                         LISTEN
    tcp        0      0 molly:ssh                   *:*                         LISTEN
    
  • Verify that you can ssh to the cluster IP address
  • If the above succeeded, stop the cluster-wide sshd instance
    [root@molly ~]# OCF_RESKEY_service_name=MyService /etc/init.d/sshd stop
    Stopping sshd (MyService):                                 [  OK  ]
    
  • Remove the IP address
    [root@molly ~]# ip addr del 172.31.31.1/24 dev eth0
    
  • Copy the /etc/cluster/ssh directory to all cluster nodes (using 'scp -rp', of course :) )

Configure the cluster

  • Add a resource for my modified sshd script to cluster.conf
    ...
    <resources>
            <script name="sshd" file="/etc/init.d/sshd" />
    </resources>
    ...
    <service name="MyService" ... >
            <ip address="172.31.31.1" />
            <script ref="sshd" />
    </service>
    ...