![[Monkey]](../pics/monkey_wht.jpg)
![[Network Diagram]](pics/lb-eg.jpg)
This topology provides a highly available service with minimal hardware requirements. For a more detailed explanation of the function of each component please see the overview.
The documentation that follows assumes that all nodes on the network are set up with correct interfaces and routes for each network they are connected to as per the diagram above. The return path for packets must be through the Linux Director. In most cases this will mean that the the default route should be set to the Linux Director.
The Linux Director must be able to route traffic from the external network to the server network and vice versa. Specifically, in addition to correctly configuring the interfaces and routes IPV4 forwarding must be enabled. This is done by modifying the line containing net.ipv4.ip_forward in /etc/sysctl.conf. An example /etc/sysctl.conf follows:
# Enables packet forwarding net.ipv4.ip_forward = 1 # Enables source route verification net.ipv4.conf.default.rp_filter = 1For these changes to take effect the sysctl command may be used:
/sbin/sysctl -p net.ipv4.ip_forward = 1 net.ipv4.conf.default.rp_filter = 1
The monitoring of real servers, and their insertion and removal from the pool of servers available is controlled by ldirectord. To configure ldirectord /etc/ha.d/ldirectord.cf must be installed. Information on customising this file can be found in the ldirectord(8) man page.
On supported versions of Redh Hat and Fedora to ensure that ldirectord starts up (on run-levels 2, 3, 4 and 5) and that heartbeat does not start on reboot up the chkconfig command is used:
/sbin/chkconfig --level 2345 ldirectord on /sbin/chkconfig --del heartbeat
On Debian the update-rc.d command is used:
/usr/sbin/update-rc.d ldirectord start 2 3 4 5 . /usr/sbin/update-rc.d -f heartbeat remove
To ensure that heartbeat is not running and start ldirectord with the new configuration run:
/etc/init.d/heartbeat stop /etc/init.d/ldirectord start
Ldirectord logs debugging and status information to /var/log/messages using syslog. These logs should be inspected if problems occur. Please see notes on logging to ensure that all logs are written to disk for debugging purposes.
The current Linux Virtual Server kernel table may be displayed using the ipvsadm command. A sample invocation follows showing that ldirectord found all servers to be available:
ipvsadm -L -n IP Virtual Server version 1.0.4 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.3.61:21 rr persistent 600 -> 192.168.6.4:21 Masq 1 0 0 -> 192.168.6.5:21 Masq 1 0 0 TCP 10.1.3.61:80 rr -> 192.168.6.4:80 Masq 1 0 0 -> 192.168.6.5:80 Masq 1 0 0 TCP 10.1.3.61:443 rr -> 192.168.6.4:443 Masq 1 0 0 -> 192.168.6.5:443 Masq 1 0 0
As masquerading is being used as the forwarding mechanism by LVS, as per the ldirectord.cf, the linux director must masquerade for the real servers.
On supported versions of Red Hat and Fedora, masquerading can be configured using the iptables init script.
# Flush existing rules in the nat table /etc/init.d/iptables stop Resetting built-in chains to the default ACCEPT policy: [ OK ] # Masquerade for 192.168.6.0/24 bound for any host /sbin/iptables -t nat -A POSTROUTING -j MASQUERADE -s 192.168.6.0/24 # Log all packets that attempt to be forwarded # Useful for Debugging. Questionable for Production #/sbin/iptables -t nat -A POSTROUTING -j LOG # Save the rules /etc/init.d/iptables save Saving current rules to /etc/sysconfig/iptables: [ OK ] # Make sure rules are activated on reboot (at run levels 2, 3, 4 and 5) /sbin/chkconfig --level 2345 iptables on # Activate the rules /etc/init.d/iptables start Flushing all current rules and user defined chains: [ OK ] Clearing all current rules and user defined chains: [ OK ] Applying iptables firewall rules: [ OK ]
On Debian masqurading can be configured as part of the interface configuration for the interface that is on Real Servers' network. This example entry in /etc/network/interfaces Sets up masquerading on eth1.
auto eth1 iface eth1 inet static address 192.168.6.4 netmask 255.255.255.0 up iptables -t nat -A POSTROUTING -j MASQUERADE -s 192.168.6.0/24 down iptables -t nat -D POSTROUTING -j MASQUERADE -s 192.168.6.0/24
For this change to take palce restart networking.
/etc/init.d/networking restart
To verify the masquerading rules the iptables command may be used. The following example has been truncated to show only the POSTROUTING chain.
/sbin/iptables -t nat -L -n Chain POSTROUTING (policy ACCEPT) target prot opt source destination MASQUERADE all -- 192.168.6.0/24 0.0.0.0/0
If an FTP Virtual Service is to be used then the ip_vs_ftp kernel module needs to be used. This may be done by running the following command.
/sbin/modprobe ip_vs_ftp
If this module is needed it is important to ensure that it is inserted into the kernel at boot time.
On supported versions of Red&nbs;Hat and Fedora appended the above modprobe command to /etc/rc.local.
ip_vs_ftp
The modules presence in the kernel may be checked using lsmod. The output of the following command has been truncated to only show the ip_vs_ftp module.
/sbin/lsmod Module Size Used by ip_vs_ftp 3232 0
The Real Servers should be configured to run the underlying services for their respective virtual services. For instance, an HTTP daemon, such as Apache should be configured on each Real Server if an HTTP virtual service has been set up. In addition the the request URLs as specified in /etc/ha.d/ldirectord.cf should be present and contain the receive string.
As connections are forwarded to the real servers using NAT it is important that the return path for these connections passes through the linux director. This is so that NAT process can be reversed, else the return packet received by the end user will be from the real server and not the linux director and thus the connection will be dropped. This is usually achieved, by having the internal IP address of the linux director, in this example 192.168.6.1, as the default gateway for the real servers.
Using the simple case of a single gateway with an address on the server network of 192.168.6.1 this can be done by editing the GATEWAY specified in the interface configuration for the interface connected to 192.168.6.0/24. In the case of eth0, this is, /etc/sysconfig/network-scripts/ifcfg-eth0. A sample of this file follows:
DEVICE=eth0 ONBOOT=yes BOOTPROTO=static IPADDR=192.168.6.4 NETMASK=255.255.255.0 GATEWAY=192.168.6.1
To effect this networking should be restarted:
/etc/init.d/network restart Shutting down interface eth0 [ OK ] Setting network parameters [ OK ] Bringing up interface lo [ OK ] Bringing up interface eth0 [ OK ]
auto eth0 iface eth0 inet static address 192.168.6.4 netmask 255.255.255.0 gateway 192.168.6.1
To effect this change networking should be restarted:
/etc/init.d/networking restart Reconfiguring network interfaces: done.
/sbin/ip route show 0/0 default via 192.168.6.1 dev eth0
Copyright © 2000-2005,
Horms
Last Updated: Tue May 17 17:37:24 2005 +0900
Debian is a registered trademark of Software in the Public Interest, Inc.
Red Hat, the Red Hat Shadowman logo and Fedora are
registered trademarks of Red Hat, Inc.
Red Hat may also be refered to as RedHat on this site.
Linux is a registered trademark of Linus Torvalds.
All other trademarks are the property of their respective owners.