当前位置: > Linux集群 > 负载均衡SLB >

LVS+Keepalived实现四层负载及高可用

时间:2016-05-29 03:13来源:linux.it.net.cn 作者:IT

最近要上线一个新项目,需要软负载,本来决定用nginx做反向代理,但装完nginx以后,发现nginx无法显示css、js,目前还在研究中,但由于项目较急,所以依然采用了经典架构LVS+Keepalived来实现。

环境:

LVS-01 192.168.199.117 MASTER

LVS-02 192.168.199.118 BACKUP

WEB-01 192.168.199.113

WEB-02 192.168.199.114

VIP 192.168.199.119

编译安装LVS

ln -s/usr/src/kernels/2.6.18-308.el5-x86_64/ /usr/src/linux

wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz

tar zxf ipvsadm-1.26.tar.gz

cd ipvsadm-1.26

make

 

发现报错,报错结果如下:

make[1]: *** [libipvs.o] Error1

make[1]: Leaving directory`/root/tools/ipvsadm-1.26/libipvs'

make: *** [libs] Error 2

 

经查询,有好多种说法,均没有解决我的问题:

1、  有的说是需要做内核软链接 ln-s /usr/src/kernels/2.6.18-308.el5-x86_64/ /usr/src/linux

2、  有的说是要yuminstall gcc-*

3、还有的说是yum installlibnl* popt*

以上方法可能适用于其实环境,但是在我的机器上面均无效。折腾了好久,请教高手,又是查看http://www.linuxvirtualserver.org/Documents.html官方说明文档,最终认为版本与内核不匹配,总结为以下2点:

1、  ipvsadm-1.26版本适合kernel 2.6.28-rc3及以后

2、  选择好相应的ipvsadm后,还要选择相对应的keepalived,否则会出现启动keepalived后,没有realserver列表。大致显示就是这样:

IP Virtual Server version 1.2.1(size=4096)

Prot LocalAddress:PortScheduler Flags

  -> RemoteAddress:Port          Forward Weight ActiveConn InActConn

 

好了,问题找到了,重新操作,下载低版本的ipvsadm,我的内核版本为2.6.18-308.el5-x86_64,用ipvsadm-1.24测试正常,keepalived的版本为keepalived-1.1.15,具体操作如下:

安装LVS

ln -s/usr/src/kernels/2.6.18-308.el5-x86_64/ /usr/src/linux

wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz

tar zxvf ipvsadm-1.24.tar.gz

cd ipvsadm-1.24

make

make install

cd ..

ipvsadm –version

 

安装keepalived

wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz

tar zxfkeepalived-1.1.15.tar.gz

cd keepalived-1.1.15

./configure

make

make install

cp/usr/local/etc/rc.d/init.d/keepalived /etc/init.d/

cp/usr/local/etc/sysconfig/keepalived /etc/sysconfig/

mkdir /etc/keepalived/

cp/usr/local/etc/keepalived/keepalived.conf /etc/keepalived/

cp /usr/local/sbin/keepalived/usr/sbin/

/etc/init.d/keepalived start

至此,安装完毕,下面配置文件。

 

LVS-01上面keepalived的配置文件,主与备基本一致,不同的为红色部分

vim /etc/keepalived/keepalived.conf

! Configuration File forkeepalived

global_defs {

  router_id LVS-01        #routeid

}

vrrp_instance LVS {

    state MASTER          #备上为BACKUP

    interface eth0

    virtual_router_id 251

    priority 100                #备节点上比主的数值要小

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.199.119

    }

}

virtual_server 192.168.199.11980 {

    delay_loop 6

    lb_algo wrr

    lb_kind DR

    persistence_timeout 60          #会话保持

    protocol TCP

 

    real_server 192.168.199.113 80 {

        weight 1         

        TCP_CHECK {         

        connect_timeout 10 

        nb_get_retry 3

        delay_before_retry 3

        connect_port 80

}

    }

    real_server 192.168.168.199.114 80 {

        weight 1

        TCP_CHECK {

        connect_timeout 10

        nb_get_retry 3

        delay_before_retry 3

        connect_port 80

        }

    }

}

 

启动keealived

/etc/init.d/keepalived restart

检查是否有VIP

[root@localhost ipvsadm-1.26]#ip add

1: lo:<LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue

    link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

      valid_lft forever preferred_lft forever

2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000

    link/ether 00:0c:29:70:47:f4 brdff:ff:ff:ff:ff:ff

    inet 192.168.199.117/24 brd 192.168.199.255scope global eth0

    inet 192.168.199.119/32 scope global eth0

    inet6 fe80::20c:29ff:fe70:47f4/64 scopelink

      valid_lft forever preferred_lft forever

3: sit0: <NOARP> mtu 1480qdisc noop

link/sit 0.0.0.0 brd 0.0.0.0

检查是否有realserver列表

[root@localhost ipvsadm-1.26]#ipvsadm -Ln

IP Virtual Server version 1.2.1(size=4096)

Prot LocalAddress:PortScheduler Flags

  -> RemoteAddress:Port          Forward Weight ActiveConn InActConn

TCP  192.168.199.119:80 wrr

  -> 192.168.199.113:80          Route  1    0          0       

  -> 192.168.199.114:80          Route  1    0          0

 

最后别忘了,有realserver上面配置VIP和ARP抑制,这个用脚本就好啦,具体如下:

[root@localhost tools]# cat realserver.sh

#!/bin/sh

[ -f"/etc/init.d/functions" ] && . /etc/init.d/functions

VIP=(

192.168.199.119

)

start(){

#bind VIP

for ((i=0;i<${#VIP[*]};i++))

do

  /sbin/ifconfig lo:$i ${VIP[$i]} netmask255.255.255.255 up  >/dev/null2>&1

done

#ARP restrain

echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignore

echo "2">/proc/sys/net/ipv4/conf/lo/arp_announce

echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore

echo "2">/proc/sys/net/ipv4/conf/all/arp_announce

}

 

stop(){

for ((i=0;i<${#VIP[*]};i++))

do

  /sbin/ifconfig lo:$i ${VIP[$i]} netmask255.255.255.255 down

done

echo "0">/proc/sys/net/ipv4/conf/lo/arp_ignore

echo "0">/proc/sys/net/ipv4/conf/lo/arp_announce

echo "0">/proc/sys/net/ipv4/conf/all/arp_ignore

echo "0">/proc/sys/net/ipv4/conf/all/arp_announce

}

 

case "$1" in

start)

start

if [ $? -eq 0 ];then

  action "ipvs start sucess"  /bin/true

else

  action "ipvs start failed "/bin/false

fi

;;

stop)

stop

if [ $? -eq 0 ];then

  action "ipvs stop sucess"  /bin/true

else

  action "ipvs stop failed "/bin/false

fi

;;

*)

echo "USAGE:$0{start|stop}"

esac

 

执行完成后,查看VIP

[root@localhost tools]# ip add

1: lo:<LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue

    link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet 192.168.199.119/32 brd 192.168.199.119scope global lo:0

    inet6 ::1/128 scope host

      valid_lft forever preferred_lft forever

2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000

    link/ether 00:0c:29:41:7c:7e brdff:ff:ff:ff:ff:ff

    inet 192.168.199.114/24 brd 192.168.199.255scope global eth0

    inet6 fe80::20c:29ff:fe41:7c7e/64 scopelink

      valid_lft forever preferred_lft forever

3: sit0: <NOARP> mtu 1480qdisc noop

    link/sit 0.0.0.0 brd 0.0.0.0

剩下的就是故障模拟与测试了,在此就不多写了。



(责任编辑:IT)
------分隔线----------------------------
栏目列表
推荐内容