LVS-DR模型实现负载均衡
时间:2017-02-02 00:35 来源:linux.it.net.cn 作者:IT
前言
前篇文章我们主要讲解了LVS-DR模型的架构方式以及如何实现,想了解的小伙伴点这个LVS-NAT模型实现负载均衡 ,今天我们来进行实践的LVS中三种模型中的DR模型的架构以及实现方式。(实验环境还以Web集群作为实验对象)
环境
此处我们LVS-DR模型环境架构也用四台Linux服务器,都有以下配置。
环境架构图:
工作原理:
基于DR机制实现。当用户请求到达director之后,此时Director知道client请求的为一个集群服务(Director有一块网卡,此块网卡配置了DIP【对内】,又配置了一个网卡别名VIP【对外公网访问】),那么director不会改变原有的IP头部,通过指定的调度算法,选择任意一台RealServer,通过DIP和RealServer的RIP在同一个物理网络内进行AIP广播请求,请求到此台RealServer的MAC之后,再将MAC地址封装为此台RealServer的MAC,然后发给此台RealServer,注意此时的源地址和目标地址没有改变,RealServer进行解包发现目标地址为VIP(RealServer有一块网卡:此块网卡配置了RIP【对内】,还在本地回环接口lookback口配置了一个隐藏网卡别名的VIP),自己有一个VIP,那么进行解封装,再将源地址为VIP目标地址为Client的CIP进行封装然后把数据发送给用户(此时不在经过Director,直接发给客户端),完成整个负载调度过程。
特点:
-
RS可以使用私有地址,还可以使用公网地址,此时可以直接通过互联网连入RS,以实现配置、监控等
-
RS的网关一定不能指向DIP
-
RS跟Dirctory要在同一物理网络内(不能有路由器分隔)
-
请求报文经过Directory,但响应报文一定不经过Director
-
不支持端口映射
-
RS可以使用大多数的操作系统
硬件环境:
使用软件VMware WorkStation 11,Director的DIP和RealServer的RS1、RS2为桥接模式,让其在同一个物理局域网内。(只作为测试环境)
-
一台Director:
-
版本:Red Hat Enterprise Linux Server release 6.4
-
单网卡:
-
eth1: DIP: 192.168.40.11/24
-
eth1:0 VIP: 172.16.21.10/16
这里我们为了模拟的更切合真实环境,所以Client请求的和RealServer的RIP不在同一网段,那么就需要一个Route设备,这里可选,你也可以使用全部都在一个物理网络内
-
一台Route:
-
版本:Red Hat Enterprise Linux Server release 6.4
-
双网卡:
-
eth1: 192.168.40.254/24
-
eth2: 172.16.0.254/16
-
两台RealServer:
-
版本:Red Hat Enterprise Linux Server release 6.4
-
单网卡:
-
RealServer1:eth1 :RIP1 192.168.40.12/24
lo:0 :172.16.21.10/32
-
RealServer2: eth1 :RIP2 192.168.40.13/24
lo:0 :172.16.21.10/32
软件环境:
搭建LVS集群只需要在Director服务器上安装ipvsadmin工具,此处我们使用RedHat自带的rpm包进行安装
版本:ipvsadm-1.25-10.el6.x86_64.rpm
安装前准备
同步时间
-
注意:时间同步是最重要的环节,Director和后台的RealServer的时间必须同步,并且时间差小于1秒钟。
-
本次我们使用Route服务器作为时间服务器,全部都向RouteNTPServer来同步时间。
[root@route ~]# service ntpd restart # 重启ntp服务器
Shutting down ntpd: [ OK ]
Starting ntpd: [ OK ]
[root@Director ~]# ntpdate 192.168.40.254 # 客户端同步三台都要同步这里我们就不多说了
查看内核版本
现在LVS已经是Linux标准内核的一部分,在Linux2.4内核以前,使用LVS时必须要重新编译内核以支持LVS功能模块,但是从Linux2.4内核以后,已经完全内置了LVS的各个功能模块,无需给内核打任何补丁,可以直接使用LVS提供的各种功能。
[root@Director Packages]# uname -r # 此处我们环境内核版本为2.6.32-358.el6.x86_64,所以不需要再打补丁,如果你的内核低于2.4那么则需要提前打补丁
2.6.32-358.el6.x86_64
查看内核是否支持ipvs
[root@Director ~]# modprobe -l | grep ipvs # 以下有之前所解释的十个内核所支持的算法(如果能有以下搜索到那么你的内核就支持ipvs)
kernel/net/netfilter/ipvs/ip_vs.ko
kernel/net/netfilter/ipvs/ip_vs_rr.ko
kernel/net/netfilter/ipvs/ip_vs_wrr.ko
kernel/net/netfilter/ipvs/ip_vs_lc.ko
kernel/net/netfilter/ipvs/ip_vs_wlc.ko
kernel/net/netfilter/ipvs/ip_vs_lblc.ko
kernel/net/netfilter/ipvs/ip_vs_lblcr.ko
kernel/net/netfilter/ipvs/ip_vs_dh.ko
kernel/net/netfilter/ipvs/ip_vs_sh.ko
kernel/net/netfilter/ipvs/ip_vs_sed.ko
kernel/net/netfilter/ipvs/ip_vs_nq.ko
kernel/net/netfilter/ipvs/ip_vs_ftp.ko
配置本地YUM源
我们使用本地光盘来作为YUM源
挂载本地光盘
[root@Director ~]# mount /dev/sr0 /media # 挂载本地光盘到本地目录
mount: block device /dev/sr0 is write-protected, mounting read-only
配置编辑YUM源的配置文件
三台服务器我们都需要配置,因为后面两台RealServer我们还需要安装web服务来作为集群服务。
[root@Director ~]# vim /etc/yum.repos.d/rhel-source.repo
[localhost] # 库名称(可随意)
name=localhost # 名称描述 (自定义)
baseurl=file:///media # yum源目录,源地址
enabled=1 # 是否用该yum源,0为禁用,1为使用
gpgcheck=0 # 检查GPG-KEY,0为不检查,1为检查
到此我们安装前准备就做好了,来下面进行下一步
配置RouteServer
配置两个网卡的IP地址
[root@route ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.40.254
NETMASK=255.255.255.0
[root@route ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth2
DEVICE=eth2
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=172.16.0.254
NETMASK=255.255.0.0
[root@route ~]# service network restart # 重启network服务
Shutting down interface eth1: [ OK ]
Shutting down interface eth2: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth1: [ OK ]
Bringing up interface eth2: [ OK ]
[root@route ~]# ifconfig # 查看是否配置生效
eth1 Link encap:Ethernet HWaddr 00:0C:29:22:E7:C7
inet addr:192.168.40.254 Bcast:192.168.40.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe22:e7c7/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:609 errors:0 dropped:0 overruns:0 frame:0
TX packets:292 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:57640 (56.2 KiB) TX bytes:22985 (22.4 KiB)
eth2 Link encap:Ethernet HWaddr 00:0C:29:22:E7:D1
inet addr:172.16.0.254 Bcast:172.16.255.255 Mask:255.255.0.0
inet6 addr: fe80::20c:29ff:fe22:e7d1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:5316 errors:0 dropped:0 overruns:0 frame:0
TX packets:4282 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:493923 (482.3 KiB) TX bytes:598669 (584.6 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:9965 errors:0 dropped:0 overruns:0 frame:0
TX packets:9965 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1049774 (1.0 MiB) TX bytes:1049774 (1.0 MiB)
开启本地路由转发
[root@route ~]# vim /etc/sysctl.conf
net.ipv4.ip_forward = 1 # 将0更改为1即可
[root@route ~]# sysctl -p # 刷新重新生效
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
配置DirecotrServer
安装ipvsadm工具
[root@Director Packages]# yum install -y ipvsadm-1.25-10.el6.x86_64.rpm
配置网卡,并设置IP
配置内网网卡DIP
[root@Director ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1 # 此处我们不指定GATEWAY(真实生产跟其环境架构而定)
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.40.11
NETMASK=255.255.255.0
配置外网网卡别名VIP
[root@Director ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth1 /etc/sysconfig/network-scripts/ifcfg-eth1:0 # 拷贝DIP的配置文件
[root@Director ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1:0 # 此内网网卡可不指定GATEWAY,因为和后台RealServer在同一个局域网(但也要根据真实环境而定)
DEVICE=eth1:0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=172.16.21.10
NETMASK=255.255.0.0
重启网卡服务
[root@Director ~]# service network restart
Shutting down interface eth1: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth1: [ OK ]
[root@Director ~]# ifconfig # 查看配置是否生效
eth1 Link encap:Ethernet HWaddr 00:0C:29:E5:9A:47
inet addr:192.168.40.11 Bcast:192.168.40.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fee5:9a47/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3289 errors:0 dropped:0 overruns:0 frame:0
TX packets:2421 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:305233 (298.0 KiB) TX bytes:324497 (316.8 KiB)
eth1:0 Link encap:Ethernet HWaddr 00:0C:29:E5:9A:47
inet addr:172.16.21.10 Bcast:172.16.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:17 errors:0 dropped:0 overruns:0 frame:0
TX packets:17 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1284 (1.2 KiB) TX bytes:1284 (1.2 KiB)
清空本地防火墙策略
[root@Director ~]# iptables -F
配置RealServer服务器
配置RealServer1网卡的RIP
[root@RS1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1 # 根据我们此时的环境一定要将网关指向RouteServer的192.168.40.254,因为ReaServer响应Client的时候不在一个网络不能直接路由
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.40.12
NETMASK=255.255.255.0
GATEWAY=192.168.40.254
NM_CONTROLLED=yes
配置内核参数
由于我们Director和两台RealServer都配置VIP地址,那么客户在请求VIP时,都有配置VIP,都会响应,这样的话网络就乱了,不知道包应该发给谁了,也就达不到我们所说的负载均衡的作用了,下面我们需要只要Client请求VIP时,只能Director来响应,那么我们有以下三个方案来解决这么问题。
让前段路由将请求发往VIP时,只能是Dirctor上的VIP响应
-
解决方案
-
静态地址绑定:有时候我们未必有ISP路由器的配置权限,Director调用时静态地址绑定将难以使用(Director做HA的时候那么一台坏了,还得重新绑定)
-
arptables:设定arptables规则来实现
-
修改linux内核参数:将RS上的VIP配置在lo接口的别名上,限制linux仅对对应接口的ARP请求做相应
第三种方法是我们最常用的,也是最实用的:
[root@RS1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@RS1 ~]# echo 1 > /proc/sys/net/ipv4/conf/eth1/arp_ignore
[root@RS1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@RS1 ~]# echo 2 > /proc/sys/net/ipv4/conf/eth1/arp_announce
配置RealServer1的隐藏的VIP
-
此配置步骤一定要在上一步配置完内核参数之后再进行配置,如果明白arp原理的小伙伴你懂的
[root@RS1 ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth1 /etc/sysconfig/network-scripts/ifcfg-lo:0
[root@RS1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-lo:0 # NETMASK要为三个255,自己在一个广播域里,此网络只有自己
DEVICE=lo:0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=172.16.21.10
NETMASK=255.255.255.255
GATEWAY=172.16.0.254
重启网卡配置
[root@RS1 ~]# service network restart
Shutting down interface eth1: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth1: [ OK ]
[root@RS1 ~]# ifconfig # 查看配置是否生效
eth1 Link encap:Ethernet HWaddr 00:0C:29:41:4A:CC
inet addr:192.168.40.12 Bcast:192.168.40.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe41:4acc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2749 errors:0 dropped:0 overruns:0 frame:0
TX packets:1871 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:265901 (259.6 KiB) TX bytes:241716 (236.0 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2320 errors:0 dropped:0 overruns:0 frame:0
TX packets:2320 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:293374 (286.4 KiB) TX bytes:293374 (286.4 KiB)
lo:0 Link encap:Local Loopback
inet addr:172.16.21.10 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:16436 Metric:1
添加路由条目
-
此路由条目必须要添加的,意思就是只要是访问本地VIP地址的,都从本地lo:0口进行转发给eth1(因为只有eth1才是对外出去的,lo口只是本地回环地址,不能直接和外界通信) 记得是转发给eth1,在从eth1进行传出去,这样eth1只是转发,不会作为源地址。
[root@RS1 ~]# route add -host 172.16.21.10 dev lo:0
[root@RS1 ~]# route -n # 查看添加路由条目
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.21.10 0.0.0.0 255.255.255.255 UH 0 0 0 lo
192.168.40.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth1
0.0.0.0 192.168.40.1 0.0.0.0 UG 0 0 0 eth1
安装Web服务
[root@RS1 ~]# yum install -y httpd
[root@RS1 ~]# service httpd start # 启动httpd服务
Starting httpd: httpd: apr_sockaddr_info_get() failed for RS1
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[ OK ]
[root@RS1 ~]# service httpd status # 查看httpd是否启动
httpd (pid 11249) is running...
[root@RS1 ~]# netstat -an | grep :80 # 查看web服务80端口是否监听
tcp 0 0 :::80 :::* LISTEN
[root@RS1 ~]# echo "RS1.xuxingzhuang.com" > /var/www/html/index.html # 给web服务提供网页界面
[root@RS1 ~]# curl http://localhost # 访问本地web是否可以正常访问
RS1.xuxingzhuang.com
清空本地防火墙策略
[root@RS1 ~]# iptables -F
配置RealServer2网卡的RIP
[root@RS2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.40.13
NETMASK=255.255.255.0
GATEWAY=192.168.40.254
配置内核参数
[root@RS2 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@RS2 ~]# echo 1 > /proc/sys/net/ipv4/conf/eth1/arp_ignore
[root@RS2 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@RS2 ~]# echo 2 > /proc/sys/net/ipv4/conf/eth1/arp_announce
配置RealServer1的隐藏的VIP
[root@RS2 ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth1 /etc/sysconfig/network-scripts/ifcfg-lo:0
[root@RS2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-lo:0
DEVICE=lo:0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=172.16.21.10
NETMASK=255.255.255.255
GATEWAY=172.16.0.254
重启网卡配置
[root@RS2 ~]# service network restart
Shutting down interface eth1: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth1: [ OK ]
[root@RS2 ~]# ifconfig # 查看配置是否生效
eth1 Link encap:Ethernet HWaddr 00:0C:29:9A:31:FB
inet addr:192.168.40.13 Bcast:192.168.40.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe9a:31fb/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3401 errors:0 dropped:0 overruns:0 frame:0
TX packets:3027 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:291858 (285.0 KiB) TX bytes:1029009 (1004.8 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2242 errors:0 dropped:0 overruns:0 frame:0
TX packets:2242 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:273214 (266.8 KiB) TX bytes:273214 (266.8 KiB)
lo:0 Link encap:Local Loopback
inet addr:172.16.21.10 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:16436 Metric:1
添加路由条目
[root@RS2 ~]# route add -host 172.16.21.10 dev lo:0
[root@RS2 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.21.10 0.0.0.0 255.255.255.255 UH 0 0 0 lo
192.168.40.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth1
0.0.0.0 192.168.40.254 0.0.0.0 UG 0 0 0 eth1
安装Web服务
[root@RS2 ~]# yum install -y httpd
[root@RS2 ~]# service httpd start # 启动httpd服务
Starting httpd: httpd: apr_sockaddr_info_get() failed for RS1
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[ OK ]
[root@RS2 ~]# service httpd status # 查看httpd是否启动
httpd (pid 10833) is running...
[root@RS2 ~]# netstat -an | grep :80 # 查看web服务80端口是否监听
tcp 0 0 :::80 :::* LISTEN
[root@RS2 ~]# echo "RS2.xuxingzhuang.com" > /var/www/html/index.html # 给web服务提供网页界面
[root@RS2 ~]# curl http://localhost # 访问本地web是否可以正常访问
RS2.xuxingzhuang.com
清空本地防火墙策略
[root@RS2 ~]# iptables -F
配置Driector,并将RealServer加入集群服务
ipvsadm命令的用法
管理集群服务
-
添加:-A -t|u|f service-address [-s scheduler]
-
-t: TCP协议的集群
-
-u: UDP协议的集群
-
service-address: IP:PORT
-
-f: FWM: 防火墙标记
-
service-address: Mark Number
-
修改:
-
-E
-
删除:
-
-D -t|u|f service-address
管理集群服务中的RealServer
-
添加:
-
-a -t|u|f service-address -r server-address [-g|i|m] [-w weight]
-
-t|u|f service-address:事先定义好的某集群服务
-
-r server-address: 某RS的地址,在DR模型中,可使用IP:PORT实现端口映射;
-
[-g|i|m]: LVS类型
-
-g: DR
-
-i: TUN
-
-m: DR
-
[-w weight]: 定义服务器权重
-
修改:
-
-e
-
删除:
-
d -t|u|f service-address -r server-address
集群服务后续管理
-
查看
-
-L|l
-
-n: 数字格式显示主机地址和端口
-
–stats:统计数据
-
–rate: 速率
-
–timeout: 显示tcp、tcpfin和udp的会话超时时长
-
-c: 显示当前的ipvs连接状况
-
例:ipvsadm -L -n –stats
-
删除所有集群服务
-
-C:清空ipvs规则
-
例:ipvsadm -C
-
保存规则
-
-S
-
例: ipvsadm -S > /etc/sysconfig/ipvsadm
-
载入此前的规则:
-
-R
-
例:ipvsadm -R < /etc/sysconfig/ipvsadm
将RealServer加入Web集群服务
-
我们此时使用内核的rr调度算法来作为实验,还不明白LVS的内核调度算法的小伙伴,请查看我们上一篇LVS集群服务详解
使用rr调度算法(轮叫)
[root@Director ~]# ipvsadm -A -t 172.16.21.10:80 -s rr
[root@Director ~]# ipvsadm -a -t 172.16.21.10:80 -r 192.168.40.12 -g -w 2 # 此处的-w指定权重是没有意义的,因为我们使用的为rr调度算法(轮叫),不过你也可以指定,后边我们改变算法时就不用重新定义了,省去了时间
[root@Director ~]# ipvsadm -a -t 172.16.21.10:80 -r 192.168.40.13 -g -w 1
[root@Director ~]# ipvsadm -L -n # 查看集群服务
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.21.10:80 rr
-> 192.168.40.12:80 Route 2 0 0
-> 192.168.40.13:80 Route 1 0 0
测试web集群服务rr算法
END
到底我们就基于LVS-DR模型就搭建好了一个web服务集群,DR模型在我们生活中用的还是比较多的,最主要的还是理解,后面我们还会继续写一下HA高可用集群。
(责任编辑:IT)
前言
环境
环境架构图:
工作原理:
特点:
硬件环境:使用软件VMware WorkStation 11,Director的DIP和RealServer的RS1、RS2为桥接模式,让其在同一个物理局域网内。(只作为测试环境)
这里我们为了模拟的更切合真实环境,所以Client请求的和RealServer的RIP不在同一网段,那么就需要一个Route设备,这里可选,你也可以使用全部都在一个物理网络内
软件环境:
安装前准备同步时间
[root@route ~]# service ntpd restart # 重启ntp服务器 Shutting down ntpd: [ OK ] Starting ntpd: [ OK ] [root@Director ~]# ntpdate 192.168.40.254 # 客户端同步三台都要同步这里我们就不多说了查看内核版本
[root@Director Packages]# uname -r # 此处我们环境内核版本为2.6.32-358.el6.x86_64,所以不需要再打补丁,如果你的内核低于2.4那么则需要提前打补丁 2.6.32-358.el6.x86_64 查看内核是否支持ipvs[root@Director ~]# modprobe -l | grep ipvs # 以下有之前所解释的十个内核所支持的算法(如果能有以下搜索到那么你的内核就支持ipvs) kernel/net/netfilter/ipvs/ip_vs.ko kernel/net/netfilter/ipvs/ip_vs_rr.ko kernel/net/netfilter/ipvs/ip_vs_wrr.ko kernel/net/netfilter/ipvs/ip_vs_lc.ko kernel/net/netfilter/ipvs/ip_vs_wlc.ko kernel/net/netfilter/ipvs/ip_vs_lblc.ko kernel/net/netfilter/ipvs/ip_vs_lblcr.ko kernel/net/netfilter/ipvs/ip_vs_dh.ko kernel/net/netfilter/ipvs/ip_vs_sh.ko kernel/net/netfilter/ipvs/ip_vs_sed.ko kernel/net/netfilter/ipvs/ip_vs_nq.ko kernel/net/netfilter/ipvs/ip_vs_ftp.ko 配置本地YUM源
挂载本地光盘[root@Director ~]# mount /dev/sr0 /media # 挂载本地光盘到本地目录 mount: block device /dev/sr0 is write-protected, mounting read-only 配置编辑YUM源的配置文件
[root@Director ~]# vim /etc/yum.repos.d/rhel-source.repo [localhost] # 库名称(可随意) name=localhost # 名称描述 (自定义) baseurl=file:///media # yum源目录,源地址 enabled=1 # 是否用该yum源,0为禁用,1为使用 gpgcheck=0 # 检查GPG-KEY,0为不检查,1为检查 到此我们安装前准备就做好了,来下面进行下一步 配置RouteServer配置两个网卡的IP地址[root@route ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE=eth1 TYPE=Ethernet ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=static IPADDR=192.168.40.254 NETMASK=255.255.255.0 [root@route ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth2 DEVICE=eth2 TYPE=Ethernet ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=static IPADDR=172.16.0.254 NETMASK=255.255.0.0 [root@route ~]# service network restart # 重启network服务 Shutting down interface eth1: [ OK ] Shutting down interface eth2: [ OK ] Shutting down loopback interface: [ OK ] Bringing up loopback interface: [ OK ] Bringing up interface eth1: [ OK ] Bringing up interface eth2: [ OK ] [root@route ~]# ifconfig # 查看是否配置生效 eth1 Link encap:Ethernet HWaddr 00:0C:29:22:E7:C7 inet addr:192.168.40.254 Bcast:192.168.40.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe22:e7c7/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:609 errors:0 dropped:0 overruns:0 frame:0 TX packets:292 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:57640 (56.2 KiB) TX bytes:22985 (22.4 KiB) eth2 Link encap:Ethernet HWaddr 00:0C:29:22:E7:D1 inet addr:172.16.0.254 Bcast:172.16.255.255 Mask:255.255.0.0 inet6 addr: fe80::20c:29ff:fe22:e7d1/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5316 errors:0 dropped:0 overruns:0 frame:0 TX packets:4282 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:493923 (482.3 KiB) TX bytes:598669 (584.6 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:9965 errors:0 dropped:0 overruns:0 frame:0 TX packets:9965 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1049774 (1.0 MiB) TX bytes:1049774 (1.0 MiB) 开启本地路由转发[root@route ~]# vim /etc/sysctl.conf net.ipv4.ip_forward = 1 # 将0更改为1即可 [root@route ~]# sysctl -p # 刷新重新生效 net.ipv4.ip_forward = 1 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.ipv4.tcp_syncookies = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.shmmax = 68719476736 kernel.shmall = 4294967296 配置DirecotrServer安装ipvsadm工具[root@Director Packages]# yum install -y ipvsadm-1.25-10.el6.x86_64.rpm 配置网卡,并设置IP配置内网网卡DIP[root@Director ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1 # 此处我们不指定GATEWAY(真实生产跟其环境架构而定) DEVICE=eth1 TYPE=Ethernet ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=static IPADDR=192.168.40.11 NETMASK=255.255.255.0 配置外网网卡别名VIP[root@Director ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth1 /etc/sysconfig/network-scripts/ifcfg-eth1:0 # 拷贝DIP的配置文件 [root@Director ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1:0 # 此内网网卡可不指定GATEWAY,因为和后台RealServer在同一个局域网(但也要根据真实环境而定) DEVICE=eth1:0 TYPE=Ethernet ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=static IPADDR=172.16.21.10 NETMASK=255.255.0.0 重启网卡服务[root@Director ~]# service network restart Shutting down interface eth1: [ OK ] Shutting down loopback interface: [ OK ] Bringing up loopback interface: [ OK ] Bringing up interface eth1: [ OK ] [root@Director ~]# ifconfig # 查看配置是否生效 eth1 Link encap:Ethernet HWaddr 00:0C:29:E5:9A:47 inet addr:192.168.40.11 Bcast:192.168.40.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fee5:9a47/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:3289 errors:0 dropped:0 overruns:0 frame:0 TX packets:2421 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:305233 (298.0 KiB) TX bytes:324497 (316.8 KiB) eth1:0 Link encap:Ethernet HWaddr 00:0C:29:E5:9A:47 inet addr:172.16.21.10 Bcast:172.16.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:17 errors:0 dropped:0 overruns:0 frame:0 TX packets:17 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1284 (1.2 KiB) TX bytes:1284 (1.2 KiB) 清空本地防火墙策略[root@Director ~]# iptables -F 配置RealServer服务器配置RealServer1网卡的RIP[root@RS1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1 # 根据我们此时的环境一定要将网关指向RouteServer的192.168.40.254,因为ReaServer响应Client的时候不在一个网络不能直接路由 DEVICE=eth1 TYPE=Ethernet ONBOOT=yes BOOTPROTO=static IPADDR=192.168.40.12 NETMASK=255.255.255.0 GATEWAY=192.168.40.254 NM_CONTROLLED=yes 配置内核参数
让前段路由将请求发往VIP时,只能是Dirctor上的VIP响应
第三种方法是我们最常用的,也是最实用的: [root@RS1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore [root@RS1 ~]# echo 1 > /proc/sys/net/ipv4/conf/eth1/arp_ignore [root@RS1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce [root@RS1 ~]# echo 2 > /proc/sys/net/ipv4/conf/eth1/arp_announce 配置RealServer1的隐藏的VIP
[root@RS1 ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth1 /etc/sysconfig/network-scripts/ifcfg-lo:0 [root@RS1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-lo:0 # NETMASK要为三个255,自己在一个广播域里,此网络只有自己 DEVICE=lo:0 TYPE=Ethernet ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=static IPADDR=172.16.21.10 NETMASK=255.255.255.255 GATEWAY=172.16.0.254 重启网卡配置[root@RS1 ~]# service network restart Shutting down interface eth1: [ OK ] Shutting down loopback interface: [ OK ] Bringing up loopback interface: [ OK ] Bringing up interface eth1: [ OK ] [root@RS1 ~]# ifconfig # 查看配置是否生效 eth1 Link encap:Ethernet HWaddr 00:0C:29:41:4A:CC inet addr:192.168.40.12 Bcast:192.168.40.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe41:4acc/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2749 errors:0 dropped:0 overruns:0 frame:0 TX packets:1871 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:265901 (259.6 KiB) TX bytes:241716 (236.0 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:2320 errors:0 dropped:0 overruns:0 frame:0 TX packets:2320 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:293374 (286.4 KiB) TX bytes:293374 (286.4 KiB) lo:0 Link encap:Local Loopback inet addr:172.16.21.10 Mask:255.255.255.255 UP LOOPBACK RUNNING MTU:16436 Metric:1 添加路由条目
[root@RS1 ~]# route add -host 172.16.21.10 dev lo:0 [root@RS1 ~]# route -n # 查看添加路由条目 Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 172.16.21.10 0.0.0.0 255.255.255.255 UH 0 0 0 lo 192.168.40.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth1 0.0.0.0 192.168.40.1 0.0.0.0 UG 0 0 0 eth1 安装Web服务[root@RS1 ~]# yum install -y httpd [root@RS1 ~]# service httpd start # 启动httpd服务 Starting httpd: httpd: apr_sockaddr_info_get() failed for RS1 httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName [ OK ] [root@RS1 ~]# service httpd status # 查看httpd是否启动 httpd (pid 11249) is running... [root@RS1 ~]# netstat -an | grep :80 # 查看web服务80端口是否监听 tcp 0 0 :::80 :::* LISTEN [root@RS1 ~]# echo "RS1.xuxingzhuang.com" > /var/www/html/index.html # 给web服务提供网页界面 [root@RS1 ~]# curl http://localhost # 访问本地web是否可以正常访问 RS1.xuxingzhuang.com 清空本地防火墙策略[root@RS1 ~]# iptables -F 配置RealServer2网卡的RIP[root@RS2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE=eth1 TYPE=Ethernet ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=static IPADDR=192.168.40.13 NETMASK=255.255.255.0 GATEWAY=192.168.40.254 配置内核参数[root@RS2 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore [root@RS2 ~]# echo 1 > /proc/sys/net/ipv4/conf/eth1/arp_ignore [root@RS2 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce [root@RS2 ~]# echo 2 > /proc/sys/net/ipv4/conf/eth1/arp_announce 配置RealServer1的隐藏的VIP[root@RS2 ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth1 /etc/sysconfig/network-scripts/ifcfg-lo:0 [root@RS2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-lo:0 DEVICE=lo:0 TYPE=Ethernet ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=static IPADDR=172.16.21.10 NETMASK=255.255.255.255 GATEWAY=172.16.0.254 重启网卡配置[root@RS2 ~]# service network restart Shutting down interface eth1: [ OK ] Shutting down loopback interface: [ OK ] Bringing up loopback interface: [ OK ] Bringing up interface eth1: [ OK ] [root@RS2 ~]# ifconfig # 查看配置是否生效 eth1 Link encap:Ethernet HWaddr 00:0C:29:9A:31:FB inet addr:192.168.40.13 Bcast:192.168.40.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe9a:31fb/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:3401 errors:0 dropped:0 overruns:0 frame:0 TX packets:3027 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:291858 (285.0 KiB) TX bytes:1029009 (1004.8 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:2242 errors:0 dropped:0 overruns:0 frame:0 TX packets:2242 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:273214 (266.8 KiB) TX bytes:273214 (266.8 KiB) lo:0 Link encap:Local Loopback inet addr:172.16.21.10 Mask:255.255.255.255 UP LOOPBACK RUNNING MTU:16436 Metric:1 添加路由条目[root@RS2 ~]# route add -host 172.16.21.10 dev lo:0 [root@RS2 ~]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 172.16.21.10 0.0.0.0 255.255.255.255 UH 0 0 0 lo 192.168.40.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1 169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth1 0.0.0.0 192.168.40.254 0.0.0.0 UG 0 0 0 eth1 安装Web服务[root@RS2 ~]# yum install -y httpd [root@RS2 ~]# service httpd start # 启动httpd服务 Starting httpd: httpd: apr_sockaddr_info_get() failed for RS1 httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName [ OK ] [root@RS2 ~]# service httpd status # 查看httpd是否启动 httpd (pid 10833) is running... [root@RS2 ~]# netstat -an | grep :80 # 查看web服务80端口是否监听 tcp 0 0 :::80 :::* LISTEN [root@RS2 ~]# echo "RS2.xuxingzhuang.com" > /var/www/html/index.html # 给web服务提供网页界面 [root@RS2 ~]# curl http://localhost # 访问本地web是否可以正常访问 RS2.xuxingzhuang.com 清空本地防火墙策略[root@RS2 ~]# iptables -F 配置Driector,并将RealServer加入集群服务ipvsadm命令的用法管理集群服务
管理集群服务中的RealServer
集群服务后续管理
将RealServer加入Web集群服务
使用rr调度算法(轮叫)[root@Director ~]# ipvsadm -A -t 172.16.21.10:80 -s rr [root@Director ~]# ipvsadm -a -t 172.16.21.10:80 -r 192.168.40.12 -g -w 2 # 此处的-w指定权重是没有意义的,因为我们使用的为rr调度算法(轮叫),不过你也可以指定,后边我们改变算法时就不用重新定义了,省去了时间 [root@Director ~]# ipvsadm -a -t 172.16.21.10:80 -r 192.168.40.13 -g -w 1 [root@Director ~]# ipvsadm -L -n # 查看集群服务 IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 172.16.21.10:80 rr -> 192.168.40.12:80 Route 2 0 0 -> 192.168.40.13:80 Route 1 0 0 测试web集群服务rr算法
END
(责任编辑:IT) |