bonding mode 4 测试
时间:2020-05-11 12:33 来源:linux.it.net.cn 作者:IT
mode 4 定义
IEEE 802.3ad Dynamic link aggregation. Creates
aggregation groups that share the same speed and
duplex settings. Utilizes all slaves in the active
aggregator according to the 802.3ad specification.
Slave selection for outgoing traffic is done according
to the transmit hash policy, which may be changed from
the default simple XOR policy via the xmit_hash_policy
option, documented below. Note that not all transmit
policies may be 802.3ad compliant, particularly in
regards to the packet mis-ordering requirements of
section 43.2.4 of the 802.3ad standard. Differing
peer implementations will have varying tolerances for
noncompliance.
Prerequisites:
1. Ethtool support in the base drivers for retrieving
the speed and duplex of each slave.
2. A switch that supports IEEE 802.3ad Dynamic link
aggregation.
Most switches will require some type of configuration
to enable 802.3ad mode.
配置方法
eth0
DEVICE=eth0
IPV6INIT=no
MTU=1500
NM_CONTROLLED=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
BOOTPROTO=none
eth1
DEVICE=eth1
IPV6INIT=no
MTU=1500
NM_CONTROLLED=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
BOOTPROTO=none
bond0 [注, rhel6, rhel7 中, bonding 模式需要在该配置文件中进行指定]
由于当前用于 docker container , 因此用到 openvswitch 作为容器之间的网络转换
DEVICE=bond0
BOOTPROTO=none
ONBOOT=yes
BONDING_MASTER=yes
BONDING_OPTS="miimon=100 mode=4 lacp_rate=fast xmit_hash_policy=layer2+3"
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDGE=vlanbr0
bond0.vlan
DEVICE=bond0.vlan
ONBOOT=yes
TYPE=Ethernet
BOOTPROTO=none
IPADDR=x.x.x.x
NETMASK=255.255.255.0
GATEWAY=x.x.x.x
VLAN=yes
/etc/modprobe.d/bond.conf [切记配置改文件, 否则会有问题]
alias bond0 bonding
测试
iperf server :
[容器] x.x.x.95
容器 (4b085ae77152) [宿主机位置, x.x.x.192 ]
iperf client:
[物理机]
x.x.x.182 x.x.x.183 x.x.x.184 x.x.x.185
测试结果:
当前网络通道有 eth0, eth1 (1000M)
多个主机并发访问, 可以同时利用 eth0, eth1 进行网络通讯
假设在多个主机并发访问容器时候, 可能会出现, 其中一个主机独享eth0, 而其他主机在共享 eth1,
mode 4 没有负载均衡功能, 由接收方进行 hash 确定数据由那个通道进行通讯
测试过程
启动 iperf server:
[root@xxxx ~]# docker exec 4b085ae77152 /usr/bin/iperf -s -p 2345
连接 iperf server:
iperf -c x.x.x.95 -p 2345
单主机连接
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.182 port 28680
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 28830
[ 5] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec
两次独立测试, 连接带宽最大 937Mbits/sec, 只使用到一个[随机]网卡流量
两台主机连接容器测试,
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.182 port 28928
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45644
[ 4] 0.0-10.0 sec 979 MBytes 818 Mbits/sec
[ 5] 0.0-10.0 sec 288 MBytes 241 Mbits/sec <- 共 1000M
[ 6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 29070
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45762
[ 6] 0.0-10.0 sec 935 MBytes 781 Mbits/sec
[ 4] 0.0-10.0 sec 367 MBytes 307 Mbits/sec <- 共 1000M
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.184 port 4054
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45854
[ 5] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec
[ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 共 2000M
[ 6] local 10.197.244.95 port 2345 connected with x.x.x.184 port 4524
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45914
[ 6] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec
[ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 共 2000M
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 46148
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 6406
[ 5] 0.0-10.0 sec 968 MBytes 808 Mbits/sec
[ 4] 0.0-10.0 sec 444 MBytes 372 Mbits/sec <- 共 1000M
[ 6] local 10.197.244.95 port 2345 connected with x.x.x.183 port 46210
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 6878
[ 6] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec
[ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 共 2000M
连接时候, iperf client 会随机选择通道, 因此, 双机并发连接时候, 流量为 1000 ~ 2000 Mibits/sec
三台主机同时连接
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.185 port 26094
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 24020
[ 6] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48464
[ 5] 0.0-10.0 sec 949 MBytes 793 Mbits/sec
[ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M
[ 6] 0.0-10.0 sec 384 MBytes 321 Mbits/sec
[ 7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 26648
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 24928
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48596
[ 7] 0.0-10.0 sec 941 MBytes 786 Mbits/sec
[ 4] 0.0-10.0 sec 246 MBytes 206 Mbits/sec
[ 5] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M
[ 6] local 10.197.244.95 port 2345 connected with x.x.x.185 port 27116
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 25706
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48692
[ 6] 0.0-10.0 sec 960 MBytes 802 Mbits/sec
[ 4] 0.0-10.0 sec 215 MBytes 180 Mbits/sec
[ 5] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M
[ 7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 27606
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 26558
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48810
[ 7] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M
[ 4] 0.0-10.0 sec 942 MBytes 786 Mbits/sec
[ 5] 0.0-10.0 sec 239 MBytes 200 Mbits/sec
[ 6] local 10.197.244.95 port 2345 connected with x.x.x.186 port 4406
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 28322
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.182 port 33646
[ 6] 0.0-10.0 sec 979 MBytes 819 Mbits/sec
[ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M
[ 5] 0.0-10.0 sec 328 MBytes 275 Mbits/sec
三台主机同时连接, 流量为 1000 ~ 2000 Mibits/sec
[ 明显地, 在共享同一网络通道的两个 CLIENT 的流量并不平均 ]
四台主机测试
[ 7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 29964
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 30564
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49332
[ 6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34068
[ 7] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M
[ 4] 0.0-10.1 sec 563 MBytes 469 Mbits/sec
[ 5] 0.0-10.0 sec 194 MBytes 163 Mbits/sec
[ 6] 0.0-10.0 sec 482 MBytes 404 Mbits/sec
[ 8] local 10.197.244.95 port 2345 connected with x.x.x.185 port 30360
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 31230
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49422
[ 6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34194
[ 8] 0.0-10.1 sec 709 MBytes 591 Mbits/sec
[ 4] 0.0-10.0 sec 206 MBytes 173 Mbits/sec
[ 5] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M
[ 6] 0.0-10.0 sec 377 MBytes 316 Mbits/sec
[ 7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 30682
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 31774
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49500
[ 6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34280
[ 7] 0.0-10.1 sec 568 MBytes 472 Mbits/sec
[ 4] 0.0-10.0 sec 157 MBytes 131 Mbits/sec
[ 5] 0.0-10.0 sec 518 MBytes 434 Mbits/sec
[ 6] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M
[ 8] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34408
[ 4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49794
[ 5] local 10.197.244.95 port 2345 connected with x.x.x.184 port 32418
[ 6] local 10.197.244.95 port 2345 connected with x.x.x.185 port 31080
[ 8] 0.0-10.1 sec 622 MBytes 518 Mbits/sec
[ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M
[ 5] 0.0-10.0 sec 173 MBytes 144 Mbits/sec
[ 6] 0.0-10.0 sec 491 MBytes 411 Mbits/sec
四台主机同时连接, 流量为 1000 ~ 2000 Mibits/sec
[ 明显地, 三台主机在同一通道, 而另外一个主机独享流量] 即, 没有负载均衡功能
(责任编辑：IT)
| mode 4 定义 IEEE 802.3ad Dynamic link aggregation. Creates aggregation groups that share the same speed and duplex settings. Utilizes all slaves in the active aggregator according to the 802.3ad specification. Slave selection for outgoing traffic is done according to the transmit hash policy, which may be changed from the default simple XOR policy via the xmit_hash_policy option, documented below. Note that not all transmit policies may be 802.3ad compliant, particularly in regards to the packet mis-ordering requirements of section 43.2.4 of the 802.3ad standard. Differing peer implementations will have varying tolerances for noncompliance. Prerequisites: 1. Ethtool support in the base drivers for retrieving the speed and duplex of each slave. 2. A switch that supports IEEE 802.3ad Dynamic link aggregation. Most switches will require some type of configuration to enable 802.3ad mode. 配置方法 eth0 DEVICE=eth0 IPV6INIT=no MTU=1500 NM_CONTROLLED=no ONBOOT=yes MASTER=bond0 SLAVE=yes USERCTL=no BOOTPROTO=none eth1 DEVICE=eth1 IPV6INIT=no MTU=1500 NM_CONTROLLED=no ONBOOT=yes MASTER=bond0 SLAVE=yes USERCTL=no BOOTPROTO=none bond0 [注, rhel6, rhel7 中, bonding 模式需要在该配置文件中进行指定] 由于当前用于 docker container , 因此用到 openvswitch 作为容器之间的网络转换 DEVICE=bond0 BOOTPROTO=none ONBOOT=yes BONDING_MASTER=yes BONDING_OPTS="miimon=100 mode=4 lacp_rate=fast xmit_hash_policy=layer2+3" TYPE=OVSPort DEVICETYPE=ovs OVS_BRIDGE=vlanbr0 bond0.vlan DEVICE=bond0.vlan ONBOOT=yes TYPE=Ethernet BOOTPROTO=none IPADDR=x.x.x.x NETMASK=255.255.255.0 GATEWAY=x.x.x.x VLAN=yes /etc/modprobe.d/bond.conf [切记配置改文件, 否则会有问题] alias bond0 bonding 测试 iperf server : [容器] x.x.x.95 容器 (4b085ae77152) [宿主机位置, x.x.x.192 ] iperf client: [物理机] x.x.x.182 x.x.x.183 x.x.x.184 x.x.x.185 测试结果: 当前网络通道有 eth0, eth1 (1000M) 多个主机并发访问, 可以同时利用 eth0, eth1 进行网络通讯 假设在多个主机并发访问容器时候, 可能会出现, 其中一个主机独享eth0, 而其他主机在共享 eth1, mode 4 没有负载均衡功能, 由接收方进行 hash 确定数据由那个通道进行通讯 测试过程 启动 iperf server: [root@xxxx ~]# docker exec 4b085ae77152 /usr/bin/iperf -s -p 2345 连接 iperf server: iperf -c x.x.x.95 -p 2345 单主机连接 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.182 port 28680 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec [ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 28830 [ 5] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec 两次独立测试, 连接带宽最大 937Mbits/sec, 只使用到一个[随机]网卡流量 两台主机连接容器测试, [ 4] local 10.197.244.95 port 2345 connected with x.x.x.182 port 28928 [ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45644 [ 4] 0.0-10.0 sec 979 MBytes 818 Mbits/sec [ 5] 0.0-10.0 sec 288 MBytes 241 Mbits/sec <- 共 1000M [ 6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 29070 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45762 [ 6] 0.0-10.0 sec 935 MBytes 781 Mbits/sec [ 4] 0.0-10.0 sec 367 MBytes 307 Mbits/sec <- 共 1000M [ 5] local 10.197.244.95 port 2345 connected with x.x.x.184 port 4054 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45854 [ 5] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec [ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 共 2000M [ 6] local 10.197.244.95 port 2345 connected with x.x.x.184 port 4524 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45914 [ 6] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec [ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 共 2000M [ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 46148 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 6406 [ 5] 0.0-10.0 sec 968 MBytes 808 Mbits/sec [ 4] 0.0-10.0 sec 444 MBytes 372 Mbits/sec <- 共 1000M [ 6] local 10.197.244.95 port 2345 connected with x.x.x.183 port 46210 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 6878 [ 6] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec [ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 共 2000M 连接时候, iperf client 会随机选择通道, 因此, 双机并发连接时候, 流量为 1000 ~ 2000 Mibits/sec 三台主机同时连接 [ 5] local 10.197.244.95 port 2345 connected with x.x.x.185 port 26094 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 24020 [ 6] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48464 [ 5] 0.0-10.0 sec 949 MBytes 793 Mbits/sec [ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M [ 6] 0.0-10.0 sec 384 MBytes 321 Mbits/sec [ 7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 26648 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 24928 [ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48596 [ 7] 0.0-10.0 sec 941 MBytes 786 Mbits/sec [ 4] 0.0-10.0 sec 246 MBytes 206 Mbits/sec [ 5] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M [ 6] local 10.197.244.95 port 2345 connected with x.x.x.185 port 27116 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 25706 [ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48692 [ 6] 0.0-10.0 sec 960 MBytes 802 Mbits/sec [ 4] 0.0-10.0 sec 215 MBytes 180 Mbits/sec [ 5] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M [ 7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 27606 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 26558 [ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48810 [ 7] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M [ 4] 0.0-10.0 sec 942 MBytes 786 Mbits/sec [ 5] 0.0-10.0 sec 239 MBytes 200 Mbits/sec [ 6] local 10.197.244.95 port 2345 connected with x.x.x.186 port 4406 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 28322 [ 5] local 10.197.244.95 port 2345 connected with x.x.x.182 port 33646 [ 6] 0.0-10.0 sec 979 MBytes 819 Mbits/sec [ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M [ 5] 0.0-10.0 sec 328 MBytes 275 Mbits/sec 三台主机同时连接, 流量为 1000 ~ 2000 Mibits/sec [ 明显地, 在共享同一网络通道的两个 CLIENT 的流量并不平均 ] 四台主机测试 [ 7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 29964 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 30564 [ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49332 [ 6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34068 [ 7] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M [ 4] 0.0-10.1 sec 563 MBytes 469 Mbits/sec [ 5] 0.0-10.0 sec 194 MBytes 163 Mbits/sec [ 6] 0.0-10.0 sec 482 MBytes 404 Mbits/sec [ 8] local 10.197.244.95 port 2345 connected with x.x.x.185 port 30360 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 31230 [ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49422 [ 6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34194 [ 8] 0.0-10.1 sec 709 MBytes 591 Mbits/sec [ 4] 0.0-10.0 sec 206 MBytes 173 Mbits/sec [ 5] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M [ 6] 0.0-10.0 sec 377 MBytes 316 Mbits/sec [ 7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 30682 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 31774 [ 5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49500 [ 6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34280 [ 7] 0.0-10.1 sec 568 MBytes 472 Mbits/sec [ 4] 0.0-10.0 sec 157 MBytes 131 Mbits/sec [ 5] 0.0-10.0 sec 518 MBytes 434 Mbits/sec [ 6] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M [ 8] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34408 [ 4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49794 [ 5] local 10.197.244.95 port 2345 connected with x.x.x.184 port 32418 [ 6] local 10.197.244.95 port 2345 connected with x.x.x.185 port 31080 [ 8] 0.0-10.1 sec 622 MBytes 518 Mbits/sec [ 4] 0.0-10.0 sec 1.09 GBytes 937 Mbits/sec <- 独享 1000M [ 5] 0.0-10.0 sec 173 MBytes 144 Mbits/sec [ 6] 0.0-10.0 sec 491 MBytes 411 Mbits/sec 四台主机同时连接, 流量为 1000 ~ 2000 Mibits/sec [ 明显地, 三台主机在同一通道, 而另外一个主机独享流量] 即, 没有负载均衡功能 (责任编辑：IT) |