Linux bridge: MAC addresses and dynamic ports

Scenario: KVM virtualization host running several bridged guests. The host has a bridge interface br0 that starts out containing only eth0, and other interfaces are dynamically added and removed from the bridge as guests are started and stopped.
The problem is, the host seems to randomly suffer some loss of connectivity (from a few to 30-40 seconds) when some guest is started or stopped. Initially one might think of something related to STP, but it turns out that it is disabled (and even then, ports appearing or disappearing should not affect existing ports).

What happens here is that, when a new guest is started, a tap interface is created and enslaved to the bridge (the tap interface is usually connected to the guest’s own ethernet interface). This interface has a name like vnet5, for example. Minor but important detail, this tap interface has a more or less random MAC address. (Note that we’re talking of the host-side tap interface, not the guest-side virtual interface, whose MAC address is configurable in the guest’s configuration file or on the command line). It is theoretically possible to set the MAC address of this host-side tap interface, but it’s quite involved and basically nobody does that (and also virt-manager has no provision to do that).

Now, by default bridge interfaces in Linux use, for their MAC address, the lowest MAC address among the enslaved interfaces. So if the newly created interface has a lower MAC, the bridge changes its MAC address and uses that of the new interface. When the guest is stopped, the tap interface is removed from the bridge and destroyed, at which point the bridge’s MAC address has to change again. Depending on when and how often guests are started/stopped, the host’s bridge interface may be changing its MAC address quite often.

Of course, this may (and does) cause connectivity problems for traffic to/from the host (fortunately, traffic to/from the guests is not affected), because neighboring computers and network devices suddenly have to update their ARP caches with the new MAC address before they can talk to the host. Depending on the traffic patterns, this may take anything from a few seconds to several tens of seconds.

Fortunately, there is a way to ensure that the bridge’s MAC address is fixed and never changes, thus entirely avoiding the problem. Thanks to this thread and this thread by people with similar problems, I found out that if the bridge’s MAC address is forced to a specific value, the bridge “remembers” that and makes the address permanent. But there’s a caveat: the address must belong to one of the devices enslaved to the bridge. In our example (and probably also in the most common case), the ideal candidate is obviously eth0, which is permanently enslaved.

There are two ways to set the MAC address of an interface. The old way is using ifconfig:

# 00:0a:e7:2c:44:2a is eth0's MAC address
# ifconfig br0 hw ether 00:0a:e7:2c:44:2a

The newer and better way is with iproute2:

# ip link set br0 address 00:0a:e7:2c:44:2a

And also, it is desirable to set the permanent MAC address at boot time
file /etc/conf.d/net:

depend_br0() {
    need.enp1s0
}
 
config_enp1s0="null"
 
bridge_br0="enp1s0"
config_br0="10.0.0.1/24"  # the ip of the original eth1, or dhcp
mac_br0="f4:f2:6d:04:a5:11"
 
brctl_br0="setfd 0
        sethello 30
        stp off"
 
vlan_start_br0="no"
vlans_br0="1 2 3"

Source http://backreference.org/2010/07/28/linux-bridge-mac-addresses-and-dynamic-ports/

if the bridge’s MAC address is forced to a specific value, the bridge «remembers» that and makes the address permanent.
маком бриджа берется самый низкий по значению мак из прицепленных интерфейсов. именно поэтому либвирт поднимает свои tap-ы с маками начинающимися на FE (сам баг открывал пару лет назад).

Source https://www.linux.org.ru/forum/admin/7855150?cid=7860897

Добавить комментарий