|
linux randomly routing ipv6 packets to wrong interface
![[de]](/s/countries/de.gif) Shadow Hawkins on Monday, 10 January 2011 03:30:19
hi,
I have a debian linux squeeze xen machine. On this machine I have a static sixxs tunnel set up with aiccu and a manualy set up bridge with some xen vm's. On the bridge I have routes to a /64 part of my sixxs subnet. When I try to send ipv6 packets from any of my vm's to the internet, 70% of this packets do no reach the internet. This huge packet loss makes using ipv6 impossible.
To investigate this I used the following ip6tables rule: ip6tables -A FORWARD -t mangle -j LOG --log-level 0
Packets which do not reach the Internet look like this:
kernel:[125780.583485] IN=routebr OUT=routebr SRC=2001:4dd0:fc3c:78dc:0216:3eff:fee6:7d01 DST=2001:0838:0001:0001:0210:dcff:fe20:7c7c LEN=104 TC=0 HOPLIMIT=64 FLOWLBL=0 PROTO=ICMPv6 TYPE=128 CODE=0 ID=5215 SEQ=55
Packets which reach the Internet look like this:
kernel:[125781.597121] IN=routebr OUT=sixxs SRC=2001:4dd0:fc3c:78dc:0216:3eff:fee6:7d01 DST=2001:0838:0001:0001:0210:dcff:fe20:7c7c LEN=104 TC=0 HOPLIMIT=63 FLOWLBL=0 PROTO=ICMPv6 TYPE=128 CODE=0 ID=5215 SEQ=56
To me it seems like the kernel is randomly picking the wrong interface to forward the packets to. Do you have any idea's what could cause this?
Here is how i setup the bridge and some output of routing informations. I have no ip6table rules.
brctl addbr routebr
ip link set up dev routebr
ip -6 addr add 2001:4dd0:fc3c:78dc::1/64 dev routebr
$ ip -6 route show
2001:4dd0:fc3c:78dc::/64 dev routebr metric 1024 mtu 1500 advmss 1440 hoplimit 4294967295
2001:4dd0:fc3c:f42d::/64 dev natbr proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
2001:4dd0:ff00:491::/64 via :: dev sixxs proto kernel metric 256 mtu 1280 advmss 1220 hoplimit 4294967295
fe80::/64 dev eth0 proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 dev tunnelbr proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 dev routebr proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 dev natbr proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 via :: dev sixxs proto kernel metric 256 mtu 1280 advmss 1220 hoplimit 4294967295
fe80::/64 dev vif1.0 proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 dev vif2.0 proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 dev vif3.0 proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
default via 2001:4dd0:ff00:491::1 dev sixxs metric 1024 mtu 1280 advmss 1220 hoplimit 4294967295
Thank you for your time
johannes
linux randomly routing ipv6 packets to wrong interface
Shadow Hawkins on Tuesday, 11 January 2011 18:07:16
Now I know the cause and a solution.
The cause:
When I do
brctl addbr routebr
ip link set up dev routebr
the bridge get's a random mac address and calculates the link local address using this random mac address.
When xen add's a tap device to the bridge, the mac address of the bridge changes, but the link local address remains the same. When a vm trys to send a packet to the router it calculates the routers mac address from it's link local address. This mac address is not valid anymore, so the packets are not picked up by the router. Sometimes the vm does a neighboor solicitation to multicast mac address and get's in return the correct router mac address. Then packets can flow until the timeout in the neigbour cache run's out and the vm sends to the wrong mac address again.
The solution:
brctl addbr routebr
mktapdev routebrtap
ip link set address ab:cd:ef:gh:ij:kl dev routebrtap
brctl addif routebr routebrtap
ip link set address ab:cd:ef:gh:ij:kl dev routebr
ip link set up dev routebr
To solve this, I give the bridge a fixed mac adress before calling `ip link set up`. Now the link local address always coresponds to the bridges mac address. Bridges can only take mac addresses from interfaces on the bridge. I create a tap device, give the tap device a fixed mac address, add the tap device to the bridge, and now I can give the bridge the exact same mac address. Only after doing this, I call `ip link set up`, so the link local address get's chosen based on the fixed mac address.
|
Posting is only allowed when you are