Forum - Heartbeat tunnel working but bad loss statistics and bad pings :: SixXS

Heartbeat tunnel working but bad loss statistics and bad pings

Shadow Hawkins on Friday, 05 December 2008 09:40:52

Hello all, I have a heartbeat tunnel T16455 up with PoP dedus01. After several routing issues which are hopefully solved I got the effect that the tunnel is actually working over the last days (at least it was working when I was testing/using it) but the loss statistics are bad. They show between 60 and 100% ping losses. Rarely all pings make it through to show a 0% loss - at least according to the statistics. I don't understand this because I don't see any blocked packets in my firewall. I also logged the ping packets with tcpdump and checked the connection during these checks. Everything fine. Sure I cannot say what pings get back to the sender through the tunnel. But the replies are sent out properly. AICCU is used with this tunnel, time is set via NTP properly. The second issue I got and I also gut this with tunnel T18281 (static IPv4) over the same PoP dedus01: Pings to the "outside" IPv6 world show that the connection is not very constant or permanent. I got good pings 5-10 times and then I got a response time from between 200 and 1000ms. It's noticable also when using PuTTY to the remote machine for example. I attached two ping logs below. Loss statistics here are just fine, nothing to complain about. I also attached routing table and some more information below. Maybe someone can help me out. Kindest regards, Martin User: MSV6-SIXXS Heartbeat tunnel: T16455 Static tunnel: T18281 PoP: dedus01 OS used: Debian with kernel 2.6.x Here the routing table plus ping log of the heartbeat tunnel. If you need more information I'll be happy to provide it.

Kernel IPv6 routing table Destination Next Hop Flags Metric Ref Use Iface 2a01:198:200:18e::/64 :: U 256 56523 0 sixxs 2a01:198:2b4::/64 :: U 256 0 0 eth0 2000::/3 2a01:198:200:18e::1 UG 1024 93007 0 sixxs fe80::/64 :: U 256 0 0 eth0 fe80::/64 :: U 256 0 0 eth1 fe80::/64 :: U 256 0 0 sixxs ::/0 2a01:198:200:18e::1 UG 1024 0 0 sixxs ::1/128 :: U 0 601 1 lo 2a01:198:200:18e::/128 :: U 0 0 1 lo 2a01:198:200:18e::2/128 :: U 0 17682 1 lo 2a01:198:2b4::/128 :: U 0 0 1 lo 2a01:198:2b4::1/128 :: U 0 6826 1 lo fe80::/128 :: U 0 0 1 lo fe80::/128 :: U 0 0 1 lo fe80::/128 :: U 0 0 1 lo fe80::4e2a:c830/128 :: U 0 0 1 lo fe80::ac1b:1/128 :: U 0 0 1 lo fe80::ac1b:201/128 :: U 0 0 1 lo fe80::ac1c:6/128 :: U 0 0 1 lo fe80::ac1c:106/128 :: U 0 0 1 lo fe80::ac1c:206/128 :: U 0 0 1 lo fe80::ac1c:306/128 :: U 0 0 1 lo fe80::ac1c:406/128 :: U 0 0 1 lo fe80::202:ffff:fe00:b55a/128 :: U 0 0 1 lo fe80::2d0:b7ff:fe9f:c59b/128 :: U 0 7712 1 lo ff00::/8 :: U 256 0 0 eth0 ff00::/8 :: U 256 0 0 eth1 ff00::/8 :: U 256 0 0 sixxs

ping6 ipv6.google.com PING ipv6.google.com(2001:4860:0:1001::68) 56 data bytes 64 bytes from 2001:4860:0:1001::68: icmp_seq=1 ttl=59 time=26.0 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=2 ttl=59 time=241 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=3 ttl=59 time=29.3 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=4 ttl=59 time=25.5 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=5 ttl=59 time=33.7 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=6 ttl=59 time=744 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=7 ttl=59 time=25.0 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=8 ttl=59 time=25.6 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=9 ttl=59 time=24.8 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=10 ttl=59 time=242 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=11 ttl=59 time=24.9 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=12 ttl=59 time=26.2 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=13 ttl=59 time=25.8 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=14 ttl=59 time=29.2 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=15 ttl=59 time=25.4 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=16 ttl=59 time=25.0 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=17 ttl=59 time=25.9 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=18 ttl=59 time=25.4 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=19 ttl=59 time=26.8 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=20 ttl=59 time=268 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=21 ttl=59 time=29.4 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=22 ttl=59 time=25.6 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=23 ttl=59 time=25.6 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=24 ttl=59 time=24.8 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=25 ttl=59 time=25.2 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=26 ttl=59 time=26.6 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=27 ttl=59 time=25.7 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=28 ttl=59 time=434 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=29 ttl=59 time=24.5 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=30 ttl=59 time=26.2 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=31 ttl=59 time=25.8 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=32 ttl=59 time=56.7 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=33 ttl=59 time=26.3 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=34 ttl=59 time=25.9 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=35 ttl=59 time=26.4 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=36 ttl=59 time=27.7 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=37 ttl=59 time=26.4 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=38 ttl=59 time=328 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=39 ttl=59 time=27.8 ms 64 bytes from 2001:4860:0:1001::68: icmp_seq=40 ttl=59 time=26.1 ms

ping6 2001:8d8:81:1540::1 PING 2001:8d8:81:1540::1(2001:8d8:81:1540::1) 56 data bytes 64 bytes from 2001:8d8:81:1540::1: icmp_seq=1 ttl=56 time=25.7 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=2 ttl=56 time=25.4 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=3 ttl=56 time=25.0 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=4 ttl=56 time=25.0 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=5 ttl=56 time=207 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=6 ttl=56 time=23.7 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=7 ttl=56 time=24.5 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=8 ttl=56 time=25.8 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=9 ttl=56 time=184 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=10 ttl=56 time=24.1 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=11 ttl=56 time=25.5 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=12 ttl=56 time=26.3 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=13 ttl=56 time=24.7 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=14 ttl=56 time=628 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=15 ttl=56 time=24.7 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=16 ttl=56 time=25.6 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=17 ttl=56 time=25.2 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=18 ttl=56 time=24.8 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=19 ttl=56 time=25.7 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=20 ttl=56 time=24.9 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=21 ttl=56 time=24.9 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=22 ttl=56 time=498 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=23 ttl=56 time=24.5 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=24 ttl=56 time=24.5 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=25 ttl=56 time=24.1 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=26 ttl=56 time=183 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=27 ttl=56 time=24.6 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=28 ttl=56 time=24.2 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=29 ttl=56 time=25.1 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=30 ttl=56 time=25.1 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=31 ttl=56 time=25.1 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=32 ttl=56 time=26.1 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=33 ttl=56 time=25.2 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=34 ttl=56 time=25.2 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=35 ttl=56 time=24.9 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=36 ttl=56 time=274 ms 64 bytes from 2001:8d8:81:1540::1: icmp_seq=37 ttl=56 time=25.3 ms

Heartbeat tunnel working but bad loss statistics and bad pings

Jeroen Massar SixXS Staff

on Friday, 05 December 2008 09:59:41

When pinging remote hosts, try to remember that the are a lot of intermediate hops, thus those latency increases can happen on any of them. Next to that, a lot of these intermediate hops are doing ICMP rate limitting, and guess what, due to a lot of people doing these ICMP tests all the time (especially those nice people who need to do smokepings and other such things) those rate limits get hit a lot and your ICMP packet won't be returned. Lastly, check Connection tracking FAQ which seems to bite a lot of people too.

Heartbeat tunnel working but bad loss statistics and bad pings

Shadow Hawkins on Friday, 05 December 2008 12:56:08

Yeah, pinging and then believing the connection is not stable would not be useful. But when using PuTTY for example I see the same lags as I see in the pings. It takes half a second or even a second until something happens from time to time (every 5-10 seconds). And when pinging ipv6.google.com and seeing the same I believe that the PoP might have a problem. I just pinged ipv6and4.labs.wikimedia.org and saw the same effect. Is anybody else who uses the same PoP having similar issues?

Lastly, check Connection tracking FAQ which seems to bite a lot of people too.

Ah thank you very much for this tip. I just inserted a rule into my firewall. Hopefully this works and helps with the problem. I'll be back with more in some hours. ;-) Regards, Martin

Heartbeat tunnel working but bad loss statistics and bad pings

Jeroen Massar SixXS Staff

on Friday, 05 December 2008 14:24:48

I just pinged ipv6and4.labs.wikimedia.org and saw the same effect.

And which hop was causing it? Was it on your end, on the PoP, was it on an IPv4(!) hop between them (oh, yes, the IPv4 infra your tunnel runs over can also affect it, and some ISP are so 'nice' to "QoS" 'unknown' traffic and then just delay or drop it), or was it on an IPv6 router somewhere between you and your destination? See how many possibilities there are? Just pinging, especially because of rate limits really thus is quite futile. Wireshark dumps might bring you somewhere, but they will only show that packets might be slower, not where it happens, and that is really difficult to determine.

Heartbeat tunnel working but bad loss statistics and bad pings

Shadow Hawkins on Friday, 05 December 2008 15:46:27

Well, what I can do without problems is ping6ing any destination host directly from my router and trying IPv6 SSH to three of my servers in the internet from any client inside my network. And I can ping from these servers back on my router, which I did and showed the same effect. I use IPv6 tunnels of the server providers, not SIXXS tunnels. Running a traceroute to the hosts makes not much sense because of the time the traceroute runs. What I just did is looking at traceroute6 and noticed that the first hop after the tunnel exit ist 2a01:198:4:6::1 for every destination. I pinged this host and see the same effect.

PING 2a01:198:4:6::1(2a01:198:4:6::1) 56 data bytes 64 bytes from 2a01:198:4:6::1: icmp_seq=1 ttl=63 time=20.0 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=2 ttl=63 time=20.1 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=3 ttl=63 time=15.5 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=4 ttl=63 time=19.8 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=5 ttl=63 time=91.7 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=6 ttl=63 time=16.2 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=7 ttl=63 time=16.7 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=8 ttl=63 time=15.4 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=9 ttl=63 time=17.2 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=10 ttl=63 time=17.2 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=11 ttl=63 time=264 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=12 ttl=63 time=15.1 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=13 ttl=63 time=14.7 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=14 ttl=63 time=15.2 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=15 ttl=63 time=152 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=16 ttl=63 time=15.3 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=17 ttl=63 time=17.0 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=18 ttl=63 time=17.0 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=19 ttl=63 time=15.3 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=20 ttl=63 time=19.3 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=21 ttl=63 time=16.3 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=22 ttl=63 time=19.3 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=23 ttl=63 time=15.5 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=24 ttl=63 time=301 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=25 ttl=63 time=22.4 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=26 ttl=63 time=15.5 ms 64 bytes from 2a01:198:4:6::1: icmp_seq=27 ttl=63 time=19.9 ms

Yes, I again just pinged the host. But as I said: I also have these lags when using SSH on my IPv6 enabled servers. So it is not a problem with ping limitations or something like this at the destination hosts. Regards, Martin

Heartbeat tunnel working but bad loss statistics and bad pings

Shadow Hawkins on Sunday, 07 December 2008 14:44:12

Hello!

Lastly, check Connection tracking FAQ which seems to bite a lot of people too.

Ah thank you very much for this tip. I just inserted a rule into my firewall.

Hopefully this works and helps with the problem. I'll be back with more in some

hours. ;-)

Ok, it seemed to help for the first day. Now I see 100 ping loss for the last hours. When I check my connection, everything seems fine. At least I can use IPv6. Hm...?!? Regards, Martin

Please note Posting is only allowed when you are logged in.