Routing problem
Shadow Hawkins on Saturday, 28 October 2006 02:40:24
At last both my workstation, phoenix.pupeno.com, and my server, blue.powrefulnet.net, have IPv^. :D
But I can't route between them. From phoenix to blue I die in the second hoop:
phoenix ~ # traceroute6 blue.powerfulnet.net
traceroute to blue.powerfulnet.net (2001:41c8:1:5423::2) from 2001:4830:1200:4c::2, 30 hops max, 16 byte packets
1 gw-77.ewr-01.us.sixxs.net (2001:4830:1200:4c::1) 274.012 ms 300.641 ms 271.233 ms
2 0.ge0-3.cr1.ewr1.us.occaid.net (2001:4830:e2:29::1) 266.796 ms !N 286.751 ms !N 269.811 ms !N
From phoenix to sixxs.net it works:
phoenix ~ # traceroute6 sixxs.net
traceroute to sixxs.net (2001:838:1:1:210:dcff:fe20:7c7c) from 2001:4830:1200:4c::2, 30 hops max, 16 byte packets
1 gw-77.ewr-01.us.sixxs.net (2001:4830:1200:4c::1) 302.798 ms 269.38 ms 269.691 ms
2 0.ge0-3.cr1.ewr1.us.occaid.net (2001:4830:e2:29::1) 268.57 ms 271.076 ms 269.665 ms
3 v3323-mpd.cr1.lhr1.uk.occaid.net (2001:4830:fe:1010::1) 344.039 ms 345.314 ms 395.612 ms
4 2001:7f8:4::2331:1 (2001:7f8:4::2331:1) 346.425 ms 379.147 ms 356.54 ms
5 ge0-0-44.br0.nlams1.realroute.net (2001:1598:a:3::1) 351.361 ms 354.211 ms 351.246 ms
6 2001:1598:b031:2::1 (2001:1598:b031:2::1) 353.352 ms 352.615 ms 355.099 ms
7 ams-ix.ipv6.concepts.nl (2001:7f8:1::a501:2871:1) 354.211 ms 356.879 ms 354.606 ms
8 se1.breda.ipv6.concepts-ict.net (2001:838:0:10::2) 356.098 ms 354.168 ms 355.105 ms
9 noc.sixxs.net (2001:838:1:1:210:dcff:fe20:7c7c) 359.36 ms 356.233 ms 355.345 ms
Now from the server side, from blue to phoenix it dies on the 5th hop:
blue:~# traceroute6 phoenix.pupeno.com
traceroute to phoenix.pupeno.com (2001:4830:1200:4c::2) from 2001:41c8:1:5423::4, 30 hops max, 16 byte packets
1 2001:41c8:1:5423::1 (2001:41c8:1:5423::1) 0.355 ms 0.273 ms 0.269 ms
2 2001:41c8:0:3::3 (2001:41c8:0:3::3) 0.422 ms 0.565 ms 0.513 ms
3 2001:ba8:0:1af::1 (2001:ba8:0:1af::1) 1.311 ms 0.988 ms 0.886 ms
4 v6-tunnel102-uk6x.ipv6.btexact.com (2001:7f8:2:803c::2) 1.292 ms 1.338 ms 1.302 ms
5 tiscali-uk6x.ipv6.btexact.com (2001:7f8:2:1::3) 14.168 ms 12.834 ms 12.89 ms
6 * * *
7 * * *
8 * * *
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
From the server, phoenix, to sixxs.net:
blue:~# traceroute6 sixxs.net
traceroute to sixxs.net (2001:838:1:1:210:dcff:fe20:7c7c) from 2001:41c8:1:5423::4, 30 hops max, 16 byte packets
1 2001:41c8:1:5423::1 (2001:41c8:1:5423::1) 0.65 ms 0.909 ms 0.646 ms
2 2001:41c8:0:3::3 (2001:41c8:0:3::3) 1.186 ms 1.798 ms 1.615 ms
3 2001:ba8:0:1af::1 (2001:ba8:0:1af::1) 2.301 ms 42.645 ms 1.445 ms
4 v6-tunnel102-uk6x.ipv6.btexact.com (2001:7f8:2:803c::2) 80.492 ms 9.404 ms 52.911 ms
5 tiscali-uk6x.ipv6.btexact.com (2001:7f8:2:1::3) 51.233 ms 24.59 ms 12.949 ms
6 * * *
7 * * *
8 2001:7f8:4::2331:1 (2001:7f8:4::2331:1) 16.332 ms 14.366 ms 14.51 ms
9 ge0-0-44.br0.nlams1.realroute.net (2001:1598:a:3::1) 16.89 ms 20.298 ms 16.706 ms
10 * * *
11 ams-ix.ipv6.concepts.nl (2001:7f8:1::a501:2871:1) 14.787 ms 14.658 ms 14.889 ms
12 se1.breda.ipv6.concepts-ict.net (2001:838:0:10::2) 18.415 ms 16.793 ms 17.673 ms
13 noc.sixxs.net (2001:838:1:1:210:dcff:fe20:7c7c) 17.084 ms 16.693 ms 16.93 ms
Again problems at tiscali-uk6x.ipv6.btexact.com, but eventually it reached sixxs.net. Does anybody know what's going on here and/or how to solve it ?
Thank you.
PS: blue's ISP is Bytemark.
Routing problem
Jeroen Massar on Saturday, 28 October 2006 02:46:06
Contact Bytemark and suggest that they use a real transit instead of the tunneled provider they are using now. Can't do anything about that from here.
Routing problem
Shadow Hawkins on Friday, 17 November 2006 22:11:23
Hi all,
I also have a routing problem:
IPv6 between my server and my home machine used to work, but stopped working a few days ago (yumi is the dedicated server and melchior my home workstation):
|yumi:~# traceroute6 melchior.uguu.de
|traceroute to melchior.uguu.de (2001:a60:f020::f1) from 2002:550a:d2b7::1, 30 hops max, 16 byte packets
| 1 2001:a60:0:401::1 (2001:a60:0:401::1) 7.916 ms 7.376 ms 7.706 ms
| 2 * * *
| 3 * * *
| 4 * * *
|ranma@melchior:~$ traceroute6 yumi.uguu.de
|traceroute to yumi.uguu.de (2002:550a:d2b7::1) from 2001:a60:f020:0:217:31ff:feaa:1e89, 30 hops max, 24 byte packets
| 1 linksys.uguu.de (2001:a60:f020::fd) 2.47 ms 0.231 ms 0.209 ms
| 2 * * *
| 3 * * *
| 4 * * *
yumi is using a 6to4 tunnel:
|iface tun6to4 inet6 v4tunnel
| address 2002:550a:d2b7::1
| netmask 16
| endpoint any local 85.10.210.183 ttl 64
| up ip -6 route add 2000::/3 via ::192.88.99.1 dev tun6to4 metric 1
| down ip -6 route flush dev tun6to4
the local router linksys is using a 6to4 tunnel for 2002::/16 and the sixxs tunnel for everything else IPv6.
Traceroute to ftp.freenet.de work from both hosts:
|ranma@melchior:~$ traceroute6 ftp.freenet.de
|traceroute to ftp-0.freenet.de (2001:748:100:50::3) from 2001:a60:f020:0:217:31ff:feaa:1e89, 30 hops max, 24 byte packets
| 1 linksys.uguu.de (2001:a60:f020::fd) 0.467 ms 0.23 ms 0.204 ms
| 2 gw-42.muc-02.de.sixxs.net (2001:a60:f000:29::1) 8.473 ms 8.496 ms 8.307 ms
| 3 rt4.muc1.m-online.net (2001:a60:0:200::1) 11.088 ms 8.733 ms 10.915 ms
| 4 rt-decix.m-online.net (2001:a60::69:0:1:1) 16.205 ms 16.223 ms 16.316 ms
| 5 decix.mcbone.net (2001:7f8::1536:0:1) 16.815 ms 16.036 ms 16.279 ms
| 6 ge-2-0-0-0.dus2-j.mcbone.net (2001:748:0:10::2) 20.073 ms 19.985 ms 19.865 ms
| 7 F2-0-56.dus2-v6.mcbone.net (2001:748:1:a::16) 19.939 ms 19.867 ms 19.981ms
| 8 ftp0.freenet.de (2001:748:100:50::3) 20.451 ms 17.95 ms 19.874 ms
|yumi:~# traceroute6 ftp.freenet.de
|traceroute to ftp-0.freenet.de (2001:748:100:50::5) from 2002:550a:d2b7::1, 30 hops max, 16 byte packets
| 1 2001:a60:0:401::1 (2001:a60:0:401::1) 7.918 ms 7.429 ms 7.267 ms
| 2 fe-0-2-2.rt6.muc1.m-online.net (2001:a60:0:203::1:1) 8.341 ms 8.139 ms 7.65 ms
| 3 r1.nue2.m-online.net (2001:a60::911:201) 10.22 ms 10.139 ms 10.021 ms
| 4 ge-2-0-0-0.dus2-j.mcbone.net (2001:748:0:10::2) 16.541 ms 16.727 ms 16.256 ms
| 5 F2-0-56.dus2-v6.mcbone.net (2001:748:1:a::16) 17.34 ms 16.65 ms 16.534 ms
| 6 ftp2.freenet.de (2001:748:100:50::5) 16.349 ms 16.828 ms 17.076 ms
Traceroute to a 6to4 host also works from both hosts:
yumi<->ari works (not surprising)
|yumi:~# traceroute6 ari.tomodachi.de
|traceroute to ari.tomodachi.de (2002:d9a0:d777::1) from 2002:550a:d2b7::1, 30 hops max, 16 byte packets
| 1 ari.tomodachi.de (2002:d9a0:d777::1) 8.196 ms 7.309 ms 7.522 ms
|ari:~# traceroute6 yumi.uguu.de
|traceroute to yumi.uguu.de (2002:550a:d2b7::1) from 2002:d9a0:d777::1, 30 hops max, 16 byte packets
| 1 2002:550a:d2b7::1 (2002:550a:d2b7::1) 7.428 ms 6.794 ms 6.48 ms
melchior<->ari works (interesting)
|ranma@melchior:~$ traceroute6 ari.tomodachi.de
|traceroute to ari.tomodachi.de (2002:d9a0:d777::1) from 2001:a60:f020:0:217:31ff:feaa:1e89, 30 hops max, 24 byte packets
| 1 linksys.uguu.de (2001:a60:f020::fd) 2.984 ms 0.228 ms 0.221 ms
| 2 2002:d9a0:d777::1 (2002:d9a0:d777::1) 19.66 ms 19.876 ms 19.449 ms
|ari:~# traceroute6 melchior.uguu.de
|traceroute to melchior.uguu.de (2001:a60:f020::f1) from 2002:d9a0:d777::1, 30 hops max, 16 byte packets
| 1 2002:c058:6301::1 (2002:c058:6301::1) 28.885 ms 7.233 ms 7.301 ms
| 2 bbcr00-cc1-b18-a.six-de.net (2001:4b88:0:4:2:0:11:900) 8.118 ms 6.893 ms 7.519 ms
| 3 bbcr02-fra4-decixii-a.six-de.net (2001:4b88:0:4:2:2:11:0) 8.577 ms 8.486 ms 7.644 ms
| 4 rt-decix.m-online.net (2001:7f8::223f:0:1) 10.251 ms 9.683 ms 9.607 ms
| 5 * * *
| 6 * * *
| 7 linksys.uguu.de (2001:a60:f020::fd) 18.673 ms 18.507 ms 18.675 ms
| 8 melchior.uguu.de (2001:a60:f020::f1) 19.695 ms 18.066 ms 18.496 ms
So...
Where to go from there?
Thanks,
Tobias
Routing problem
Shadow Hawkins on Friday, 17 November 2006 22:49:56
First of all you should test whether the packets are lost on the forward path or the return path. Send a ping from yumi to melchior and tcpdump whether you see the echo requests on your SixXS tunnel. Check for echo replies being sent on your 6to4 tunnel on the Linksys in tcpdump, they should be sent directly in IPv4 to yumi. Check there.
Then send a ping from melchior to yumi and check whether you can see the proto41 packets in IPv4 on yumi.
I would strongly advise shutting down the additional 6to4 tunnel on your linksys, running both tunneled production space and 6to4 on the same CPE cries for trouble (for example due to failing uRPF checks). You don't need that, the relay is close enough.
Routing problem
Shadow Hawkins on Friday, 17 November 2006 23:50:50
Well, I added the 6to4 tunnel on linksys only after the problems showed up. :)
So, I guess I'll disable it.
Then, pinging yumi from melchior and running tcpdump, I can see pings leaving over the sixxs and ppp0 devices on linksys:
|root@linksys:~# tcpdump -n -s 2000 -i ppp0 proto 41
|tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
|listening on ppp0, link-type LINUX_SLL (Linux cooked), capture size 2000 bytes
|22:32:39.734505 IP 82.135.4.249 > 62.245.150.2: 2001:a60:f020:0:217:31ff:feaa:1e89 > 2002:550a:d2b7::1: icmp6: echo request seq 171
|22:32:40.734673 IP 82.135.4.249 > 62.245.150.2: 2001:a60:f020:0:217:31ff:feaa:1e89 > 2002:550a:d2b7::1: icmp6: echo request seq 172
On yumi nothing IPv6/Tunnel related is coming in over eth0.
The other way around looks more or less the same, I get outgoing tunneled ping request on eth0, but nothing is coming in on ppp0 on the linksys...
|yumi:~# tcpdump -n -s 2000 -i eth0 proto 41 and not port 22
|tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 2000 bytes
|23:44:26.584120 IP 85.10.210.183 > 192.88.99.1: 2002:550a:d2b7::1 > 2001:a60:f020::fd: icmp6: echo request seq 1
|23:44:27.585006 IP 85.10.210.183 > 192.88.99.1: 2002:550a:d2b7::1 > 2001:a60:f020::fd: icmp6: echo request seq 2
|23:44:28.584889 IP 85.10.210.183 > 192.88.99.1: 2002:550a:d2b7::1 > 2001:a60:f020::fd: icmp6: echo request seq 3
|23:44:29.584767 IP 85.10.210.183 > 192.88.99.1: 2002:550a:d2b7::1 > 2001:a60:f020::fd: icmp6: echo request seq 4
With the 6to4 tunnel on linksys in place, at least the ping requests reach yumi, but the responses still get lost.
Looking at some forum threads, it seems I should get a sixxs tunnel for my server too...
|root@linksys:~# ip -6 route ls
|2001:a60:f000:29::/64 via :: dev sixxs metric 256 mtu 1280 advmss 1220
|2001:a60:f020::/64 dev br0 metric 256 mtu 1500 advmss 1220
[fe80::/64 and ff00::/8 routes skipped]
|fec0::/64 dev br0 metric 256 mtu 1500 advmss 1220
|fec1::/48 via fe80::c01 dev tun_natsumi proto zebra metric 1024 mtu 1427 advmss 1367
|default via 2001:a60:f000:29::1 dev sixxs metric 1024 mtu 1280 advmss 1220
|yumi:~# ip -6 route ls
|2002::/16 dev tun6to4 metric 256 expires -2432339sec mtu 1480 advmss 1420 hoplimit 4294967295
|2000::/3 via ::192.88.99.1 dev tun6to4 metric 1 expires -2432339sec mtu 1480 advmss 1420 hoplimit 4294967295
[fe80::/64 and ff00::/8 routes skipped]
|fec0::/48 via fe80::8fd dev tun_nukunuku proto zebra metric 1024 expires -324764sec mtu 1427 advmss 1367 hoplimit 4294967295
|unreachable default dev lo proto none metric -1 error -101 hoplimit 255
(tun_natsumi and tun_nukunuku should actually be renamed tun_yumi and tun_linksys, they belong to an openvpn connection between the two machines)
[Edit: forgot to include the second tcpdump output]
Oh, and after removing the 6to4 tunnel traceroute to ari.tomodachi.de no longer works:
|ranma@melchior:~$ traceroute6 ari.tomodachi.de
|traceroute to ari.tomodachi.de (2002:d9a0:d777::1) from |2001:a60:f020:0:217:31ff:feaa:1e89, 30 hops max, 24 byte packets
| 1 linksys.uguu.de (2001:a60:f020::fd) 0.242 ms 0.238 ms 0.2 ms
| 2 gw-42.muc-02.de.sixxs.net (2001:a60:f000:29::1) 8.38 ms 8.222 ms 8.315 ms
| 3 rt4.muc1.m-online.net (2001:a60:0:200::1) 8.985 ms 8.696 ms 8.697 ms
| 4 * * *
| 5 * * *
| 6 * * *
When pinging from ari to linksys the ping replies now get lost.
Routing problem
Shadow Hawkins on Sunday, 19 November 2006 02:14:06
Hi Tobias,
I've contacted M-net about their local 6to4 relay, I see issues reaching some sites as well. Could you verify that yumi's chosen 6to4 relay is also at M-net (trace to 192.88.99.1)?
Regards,
Bernhard
Routing problem
Shadow Hawkins on Sunday, 19 November 2006 09:26:31
Yes, it is:
|ranma@yumi:~$ traceroute 192.88.99.1
|traceroute to 192.88.99.1 (192.88.99.1), 30 hops max, 38 byte packets
| 1 85-10-210-161.clients.your-server.de (85.10.210.161) 0.940 ms 0.642 ms 0.581 ms
| 2 hos-tr1.juniper1.rz4.hetzner.de (213.239.222.238) 0.381 ms 0.378 ms 0.338 ms
| 3 hos-bb1.juniper2.s02.hetzner.de (213.239.240.232) 0.592 ms 0.555 ms 0.545 ms
| 4 GigE-0.NEFkom.DHK.N-IX.net (195.85.217.67) 0.516 ms 0.547 ms 0.485 ms
| 5 ge-1-2-0.rt7.muc3.m-online.net (212.114.255.54) 6.850 ms 6.862 ms 6.862 ms
| 6 fa0-0.rt3.muc1.m-online.net (212.18.7.67) 7.138 ms * 7.075 ms
Routing problem
Shadow Hawkins on Sunday, 19 November 2006 10:54:08
BTW, it seems to be working again now:
|ranma@melchior:~$ traceroute6 ari.tomodachi.de
|traceroute to ari.tomodachi.de (2002:d9a0:d777::1) from 2001:a60:f020:0:217:31ff:feaa:1e89, 30 hops max, 24 byte packets
| 1 linksys.uguu.de (2001:a60:f020::fd) 0.963 ms 0.241 ms 0.21 ms
| 2 gw-42.muc-02.de.sixxs.net (2001:a60:f000:29::1) 8.306 ms 8.345 ms 11.295 ms
| 3 rt4.muc1.m-online.net (2001:a60:0:200::1) 8.901 ms 11.276 ms 8.917 ms
| 4 rt3.muc1.m-online.net (2001:a60::89:1:1:3) 11.331 ms 10.888 ms 11.297 ms
| 5 2002:d9a0:d777::1 (2002:d9a0:d777::1) 20.088 ms 19.944 ms 19.935 ms
Multicast doesn't work though, which was working yesterday, when I tried it for the first time. At least I could receive Frequence3 Radio using 'vlc udp://@[ff1e::8888]:8888' and mrd6 on my router. Today I don't see any pim messages from demuc02.
Thanks
Routing problem
Shadow Hawkins on Sunday, 19 November 2006 11:09:28
According to M-net one link between the router connected to the SixXS POP and the router with the 6to4 relay was missing MPLS. This has been fixed.
I'll have a look at Multicast later.
Regards,
Bernhard
Routing problem
Shadow Hawkins on Sunday, 26 November 2006 14:30:35
Multicast is working again:
|13:17:13.141918 fe80::3ef5:9602 > ff02::d: pim v2 Hello (Hold-time 1m45s) (Genid: 0x597220f0) (DR-Priority: 1) (addr-list) [hlim 1]
|13:17:43.141593 fe80::3ef5:9602 > ff02::d: pim v2 Hello (Hold-time 1m45s) (Genid: 0x597220f0) (DR-Priority: 1) (addr-list) [hlim 1]
|13:18:13.145207 fe80::3ef5:9602 > ff02::d: pim v2 Hello (Hold-time 1m45s) (Genid: 0x597220f0) (DR-Priority: 1) (addr-list) [hlim 1]
|13:18:29.393364 fe80::3ef5:9602 > ff02::1: HBH icmp6: multicast listener query max resp delay: 10000 addr: :: [hlim 1]
|13:18:43.137501 fe80::3ef5:9602 > ff02::d: pim v2 Hello (Hold-time 1m45s) (Genid: 0x597220f0) (DR-Priority: 1) (addr-list) [hlim 1]
|13:19:13.140628 fe80::3ef5:9602 > ff02::d: pim v2 Hello (Hold-time 1m45s) (Genid: 0x597220f0) (DR-Priority: 1) (addr-list) [hlim 1]
I'm listening to Frequence 3 (udp://@[ff1e::8888]:8888) right now:
|13:21:43.979604 2001:660:3302:2821:220:edff:fead:b6b8.56225 > ff1e::8888.8888: UDP, length: 940
|13:21:44.010379 2001:660:3302:2821:220:edff:fead:b6b8.56225 > ff1e::8888.8888: UDP, length: 940
|13:21:44.040933 2001:660:3302:2821:220:edff:fead:b6b8.56225 > ff1e::8888.8888: UDP, length: 940
FWIW, ecmh (static mips build) doesn't work because it gets confused over the vlan interfaces on my wrt54gs, the client gets each packet twice, which doesn't decode well...
mrd also has the advantage of being available as a package in the openwrt svn package repository, this is my /etc/mrd.conf:
------------------------cut here---------------------------
log {
/* Logs are controlled via the 'attach' method */
/* syntax (one of):
attach syslog [level]
attach stderr [level]
attach name filename [level]
where level is one of:
quiet, normal, verbose, debug or extradebug
*/
attach stderr debug;
/* attach default "mrd.log" debug; */
}
load-module console;
load-module mld;
load-module pim;
console {
/* Allow access from any host with admin/admin */
/* allow-access admin admin any; */
/* Command format: */
/* allow-access [username [password [address mask]]]; */
}
mrib {
local 2001:a60:f020::/48;
prefix 2000::/3 via 2001:a60:f000:29::1;
}
/* Groups configuration */
groups {
/* group mask */
ff0e::/16 {
pim rp 2001:660:3007:300:1::;
}
ff1e::/16 {
pim rp 2001:660:3007:300:1::;
}
ff3e::/16 {
pim rp 2001:660:3007:300:1::;
}
}
---------------------------cut here---------------------------
Routing problem
Shadow Hawkins on Friday, 17 November 2006 23:23:07
Jose,
your problem is most probably
traceroute to blue.powerfulnet.net (2001:41c8:1:5423::2) from 2001:4830:1200:4c::2, 30 hops max, 16 byte packets
1 gw-77.ewr-01.us.sixxs.net (2001:4830:1200:4c::1) 274.012 ms 300.641 ms 271.233 ms
2 0.ge0-3.cr1.ewr1.us.occaid.net (2001:4830:e2:29::1) 266.796 ms !N 286.751 ms !N 269.811 ms !N
that OCCAID still (for a couple of months now) has an incomplete routing table (!N means "no route to destination"). See http://www.sixxs.net/tools/grh/status/, OCCAID has 552 at the moment, a full heavily filtered table are about 670 routes, an unfiltered table like OCCAID was known to carry is >700 routes large.
You have to wait for OCCAID to fix their routing, but I've lost my hope.
EDIT: Jeroen is quite right about BYTEMARK having crappy connectivity, they appear to only have one upstream (Jump Networks, AS8943) which relies on a crappy routeswap facility called Uk6x (AS1752) for external connectivity.
According to RIPE records Bytemark buys connectivity at Bogons Ltd. (AS3213) which is not much better in this regards but would provide an alternative path. And Jump has Level3 (AS3356) and KPN (AS286), both IPv6-enabled and sane carriers (although probably not able to deliver natively).
Kick Bytemark to get better connectivity, if you are lucky another path might make it through to OCCAID.
Regards,
Bernhard
Posting is only allowed when you are logged in. |