|
Ticket ID: SIXXS #1211602 Ticket Status: User PoP: deham01 - Easynet (Hamburg)
deham01 availability / routing problem
![[de]](/s/countries/de.gif) Shadow Hawkins on Wednesday, 23 September 2009 08:32:29
Hi,
deham01 routing seems to be down from at least these two sites:
- BBH2-SIXXS T13275 (via arcor adsl, Braunschweig)
- PMC1-SIXXS T13584 (via qsc sdsl, Hildesheim)
Problem start: somewhere between 22.09.2009/17:30 and 23.09.2009/07:30
Problem end: -
Description:
- no ipv6 routing for affected sites
- In both cases, the aiccu tunnel is created without problems, but
the pops ipv6 end of the tunnel doesn't ping.
hildesheim:~# ip a s sixxs
19: sixxs@NONE: <POINTOPOINT,NOARP,UP,10000> mtu 1280 qdisc noqueue
link/sit 212.202.120.50 peer 212.224.0.188
inet6 2001:6f8:900:a6e::2/64 scope global
valid_lft forever preferred_lft forever
[...]
stargate:~# ping6 -c3 2001:6f8:900:a6e::1
PING 2001:6f8:900:a6e::1(2001:6f8:900:a6e::1) 56 data bytes
3 packets transmitted, 0 received, 100% packet loss, time 2014ms
- the ipv4 addresses of deham01 are ok
- local 4/6 routing table checked, no problems
- tcpdump shows outgoing packets on both, the tunnel
and the underlying interface, but no reply at all
- ping/traceroute from outside (e.g. ede01) ends at the pops side of the tunnel
e.g:
fischmarkt:~# ping6 stargate.ext.pengutronix.de
PING stargate.ext.pengutronix.de(stargate1.ipv6.pengutronix.de) 56 data bytes
From deham01.sixxs.net icmp_seq=1 Destination unreachable: No route
fischmarkt:~# traceroute to stargate.ext.pengutronix.de (2001:6f8:1178:4::1) from 2001:7b8:2ff:1a3::2, 30 hops max, 16 byte packets
1 gw-420.ede-01.nl.sixxs.net (2001:7b8:2ff:1a3::1) 22.244 ms 21.204 ms 21.311 ms
2 sixxs-gw.jun1.kelvin.network.bit.nl (2001:7b8:3:4f::2) 21.415 ms 21.423 ms 21.415 ms
3 803.ae0.jun1.fra4.network.bit.nl (2001:7b8:0:323::2) 28.748 ms 30.919 ms 29.693 ms
4 fe0-1-2-0.br1.ixfra.de.easynet.net (2001:7f8::11ed:0:1) 29.911 ms 30.833 ms 30.623 ms
5 2001:6f8:1:0:87:86:77:248 (2001:6f8:1:0:87:86:77:248) 30.396 ms 31.042 ms 31.219 ms
6 2001:6f8:1:0:87:86:71:233 (2001:6f8:1:0:87:86:71:233) 30.255 ms 31.052 ms 30.673 ms
7 ge1-14-151.br3.isham.de.easynet.net (2001:6f8:800:0:1:0:d4e0:5e0) 42.323 ms 42.161 ms 42.921 ms
8 ge9-15.cr20.isham.de.easynet.net (2001:6f8:800:0:1:0:d4e0:45d) 43.551 ms 43.404 ms 42.824 ms
9 deham01.sixxs.net (2001:6f8:800:1003::2) 43.604 ms !N 43.624 ms !N 44.97 ms !N
- os type is Linux 2.6 amd64, aiccu 20070115-9 for Braunschweig
- os type is Linux 2.6 i386, aiccu 20070115-7 for Hildesheim
- tunnels worked for several months without problems
- Aiccu Test from Braunschweig, firewall temp. disabled:
hobbes:~# aiccu test
Tunnel Information for T13275:
POP Id : deham01
IPv6 Local : 2001:6f8:900:a15::2/64
IPv6 Remote : 2001:6f8:900:a15::1/64
Tunnel Type : ayiya
Adminstate : enabled
Userstate : enabled
#######
####### AICCU Quick Connectivity Test
#######
####### [1/8] Ping the IPv4 Local/Your Outer Endpoint (192.168.23.42)
### This should return so called 'echo replies'
### If it doesn't then check your firewall settings
### Your local endpoint should always be pingable
### It could also indicate problems with your IPv4 stack
PING 192.168.23.42 (192.168.23.42) 56(84) bytes of data.
64 bytes from 192.168.23.42: icmp_seq=1 ttl=64 time=0.028 ms
64 bytes from 192.168.23.42: icmp_seq=2 ttl=64 time=0.020 ms
64 bytes from 192.168.23.42: icmp_seq=3 ttl=64 time=0.038 ms
--- 192.168.23.42 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.020/0.028/0.038/0.009 ms
######
Did this work? [Y/n] y
####### [2/8] Ping the IPv4 Remote/PoP Outer Endpoint (212.224.0.188)
### These pings should reach the PoP and come back to you
### In case there are problems along the route between your
### host and the PoP this could not return replies
### Check your firewall settings if problems occur
PING 212.224.0.188 (212.224.0.188) 56(84) bytes of data.
64 bytes from 212.224.0.188: icmp_seq=1 ttl=57 time=31.5 ms
64 bytes from 212.224.0.188: icmp_seq=2 ttl=57 time=29.9 ms
64 bytes from 212.224.0.188: icmp_seq=3 ttl=57 time=29.5 ms
--- 212.224.0.188 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 29.550/30.328/31.512/0.850 ms
######
Did this work? [Y/n] y
####### [3/8] Traceroute to the PoP (212.224.0.188) over IPv4
### This traceroute should reach the PoP
### In case this traceroute fails then you have no connectivity
### to the PoP and this is most probably the problem
traceroute to 212.224.0.188 (212.224.0.188), 64 hops max, 52 byte packets
1 192.168.23.1 (192.168.23.1) 3 ms 1 ms 1 ms
2 dslb-092-076-128-001.pools.arcor-ip.net (92.76.128.1) 6 ms 6 ms 6 ms
3 88.79.26.37 (88.79.26.37) 7 ms 7 ms 6 ms
4 88.79.26.113 (88.79.26.113) 7 ms 7 ms 7 ms
5 92.79.212.81 (92.79.212.81) 8 ms 7 ms 7 ms
6 92.79.213.122 (92.79.213.122) 14 ms 14 ms 14 ms
7 te0-0-0-5.er10.ixfra.de.easynet.net (80.81.192.14) 16 ms 16 ms 16 ms
8 te0-0-0.gr10.isham.de.easynet.net (87.86.77.65) 29 ms 29 ms 29 ms
9 ge3-7.br2.isham.de.easynet.net (87.86.71.241) 27 ms 26 ms 27 ms
10 ge7-1.cr20.isham.de.easynet.net (212.224.4.89) 29 ms 29 ms 30 ms
11 deham01.sixxs.net (212.224.0.188) 30 ms 30 ms 30 ms
######
Did this work? [Y/n] y
###### [4/8] Checking if we can ping IPv6 localhost (::1)
### This confirms if your IPv6 is working
### If ::1 doesn't reply then something is wrong with your IPv6 stack
PING ::1(::1) 56 data bytes
64 bytes from ::1: icmp_seq=1 ttl=64 time=0.019 ms
64 bytes from ::1: icmp_seq=2 ttl=64 time=0.020 ms
64 bytes from ::1: icmp_seq=3 ttl=64 time=0.028 ms
--- ::1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.019/0.022/0.028/0.005 ms
######
Did this work? [Y/n] y
###### [5/8] Ping the IPv6 Local/Your Inner Tunnel Endpoint (2001:6f8:900:a15::2)
### This confirms that your tunnel is configured
### If it doesn't reply then check your interface and routing tables
PING 2001:6f8:900:a15::2(2001:6f8:900:a15::2) 56 data bytes
64 bytes from 2001:6f8:900:a15::2: icmp_seq=1 ttl=64 time=0.021 ms
64 bytes from 2001:6f8:900:a15::2: icmp_seq=2 ttl=64 time=0.027 ms
64 bytes from 2001:6f8:900:a15::2: icmp_seq=3 ttl=64 time=0.021 ms
--- 2001:6f8:900:a15::2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.021/0.023/0.027/0.003 ms
######
Did this work? [Y/n] y
###### [6/8] Ping the IPv6 Remote/PoP Inner Tunnel Endpoint (2001:6f8:900:a15::1)
### This confirms the reachability of the other side of the tunnel
### If it doesn't reply then check your interface and routing tables
### Don't forget to check your firewall of course
### If the previous test was succesful then this could be both
### a firewalling and a routing/interface problem
PING 2001:6f8:900:a15::1(2001:6f8:900:a15::1) 56 data bytes
--- 2001:6f8:900:a15::1 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2016ms
######
Did this work? [Y/n] n
- IPV4 Traceroute to deham01 (from Braunschweig)
hobbes:~# traceroute 212.224.0.188
traceroute to 212.224.0.188 (212.224.0.188), 64 hops max, 52 byte packets
1 192.168.23.1 (192.168.23.1) 1 ms 1 ms 1 ms
2 dslb-092-076-128-001.pools.arcor-ip.net (92.76.128.1) 6 ms 5 ms 6 ms
3 88.79.26.37 (88.79.26.37) 7 ms 6 ms 7 ms
4 88.79.26.113 (88.79.26.113) 7 ms 7 ms 7 ms
5 92.79.212.81 (92.79.212.81) 7 ms 7 ms 7 ms
6 92.79.213.122 (92.79.213.122) 15 ms 15 ms 14 ms
7 te0-0-0-5.er10.ixfra.de.easynet.net (80.81.192.14) 15 ms 16 ms 16 ms
8 te0-0-0.gr10.isham.de.easynet.net (87.86.77.65) 29 ms 29 ms 29 ms
9 ge3-7.br2.isham.de.easynet.net (87.86.71.241) 27 ms 27 ms 27 ms
10 ge7-1.cr20.isham.de.easynet.net (212.224.4.89) 29 ms 29 ms 29 ms
11 deham01.sixxs.net (212.224.0.188) 30 ms 29 ms 31 ms
h
hobbes:~# traceroute6 2001:6f8:900:a15::1
traceroute to 2001:6f8:900:a15::1 (2001:6f8:900:a15::1) from 2001:6f8:900:a15::2, port 33434, from port 62292, 30 hops max, 60 byte packets
1 * * *
2 * * *
3 * * *
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
hobbes:~# ip6tables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
hobbes:~# ip -6 r s
2001:6f8:900:a15::/64 dev sixxs proto kernel metric 256 mtu 1280 advmss 1220 hoplimit 4294967295
2001:6f8:1158:1::/64 dev eth0 proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
2001:6f8:1158:42::1 dev sixxs proto kernel metric 256 mtu 1280 advmss 1220 hoplimit 4294967295
unreachable 2001:6f8:1158::/48 dev lo metric 1024 error -101 mtu 16436 advmss 16376 hoplimit 4294967295
fe80::/64 dev eth0 proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
fe80::/64 dev sixxs proto kernel metric 256 mtu 1280 advmss 1220 hoplimit 4294967295
default via 2001:6f8:900:a15::1 dev sixxs metric 1024 mtu 1280 advmss 1220 hoplimit 4294967295
State change: user
The state of this ticket has been changed to user
deham01 availability / routing problem
From deham01.sixxs.net icmp_seq=1 Destination unreachable: No route
That indicates that the tunnel/subnet is not configured on the PoP.
2001:6f8:1158:42::1 dev sixxs proto kernel metric 256 mtu 1280 advmss 1220 hoplimit 4294967295
Why are you assigning subnet prefixes to the tunnel interface?
The routing table does indicate the tunnel/64 is on that interface, but not if the actual tunnel::2 IP is on it.
deham01 availability / routing problem
Shadow Hawkins on Wednesday, 23 September 2009 11:14:41
Hi Jeroen,
first: It works again, thnx :-)
Just for the record:
FYI: The problem first occured yesterday 22.09.2009 at 22:00 CEST.
and ended today at ~ 9:40 for both sites. Maybe this helps.
http://www.penguin.de/cgi-bin/smokeping.cgi?displaymode=n;start=2009-09-22 05:06;end=now;target=Pengutronix.Stargate6
http://www.penguin.de/cgi-bin/smokeping.cgi?displaymode=n;start=2009-09-22 05:08;end=now;target=Penguin.hobbes
That indicates that the tunnel/subnet is not configured on the PoP
Hmm, but wouldn't aiccu complain, if the tunnel wasn't configured?
The tunnel itself was established without problems, only routing
was affected.
Why are you assigning subnet prefixes to the tunnel interface?
Because, currently, my workstation at home is also the tunnel endpoint.
I need a reverse-resolvable source ip address from the local /48 for
one of my services (which in this situation only works, if it is
set on the tunnel interface). This is no problem for other hosts in
the local subnet, as they are using the correct source address right-away.
From a technical point of view, I don't see a problem with this. But
if there is a better way, I'd appreciate any hint to documentation.
Greetings from Hildesheim,
Bjrn
deham01 availability / routing problem
Shadow Hawkins on Wednesday, 23 September 2009 11:24:15The routing table does indicate the tunnel/64 is on that interface, but not if the actual tunnel::2 IP is on it.
Sorry, I forgot the interface output.
It didn't show up in the routing table output,
but as you can see in the initial tickets "aiccu test"
output [5/8] it was up and available.
hobbes:~# ip -6 a s sixxs
18: sixxs: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 qlen 500
inet6 2001:6f8:900:a15::2/64 scope global
valid_lft forever preferred_lft forever
inet6 2001:6f8:1158:42::1/128 scope global
valid_lft forever preferred_lft forever
inet6 fe80::4f8:900:a15:2/64 scope link
valid_lft forever preferred_lft forever
|
![[ch]](/s/countries/ch.gif)
on Wednesday, 23 September 2009 09:50:59
Posting is only allowed when you are