|
Ticket ID: SIXXS #819459 Ticket Status: Resolved PoP: uschi02 - Your.Org, Inc. (Chicago, Illinois)
No tunnel connectivity to PoP
![[us]](/s/countries/us.gif) Shadow Hawkins on Saturday, 04 October 2008 01:29:52
Greetings,
I put in a request for a tunnel earlier this week. I had intended to originate the tunnel from a Linksys WRT54G wireless router running OpenWRT WhiteRussian rc4, which serves as my gateway router to the world. It connects to my ISP (AT&T) via PPPoE. I installed the latest AICCU client and had attempted to bring up the tunnel, but was not able to run a test past step 6/8 (Ping PoP IPv6 Address). During my troubleshooting, I had been watching the tunnel endpoint via tcpdump and had seen pings from the PoP to my router, echoed back. Feeling that perhaps something was wrong with the router, I loaded a Virtual Machine of OpenWRT (Kamakazi 7.09, x86-2.6). I brought it's firewall down (accept all), loaded AICCU, configured, and successfully brought up the tunnel. I could ping the PoP and my tunnel information showed that the tunnel was alive and active. I left it up to establish history and gain credit, though at some point during the night of the first day the VM lost connectivity.
The following day, still wanting the tunnel on my Linksys, I tried to move the tunnel back to that device, to no success (troubleshooting included opening it's firewall to the world for a time). Again, wanting to establish a history, I went ahead and tried to reestablish the tunnel from the VM, only this time I do not receive any responses back from the VM. I have given it some time to see if it is a transient problem, but it does not appear to be. I have tried to reestablish the tunnel from the Linksys, the OpenWRT VM, my primary server (Fedora Core 6, x86_64), and a Windows XP workstation, all with no success following the initial loss of the tunnel.
Information I've gathered as best as possible as follows:
Linksys device:
root@firefox:~# uname -a
Linux firefox 2.4.30 #1 Wed Nov 23 22:35:53 CET 2005 mips unknown
OpenWRT VM:
root@OpenWrt:~# uname -a
Linux OpenWrt 2.6.22 #2 Sun Sep 30 21:02:32 CEST 2007 i686 unknown
primary server:
root@foxbox {~}# uname -a
Linux foxbox.kittyfox.net 2.6.20-1.2962.fc6xen #1 SMP Tue Jun 19 19:10:51 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
root@firefox:/sbin# aiccu start
Tunnel Information for T17264:
POP Id : uschi02
IPv6 Local : 2001:4978:f:1bc::2/64
IPv6 Remote : 2001:4978:f:1bc::1/64
Tunnel Type : ayiya
Adminstate : enabled
Userstate : enabled
root@firefox:/sbin# route -n -A inet6
Kernel IPv6 routing table
Destination Next Hop Flags Metric Ref Use Iface
::1/128 :: U 0 10 1 lo
2001:4978:f:1bc::2/128 :: U 0 0 0 lo
2001:4978:f:1bc::/64 :: U 256 1 0 sixxs
fe80::4878:f:1bc:2/128 :: U 0 0 0 lo
fe80::/64 :: U 256 0 0 sixxs
ff00::/8 :: U 256 0 0 sixxs
::/0 :: UD 256 0 0 sixxs
::/0 2001:4978:f:1bc::1 UG 1024 0 0 sixxs
Ping attempts when I was receiving traffic from PoP:
root@firefox:/sbin# ping 216.14.98.22
PING 216.14.98.22 (216.14.98.22): 56 data bytes
64 bytes from 216.14.98.22: icmp_seq=0 ttl=58 time=12.2 ms
64 bytes from 216.14.98.22: icmp_seq=1 ttl=58 time=11.9 ms
64 bytes from 216.14.98.22: icmp_seq=2 ttl=58 time=12.7 ms
64 bytes from 216.14.98.22: icmp_seq=3 ttl=58 time=12.9 ms
--- 216.14.98.22 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 11.9/12.4/12.9 ms
root@firefox:/sbin# ping6 2001:4978:f:1bc::1
PING 2001:4978:f:1bc::1 (2001:4978:f:1bc::1): 56 data bytes
--- 2001:4978:f:1bc::1 ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss
root@firefox:~# tcpdump -i ppp0 -n host 216.14.98.22
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ppp0, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
20:33:03.346098 IP 75.58.40.237.2075 > 216.14.98.22.5072: UDP, length: 44
20:34:03.355997 IP 75.58.40.237.2075 > 216.14.98.22.5072: UDP, length: 44
20:34:07.160768 IP 75.58.40.237 > 216.14.98.22: icmp 64: echo request seq 0
20:34:07.168460 IP 216.14.98.22 > 75.58.40.237: icmp 64: echo reply seq 0
20:34:08.155797 IP 75.58.40.237 > 216.14.98.22: icmp 64: echo request seq 1
20:34:08.163318 IP 216.14.98.22 > 75.58.40.237: icmp 64: echo reply seq 1
20:34:09.155801 IP 75.58.40.237 > 216.14.98.22: icmp 64: echo request seq 2
20:34:09.164078 IP 216.14.98.22 > 75.58.40.237: icmp 64: echo reply seq 2
20:34:10.155799 IP 75.58.40.237 > 216.14.98.22: icmp 64: echo request seq 3
20:34:10.164345 IP 216.14.98.22 > 75.58.40.237: icmp 64: echo reply seq 3
20:34:46.333942 IP 75.58.40.237.2075 > 216.14.98.22.5072: UDP, length: 152
20:34:47.346352 IP 75.58.40.237.2075 > 216.14.98.22.5072: UDP, length: 152
20:34:48.346350 IP 75.58.40.237.2075 > 216.14.98.22.5072: UDP, length: 152
20:34:49.346350 IP 75.58.40.237.2075 > 216.14.98.22.5072: UDP, length: 152
Current attempts yield this:
root@firefox:~# aiccu start
Tunnel Information for T17264:
POP Id : uschi02
IPv6 Local : 2001:4978:f:1bc::2/64
IPv6 Remote : 2001:4978:f:1bc::1/64
Tunnel Type : ayiya
Adminstate : enabled
Userstate : enabled
root@firefox:~# ip -f inet6 route
2001:4978:f:1bc::/64 dev sixxs metric 256 mtu 1280 advmss 1220
fe80::/64 dev sixxs metric 256 mtu 1280 advmss 1220
ff00::/8 dev sixxs metric 256 mtu 1280 advmss 1220
default via 2001:4978:f:1bc::1 dev sixxs metric 1024 mtu 1280 advmss 1220
root@firefox:~# ping6 -c 3 2001:4978:f:1bc::1
PING 2001:4978:f:1bc::1 (2001:4978:f:1bc::1): 56 data bytes
--- 2001:4978:f:1bc::1 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
root@firefox:~# tcpdump -i sixxs
tcpdump: WARNING: arptype 65534 not supported by libpcap - falling back to cooked socket
tcpdump: WARNING: sixxs: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on sixxs, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
18:26:03.534716 2001:4978:f:1bc::2 > 2001:4978:f:1bc::1: icmp6: echo request seq 0
18:26:04.535704 2001:4978:f:1bc::2 > 2001:4978:f:1bc::1: icmp6: echo request seq 256
18:26:05.535659 2001:4978:f:1bc::2 > 2001:4978:f:1bc::1: icmp6: echo request seq 512
root@firefox:~# tcpdump -i ppp0 host 216.14.98.22
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ppp0, link-type LINUX_SLL (Linux cooked), capture size 96 bytes
18:26:03.535529 IP adsl-75-58-40-237.dsl.emhril.sbcglobal.net.2082 > sixxs.cx01.chi.bb.your.org.5072: UDP, length: 152
18:26:04.536407 IP adsl-75-58-40-237.dsl.emhril.sbcglobal.net.2082 > sixxs.cx01.chi.bb.your.org.5072: UDP, length: 152
18:26:05.536364 IP adsl-75-58-40-237.dsl.emhril.sbcglobal.net.2082 > sixxs.cx01.chi.bb.your.org.5072: UDP, length: 152
18:26:49.875987 IP adsl-75-58-40-237.dsl.emhril.sbcglobal.net.2082 > sixxs.cx01.chi.bb.your.org.5072: UDP, length: 44
Based on this netstat...
root@firefox:~# netstat -n
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
udp 0 0 75.58.40.237:2082 216.14.98.22:5072 ESTABLISHED
... I believe traffic inbound to the router should be cleared by the following firewall rules:
root@firefox:~# iptables -L INPUT -v -n
Chain INPUT (policy ACCEPT 26 packets, 2244 bytes)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID
77722 6212K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
...
Any help that is available would be greatly appreciated.
---
Chris Wroten
CWQ1-SIXXS
State change: user
The state of this ticket has been changed to user
No tunnel connectivity to PoP
We don't see any packets coming in from 75.58.40.237 on the uschi02 PoP.
Are you sure that the firewall is fully letting packets out?
traceroute to 75.58.40.237 (75.58.40.237), 64 hops max, 40 byte packets
1 ge-0.0.0-30.core1.chi.bb.your.org (216.14.98.21) 9.172 ms 1.267 ms 8.949 ms
2 po1-107.ar1.ord1.us.nlayer.net (69.31.111.25) 1.235 ms 1.269 ms 1.153 ms
3 ex2-tg4-0.eqchil.sbcglobal.net (151.164.249.109) 1.159 ms 1.359 ms 1.193 ms
4 151.164.94.40 (151.164.94.40) 99.049 ms 176.450 ms 8.978 ms
5 dist1-g2-3.emhril.sbcglobal.net (151.164.94.165) 1.292 ms 4.500 ms 1.312 ms
6 se1-g9-1.emhril.sbcglobal.net (68.22.72.190) 1.213 ms 1.329 ms 1.232 ms
7 adsl-75-58-40-237.dsl.emhril.sbcglobal.net (75.58.40.237) 8.478 ms 8.012 ms 8.022 ms
Traceroute works at least, thus traffic should be able to get back.
No tunnel connectivity to PoP
Shadow Hawkins on Sunday, 05 October 2008 08:02:07
I would tend to agree, traffic should be able to pass. The output firewall rules are:
Chain OUTPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID
2632 746K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
141 11989 output_rule all -- * * 0.0.0.0/0 0.0.0.0/0
130 10501 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 REJECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 reject-with tcp-reset
0 0 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
Chain output_rule (1 references)
pkts bytes target prot opt in out source destination
11 1488 ACCEPT all -- * * 0.0.0.0/0 216.14.98.22
OUTPUT hops to output_rule (intended as a user-defined chain, with OUTPUT holding common 'sane' rules). Most traffic gets to OUTPUT rule 4 and is fully accepted in that statement. I added output_rule rule 1 to explicitly define acceptance for 216.14.98.22, and for 11 packets, instead of being accepted in OUTPUT, they're accepted there instead.
I did have the tunnel down, I wasn't sure what point there was. That's my mistake not to have left it up. I brought it up on my server tonight after reading the response and it works from there now, whereas earlier today it did not. I was out for most of the afternoon, and did not change anything earlier. However, even now, the router does not get traffic out, where the server now can. Distressing. The only difference I see is that the tunnel is being masqueraded when it comes from the server, and it is not when running from the router
I may have to contact my ISP and quiz them. I asserted that I did not need filters when I set up my line, and was informed that it was set to no filtering at all. I don't know how it would be able to differentiate between origination at the router and origination at the server, so that may be a wild goose, but every time I look at this, all the configuration settings on my side seem to indicate it should be open and functional.
I'm not sure what would be best to troubleshoot this, with the time lag. Have the tunnel on the router (failing) or on the server (currently working).
No tunnel connectivity to PoP
Which kind of 'router' do you have that it supports AYIYA? I can't name a single commercial product which has this unfortunately.
AYIYA itself should not have any problems with masquerading/NAT.
No tunnel connectivity to PoP
Shadow Hawkins on Sunday, 05 October 2008 17:31:57
I listed it in the first line or two of my original post. A Linksys WRT54G running OpenWRT Whiterussian rc4. I need to upgrade it to Kamikaze, but I've delayed.
No tunnel connectivity to PoP
Shadow Hawkins on Sunday, 05 October 2008 17:55:45
Bah. Replied too quickly. Additional info:
I left the tunnel up overnight originating from the Linksys router, and had a tcpdump going to watch for traffic. I see this:
10:43:43.475827 IP (tos 0x0, ttl 64, id 27372, offset 0, flags [DF], length: 72) 75.58.40.237.2098 > 216.14.98.22.5072: [udp sum ok] UDP, length: 44
10:43:51.356183 IP (tos 0x0, ttl 58, id 9493, offset 0, flags [none], length: 176) 216.14.98.22.5072 > 75.58.40.237.2098: UDP, length: 148
10:43:51.358711 IP (tos 0x0, ttl 64, id 27373, offset 0, flags [DF], length: 180) 75.58.40.237.2098 > 216.14.98.22.5072: UDP, length: 152
10:43:59.568226 IP (tos 0x0, ttl 58, id 11818, offset 0, flags [none], length: 176) 216.14.98.22.5072 > 75.58.40.237.2098: UDP, length: 148
10:43:59.570080 IP (tos 0x0, ttl 64, id 27374, offset 0, flags [DF], length: 180) 75.58.40.237.2098 > 216.14.98.22.5072: UDP, length: 152
10:44:07.834989 IP (tos 0x0, ttl 58, id 14156, offset 0, flags [none], length: 176) 216.14.98.22.5072 > 75.58.40.237.2098: UDP, length: 148
10:44:07.837695 IP (tos 0x0, ttl 64, id 27375, offset 0, flags [DF], length: 180) 75.58.40.237.2098 > 216.14.98.22.5072: UDP, length: 152
10:44:43.485825 IP (tos 0x0, ttl 64, id 27376, offset 0, flags [DF], length: 72) 75.58.40.237.2098 > 216.14.98.22.5072: [udp sum ok] UDP, length: 44
10:45:43.495854 IP (tos 0x0, ttl 64, id 27377, offset 0, flags [DF], length: 72) 75.58.40.237.2098 > 216.14.98.22.5072: [udp sum ok] UDP, length: 44
10:46:43.505856 IP (tos 0x0, ttl 64, id 27378, offset 0, flags [DF], length: 72) 75.58.40.237.2098 > 216.14.98.22.5072: [udp sum ok] UDP, length: 44
10:47:43.515853 IP (tos 0x0, ttl 64, id 27379, offset 0, flags [DF], length: 72) 75.58.40.237.2098 > 216.14.98.22.5072: [udp sum ok] UDP, length: 44
10:48:42.013266 IP (tos 0x0, ttl 64, id 27380, offset 0, flags [DF], length: 180) 75.58.40.237.2098 > 216.14.98.22.5072: UDP, length: 152
10:48:43.016315 IP (tos 0x0, ttl 64, id 27381, offset 0, flags [DF], length: 180) 75.58.40.237.2098 > 216.14.98.22.5072: UDP, length: 152
10:48:43.525890 IP (tos 0x0, ttl 64, id 27382, offset 0, flags [DF], length: 72) 75.58.40.237.2098 > 216.14.98.22.5072: [udp sum ok] UDP, length: 44
From what I can gather, the 44 length packets are the heartbeat, the pings originating from the PoP (148 and 152 length pings back to back) are to check tunnel status, and my router replies. The 152 length pings at 10:48:42 and :43 go unanswered. That was my attempt to ping.
On a sidenote, to assert where tcpdump pulls it's packets from, I put an explicit deny into the output rules going to 216.14.98.22 and tried to ping the PoP end of the ipv6 tunnel (2001:.....) and no packets showed up. To me that indicates that if tcpdump sees it, it's passed through routing and firewalling and is now about to be written out onto the wire.
No tunnel connectivity to PoP
Which version of AICCU are you using and who compiled it?
AYIYA-alike Packets arrive at the PoP when trying to ping your tunnel endpoint, and the contents look like ICMPv6 Echo Replies, but they are not correct AYIYA packets.
Please run it in debug + verbose mode, that should show the necessary details.
No tunnel connectivity to PoP
Shadow Hawkins on Tuesday, 07 October 2008 05:22:04
I was using a backported version of the 2007.01.15 AICCU client, as built by the OpenWRT Maintainers. Build location from the Sixxs homepage pointing to http://downloads.openwrt.org/backports/0.9/, package aiccu_20070115-1_mipsel.ipk. From the Packages file, it lists the following information:
Package: aiccu
Version: 20070115-1
Depends: libpthread
Section: ipv6
Architecture: mipsel
Maintainer: OpenWrt Developers Team <openwrt-devel@openwrt.org>
MD5Sum: 7d41f8fbb9b1735ca65e083df18cdfbc
Size: 29269
Filename: aiccu_20070115-1_mipsel.ipk
Source: /home/san1/tests/openwrt/packages/ipv6/aiccu
Description: SixXS Automatic IPv6 Connectivity Client Utility
Additionally, I did a full packet capture as the AYIYA tunnel packets left my WAN interface and did indeed confirm that the encapsulated IPv6 packet is malformed. Watching the WAN interface when the tunnel originates on my server and Wireshark decodes the AYIYA packet, as well as the contained IPv6 ICMP echo request just fine.
I'm probably using the software wrong, on an outdated firmware. For the time being, I'll originate the tunnel on a system that can handle the tunnel software properly and work on upgrading the linksys router. As the problem is on my end, I think we can consider this problem done for now.
Thanks!
State change: resolved
The state of this ticket has been changed to resolved
|
![[ch]](/s/countries/ch.gif)
on Saturday, 04 October 2008 19:58:57
Posting is only allowed when you are