| Ticket ID: SIXXS #877222 Ticket Status: User PoP: uschi02 - Your.Org, Inc. (Chicago, Illinois) 
No responses from PoP endpoint ![[us]](/s/countries/us.gif) Shadow Hawkins on Saturday, 13 December 2008 01:03:06 
I have read and followed the "Reporting Problems" section on the Contact page and am providing the following details for this report based on the list of items stated there:
My 6in4 tunnel stopped working around 11am CST on 12/11/2008.  
NIC handle: HCL1-SIXXS
Tunnel ID: T18452
Setup Information:
AICCU 2007.01.15-console-linux by Jeroen Massar
Installed using Debian apt-get command on Ubuntu 8.10 Desktop Linux distro 
Linux kernel 2.6.27-9
AICCU is run on a Ubuntu Linux desktop connected to a pfSense (FreeBSD + PF firewall + NAT) box.  The pfSense box has one interface (rl0) with the public IPv4 address, and the other interface (sl0) connected to my Ubuntu desktop computer with private address (192.168.1.36) 
In the pfSense configuration, packets with protocol 41 are forward to the Ubuntu desktop IP address.  A firewall rule has also been setup to allow any traffic from the PoP IPv4 address to any hosts within the 192.168.1.33/27 subnet.  
The Ubuntu box's time has been verified to have 0 sec differences using SixXS time check tool.  
Ubuntu box interface and routing table information
eth0      Link encap:Ethernet  HWaddr 00:1e:8c:30:b9:7c  
          inet addr:192.168.1.36  Bcast:192.168.1.63  Mask:255.255.255.224
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:20720 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20724 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:14835233 (14.8 MB)  TX bytes:3187315 (3.1 MB)
          Interrupt:17 
eth1      Link encap:Ethernet  HWaddr 00:1e:8c:30:ba:12  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:18 
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:53 errors:0 dropped:0 overruns:0 frame:0
          TX packets:53 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:5928 (5.9 KB)  TX bytes:5928 (5.9 KB)
sixxs     Link encap:IPv6-in-IPv4  
          inet6 addr: 2001:4978:f:224::2/64 Scope:Global
          inet6 addr: fe80::c0a8:124/64 Scope:Link
          UP POINTOPOINT RUNNING NOARP  MTU:1280  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:496 (496.0 B)
herman@herman-desktop:~/Desktop$ route -A inet6
Kernel IPv6 routing table
Destination                    Next Hop                   Flag Met Ref Use If
2001:4978:f:224::/64           ::                         Un   256 0     1 sixxs
fe80::/64                      ::                         Un   256 0     0 sixxs
::/0                           2001:4978:f:224::1         UG   1024 0     5 sixxs
::/0                           ::                         !n   -1  1   187 lo
::1/128                        ::                         Un   0   1    12 lo
2001:4978:f:224::2/128         ::                         Un   0   1     0 lo
fe80::c0a8:124/128             ::                         Un   0   1     0 lo
ff00::/8                       ::                         U    256 0     0 sixxs
::/0                           ::                         !n   -1  1   187 lo
On the pfSense box, I used the following tcpdump to monitor the interface in which the 6in4 tunnel runs.  Note:  I used the command "ping6 ipv6.google.com" on the Ubuntu desktop to trigger the following listings.  
herman@herman-desktop:~/Desktop$ ping6 ipv6.google.com
PING ipv6.google.com(2001:4860:0:2001::68) 56 data bytes
tcpdump -i rl0 | grep -i "sixxs"
17:25:37.385030 IP c-71-63-138-218.hsd1.mn.comcast.net > sixxs.cx01.chi.bb.your.org: IP6 cl-549.chi-02.us.sixxs.net > 2001:4860:0:2001::68: ICMP6, echo request, seq 42, length 64
17:25:38.384928 IP c-71-63-138-218.hsd1.mn.comcast.net > sixxs.cx01.chi.bb.your.org: IP6 cl-549.chi-02.us.sixxs.net > 2001:4860:0:2001::68: ICMP6, echo request, seq 43, length 64
17:25:39.384956 IP c-71-63-138-218.hsd1.mn.comcast.net > sixxs.cx01.chi.bb.your.org: IP6 cl-549.chi-02.us.sixxs.net > 2001:4860:0:2001::68: ICMP6, echo request, seq 44, length 64
17:25:40.384959 IP c-71-63-138-218.hsd1.mn.comcast.net > sixxs.cx01.chi.bb.your.org: IP6 cl-549.chi-02.us.sixxs.net > 2001:4860:0:2001::68: ICMP6, echo request, seq 45, length 64
AICCU connectivity test results
Tunnel Information for T18452:
POP Id      : uschi02
IPv6 Local  : 2001:4978:f:224::2/64
IPv6 Remote : 2001:4978:f:224::1/64
Tunnel Type : 6in4-heartbeat
Adminstate  : enabled
Userstate   : enabled
#######
####### AICCU Quick Connectivity Test
#######
####### [1/8] Ping the IPv4 Local/Your Outer Endpoint (192.168.1.36)
### This should return so called 'echo replies'
### If it doesn't then check your firewall settings
### Your local endpoint should always be pingable
### It could also indicate problems with your IPv4 stack
PING 192.168.1.36 (192.168.1.36) 56(84) bytes of data.
64 bytes from 192.168.1.36: icmp_seq=1 ttl=64 time=0.027 ms
64 bytes from 192.168.1.36: icmp_seq=2 ttl=64 time=0.024 ms
64 bytes from 192.168.1.36: icmp_seq=3 ttl=64 time=0.021 ms
--- 192.168.1.36 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.021/0.024/0.027/0.002 ms
######
####### [2/8] Ping the IPv4 Remote/PoP Outer Endpoint (216.14.98.22)
### These pings should reach the PoP and come back to you
### In case there are problems along the route between your
### host and the PoP this could not return replies
### Check your firewall settings if problems occur
PING 216.14.98.22 (216.14.98.22) 56(84) bytes of data.
64 bytes from 216.14.98.22: icmp_seq=1 ttl=53 time=75.0 ms
64 bytes from 216.14.98.22: icmp_seq=2 ttl=53 time=73.8 ms
64 bytes from 216.14.98.22: icmp_seq=3 ttl=53 time=74.3 ms
--- 216.14.98.22 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2008ms
rtt min/avg/max/mdev = 73.824/74.398/75.001/0.480 ms
######
####### [3/8] Traceroute to the PoP (216.14.98.22) over IPv4
### This traceroute should reach the PoP
### In case this traceroute fails then you have no connectivity
### to the PoP and this is most probably the problem
traceroute to 216.14.98.22 (216.14.98.22), 30 hops max, 40 byte packets
 1  192.168.1.33 (192.168.1.33)  0.414 ms  0.526 ms  0.631 ms
 2  73.115.174.1 (73.115.174.1)  10.974 ms  10.969 ms  10.957 ms
 3  ge-9-1-ur02.hamlake.mn.minn.comcast.net (68.85.165.229)  11.317 ms  11.305 ms  11.413 ms
 4  te-2-2-ar02.roseville.mn.minn.comcast.net (68.87.174.69)  12.744 ms  12.732 ms  12.822 ms
 5  te-8-3-ar02.roseville.mn.minn.comcast.net (68.87.174.6)  12.808 ms  12.891 ms  13.005 ms
 6  te-0-0-0-4-cr01.omaha.ne.ibone.comcast.net (68.86.91.149)  30.962 ms  27.607 ms  28.940 ms
 7  pos-0-12-0-0-cr01.sanjose.ca.ibone.comcast.net (68.86.85.122)  75.718 ms  75.721 ms  75.711 ms
 8  Tenge13-3.br02.sjo01.pccwbtn.net (63.218.179.25)  229.986 ms  229.980 ms  229.971 ms
 9  your.org.ge2-5.br02.chc01.pccwbtn.net (63.218.5.38)  78.169 ms  78.164 ms  78.286 ms
10  sixxs.cx01.chi.bb.your.org (216.14.98.22)  77.990 ms  77.978 ms  78.071 ms
######
###### [4/8] Checking if we can ping IPv6 localhost (::1)
### This confirms if your IPv6 is working
### If ::1 doesn't reply then something is wrong with your IPv6 stack
PING ::1(::1) 56 data bytes
64 bytes from ::1: icmp_seq=1 ttl=64 time=0.030 ms
64 bytes from ::1: icmp_seq=2 ttl=64 time=0.037 ms
64 bytes from ::1: icmp_seq=3 ttl=64 time=0.038 ms
--- ::1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.030/0.035/0.038/0.003 ms
######
###### [5/8] Ping the IPv6 Local/Your Inner Tunnel Endpoint (2001:4978:f:224::2)
### This confirms that your tunnel is configured
### If it doesn't reply then check your interface and routing tables
PING 2001:4978:f:224::2(2001:4978:f:224::2) 56 data bytes
64 bytes from 2001:4978:f:224::2: icmp_seq=1 ttl=64 time=0.024 ms
64 bytes from 2001:4978:f:224::2: icmp_seq=2 ttl=64 time=0.042 ms
64 bytes from 2001:4978:f:224::2: icmp_seq=3 ttl=64 time=0.039 ms
--- 2001:4978:f:224::2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.024/0.035/0.042/0.007 ms
######
###### [6/8] Ping the IPv6 Remote/PoP Inner Tunnel Endpoint (2001:4978:f:224::1)
### This confirms the reachability of the other side of the tunnel
### If it doesn't reply then check your interface and routing tables
### Don't forget to check your firewall of course
### If the previous test was succesful then this could be both
### a firewalling and a routing/interface problem
PING 2001:4978:f:224::1(2001:4978:f:224::1) 56 data bytes
--- 2001:4978:f:224::1 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 1999ms
######
###### [7/8] Traceroute6 to the central SixXS machine (noc.sixxs.net)
### This confirms that you can reach the central machine of SixXS
### If that one is reachable you should be able to reach most IPv6 destinations
### Also check http://www.sixxs.net/ipv6calc/ which should show an IPv6 connection
### If your browser supports IPv6 and uses it of course.
traceroute to noc.sixxs.net (2001:838:1:1:210:dcff:fe20:7c7c) from 2001:4978:f:224::2, 30 hops max, 16 byte packets
 1  * * *
 2  * * *
 3  * *
######
###### [8/8] Traceroute6 to (www.kame.net)
### This confirms that you can reach a Japanese IPv6 destination
### If that one is reachable you should be able to reach most IPv6 destinations
### You should also check http://www.kame.net which should display
### a animated kame (turtle), of course only when your browser supports and uses IPv6
traceroute to www.kame.net (2001:200:0:8002:203:47ff:fea5:3085) from 2001:4978:f:224::2, 30 hops max, 16 byte packets
 1  * * *
 2  * * *
 3  *
######
###### ACCU Quick Connectivity Test (done)
 
State change: user    
The state of this ticket has been changed to user
 
No responses from PoP endpoint My 6in4 tunnel stopped working around 11am CST on 12/11/2008.That matches about the time that the PoP received the last heartbeat from you: 2008-12-11 16:05:21 (~32 hours ago) In the pfSense configuration, packets with protocol 41 are forward to the Ubuntu desktop IP address. A firewall rule has also been setup to allow any traffic from the PoP IPv4 address to any hosts within the 192.168.1.33/27 subnet.But are you blocking the outbound-only heartbeat packets?
Clearly the PoP hasn't received a proper one from your host since the above timestamp. No heartbeat -> no tunnel activation. 
No responses from PoP endpoint ![[us]](/s/countries/us.gif) Shadow Hawkins on Saturday, 13 December 2008 01:28:32 In the pfSense configuration, packets with protocol 41 are forward to the Ubuntu desktop IP address. A firewall rule has also been setup to allow any traffic from the PoP IPv4 address to any hosts within the 192.168.1.33/27 subnet.The firewall has been setup to allow any (i.e. tcp/udp) traffic from the subnet to the PoP (216.14.98.22) as well as any other external host.  The firewall has no rules that block udp 3740.  Here is the state table entry from my pfSense box.  
udp  192.168.1.36:35865 -> 71.63.138.218:54950 -> 216.14.98.22:3740  SINGLE:NO_TRAFFIC 
 |