Both my tunnels to the sixes ipv6 network which go via glonb02 have dropped. NIC Handle MCJ9-SIXXS T39369gblon02 - Goscomb Technologiesheartbeat 2a01:348:6:39f::2 T22965gblon02 - Goscomb Technologies118.107.51.2402a01:348:6:25e::2 The two systems run different configurations. The first uses aiccu and heartbeat from a freebsd box and the second runs m0n0wall in a static configuration. The problem appears to be a failure of glonb02 to create tunnels and pass ip6 packets. According to aiccu when run on the heartbeat tunnel: # /usr/local/sbin/sixxs-aiccu test /usr/local/etc/aiccu.conf ifconfig: SIOCIFCREATE2: File exists route: writing to routing socket: File exists add net default: gateway 2a01:348:6:39f::1: route already in table ####### ####### AICCU Quick Connectivity Test ####### ####### [1/8] Ping the IPv4 Local/Your Outer Endpoint (110.232.242.134) ### This should return so called 'echo replies' ### If it doesn't then check your firewall settings ### Your local endpoint should always be pingable ### It could also indicate problems with your IPv4 stack PING 110.232.242.134 (110.232.242.134): 56 data bytes 64 bytes from 110.232.242.134: icmp_seq=0 ttl=64 time=1.231 ms 121 bytes from gblon02.sixxs.net (77.75.104.126): Destination Port Unreachable Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 7100 0d99 0 0000 27 11 6eab 110.232.242.134 77.75.104.126 UDP: from port 60934, to port 3740 (decimal) 64 bytes from 110.232.242.134: icmp_seq=1 ttl=64 time=1.087 ms 124 bytes from gblon02.sixxs.net (77.75.104.126): Destination Protocol Unreachable Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 7400 0dc5 0 0000 05 29 9064 110.232.242.134 77.75.104.126 64 bytes from 110.232.242.134: icmp_seq=2 ttl=64 time=0.856 ms --- 110.232.242.134 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.856/1.058/1.231/0.154 ms ###### Did this work? [Y/n] ####### [2/8] Ping the IPv4 Remote/PoP Outer Endpoint (77.75.104.126) ### These pings should reach the PoP and come back to you ### In case there are problems along the route between your ### host and the PoP this could not return replies ### Check your firewall settings if problems occur PING 77.75.104.126 (77.75.104.126): 56 data bytes 64 bytes from 77.75.104.126: icmp_seq=0 ttl=50 time=351.580 ms 64 bytes from 77.75.104.126: icmp_seq=1 ttl=50 time=350.114 ms 84 bytes from gblon02.sixxs.net (77.75.104.126): Destination Protocol Unreachable Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 4c00 0dfb 0 0000 05 29 9056 110.232.242.134 77.75.104.126 92 bytes from gblon02.sixxs.net (77.75.104.126): Destination Protocol Unreachable Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 5400 0dfc 0 0000 05 29 904d 110.232.242.134 77.75.104.126 64 bytes from 77.75.104.126: icmp_seq=2 ttl=50 time=358.087 ms --- 77.75.104.126 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 350.114/353.260/358.087/3.465 ms ###### Did this work? [Y/n] ####### [3/8] Traceroute to the PoP (77.75.104.126) over IPv4 ### This traceroute should reach the PoP ### In case this traceroute fails then you have no connectivity ### to the PoP and this is most probably the problem traceroute to 77.75.104.126 (77.75.104.126), 64 hops max, 40 byte packets 1 loop0.core2.smithst.dcsi.net.au (203.27.231.24) 38.093 ms 46.859 ms 39.998 ms 2 gi0-1-200.core1.smithst.dcsi.net.au (203.27.231.17) 25.852 ms 63.611 ms 38.729 ms 3 gi4-0-2.core1.portmel.aussiebb.net (202.142.143.145) 43.055 ms 53.602 ms 22.528 ms 4 gi5-1.bdr1.portmel.aussiebb.net (180.150.0.149) 37.276 ms 45.148 ms 24.841 ms 5 GigabitEthernet2-11.lon55.Melbourne.telstra.net (165.228.138.149) 44.830 ms * 48.035 ms 6 TenGigabitEthernet0-12-0-2.exi-core1.Melbourne.telstra.net (203.50.80.1) 57.939 ms 60.152 ms 49.691 ms 7 Bundle-Ether12.chw-core2.Sydney.telstra.net (203.50.11.74) 59.833 ms 71.081 ms 59.712 ms 8 Bundle-Ether4.ken-core4.Sydney.telstra.net (203.50.6.41) 64.893 ms 60.177 ms 59.808 ms 9 Bundle-Ether1.pad-gw2.Sydney.telstra.net (203.50.6.29) 59.827 ms 60.650 ms 59.741 ms 10 203.50.13.154 (203.50.13.154) 59.876 ms 61.811 ms 59.885 ms 11 i-0-0-2-0.tlot-core01.bx.reach.com (202.84.140.78) 209.802 ms 219.934 ms 209.811 ms 12 i-1-1.tlot03.bi.reach.com (202.84.251.234) 204.748 ms 211.311 ms 204.906 ms 13 xe-11-2-0.edge1.losangeles6.level3.net (4.68.62.9) 214.746 ms 220.607 ms 214.855 ms 14 vlan80.csw3.LosAngeles1.Level3.net (4.69.144.190) 214.664 ms vlan70.csw2.LosAngeles1.Level3.net (4.69.144.126) 215.152 ms vlan60.csw1.LosAngeles1.Level3.net (4.69.144.62) 224.985 ms 15 ae-83-83.ebr3.LosAngeles1.Level3.net (4.69.137.41) 229.857 ms ae-73-73.ebr3.LosAngeles1.Level3.net (4.69.137.37) 218.734 ms ae-63-63.ebr3.LosAngeles1.Level3.net (4.69.137.33) 240.022 ms 16 ae-3-3.ebr1.SanJose1.Level3.net (4.69.132.9) 249.478 ms 219.749 ms 215.462 ms 17 ae-2-2.ebr2.NewYork1.Level3.net (4.69.135.186) 284.734 ms 285.790 ms 284.861 ms 18 ae-62-62.csw1.NewYork1.Level3.net (4.69.148.34) 284.751 ms ae-82-82.csw3.NewYork1.Level3.net (4.69.148.42) 301.001 ms ae-92-92.csw4.NewYork1.Level3.net (4.69.148.46) 287.889 ms 19 ae-71-71.ebr1.NewYork1.Level3.net (4.69.134.69) 296.098 ms 286.034 ms ae-81-81.ebr1.NewYork1.Level3.net (4.69.134.73) 289.796 ms 20 ae-43-43.ebr2.London1.Level3.net (4.69.137.73) 370.853 ms ae-44-44.ebr2.London1.Level3.net (4.69.137.77) 370.964 ms ae-43-43.ebr2.London1.Level3.net (4.69.137.73) 371.546 ms 21 ae-59-224.csw2.London1.Level3.net (4.69.153.142) 348.485 ms ae-58-223.csw2.London1.Level3.net (4.69.153.138) 360.713 ms ae-56-221.csw2.London1.Level3.net (4.69.153.130) 350.772 ms 22 ae-2-52.edge4.London1.Level3.net (4.69.139.106) 360.885 ms * * 23 195.50.122.242 (195.50.122.242) 420.839 ms 352.752 ms 359.761 ms 24 xe-0-0-1.rt0.the.uk.goscomb.net (46.17.60.33) 359.934 ms 360.695 ms 364.638 ms 25 xe-0-1-0.cs0.thw.uk.goscomb.net (93.89.91.14) 349.807 ms 355.839 ms 346.844 ms 26 gblon02.sixxs.net (77.75.104.126) 367.799 ms 358.697 ms 372.674 ms ###### Did this work? [Y/n] ###### [4/8] Checking if we can ping IPv6 localhost (::1) ### This confirms if your IPv6 is working ### If ::1 doesn't reply then something is wrong with your IPv6 stack PING6(56=40+8+8 bytes) ::1 --> ::1 16 bytes from ::1: Echo Request 16 bytes from ::1, icmp_seq=0 hlim=64 dst=::1%4 time=3.631 ms 16 bytes from ::1: Echo Request 16 bytes from ::1, icmp_seq=1 hlim=64 dst=::1%4 time=2.448 ms 16 bytes from ::1: Echo Request 16 bytes from ::1, icmp_seq=2 hlim=64 dst=::1%4 time=1.289 ms --- ::1 ping6 statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 1.289/2.456/3.631/0.956 ms ###### Did this work? [Y/n] ###### [5/8] Ping the IPv6 Local/Your Inner Tunnel Endpoint (2a01:348:6:39f::2) ### This confirms that your tunnel is configured ### If it doesn't reply then check your interface and routing tables PING6(56=40+8+8 bytes) 2a01:348:6:39f::2 --> 2a01:348:6:39f::2 16 bytes from 2a01:348:6:39f::2: Echo Request 16 bytes from 2a01:348:6:39f::2, icmp_seq=0 hlim=64 dst=2a01:348:6:39f::2%7 time=3.533 ms 16 bytes from 2a01:348:6:39f::2: Echo Request 16 bytes from 2a01:348:6:39f::2, icmp_seq=1 hlim=64 dst=2a01:348:6:39f::2%7 time=2.322 ms 56 bytes from fe80::260:b0ff:feb5:a53a%pcn0: Router Advertisement 16 bytes from 2a01:348:6:39f::2: Echo Request 16 bytes from 2a01:348:6:39f::2, icmp_seq=2 hlim=64 dst=2a01:348:6:39f::2%7 time=2.277 ms --- 2a01:348:6:39f::2 ping6 statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 2.277/2.711/3.533/0.582 ms ###### Did this work? [Y/n] ###### [6/8] Ping the IPv6 Remote/PoP Inner Tunnel Endpoint (2a01:348:6:39f::1) ### This confirms the reachability of the other side of the tunnel ### If it doesn't reply then check your interface and routing tables ### Don't forget to check your firewall of course ### If the previous test was succesful then this could be both ### a firewalling and a routing/interface problem PING6(56=40+8+8 bytes) 2a01:348:6:39f::2 --> 2a01:348:6:39f::1 --- 2a01:348:6:39f::1 ping6 statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss ###### Did this work? [Y/n] HE looking glass shows glomb02 end of tunnel not present core1.fmt1.he.net> ping ipv6 2a01:348:6:39f::1 numeric count 5 Sending 5, 16-byte ICMPv6 Echo to 2a01:348:6:39f::1 timeout 5000 msec, Hop Limit 64 Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. No reply from remote host.# Entry cached for another 35 seconds. core1.fmt1.he.net> ping ipv6 2a01:348:6:25e::1 numeric count 5 Sending 5, 16-byte ICMPv6 Echo to 2a01:348:6:25e::1 timeout 5000 msec, Hop Limit 64 Request timed out. Request timed out. Request timed out. Request timed out. Request timed out. No reply from remote host.# Entry cached for another 34 seconds. Additional information HE Traceroute core1.fmt1.he.net> traceroute ipv6 2a01:348:6:39f::1 Tracing the route to IPv6 node 2a01:348:6:39f::1 from 1 to 30 hops 1 82 ms 1 ms <1 ms 2001:470:0:2f::2 2 55 ms 38 ms 27 ms 2001:470:0:1b4::2 3 54 ms 83 ms 65 ms 2001:470:0:1af::1 4 71 ms 74 ms 92 ms 2001:470:0:1c6::1 5 151 ms 152 ms 153 ms 2001:470:0:128::2 6 150 ms 142 ms 149 ms 2001:7f8:4::999e:1 7 150 ms 153 ms 157 ms 2a01:348::49:1:1 8 138 ms * * 2a01:348::27:1:1 9 149 ms 164 ms 162 ms 2a01:348:0:4:0:3:1:1 10 147 ms 155 ms 144 ms 2a01:348:0:4:0:3:0:1 11 201 ms 147 ms 149 ms 2a01:348:0:4:0:3:1:1 12 152 ms * 164 ms 2a01:348:0:4:0:3:0:1 13 161 ms 149 ms 161 ms 2a01:348:0:4:0:3:1:1 14 149 ms 139 ms 153 ms 2a01:348:0:4:0:3:0:1 15 145 ms 150 ms 150 ms 2a01:348:0:4:0:3:1:1 16 155 ms 182 ms 161 ms 2a01:348:0:4:0:3:0:1 17 149 ms 149 ms 149 ms 2a01:348:0:4:0:3:1:1 18 212 ms * 361 ms 2a01:348:0:4:0:3:0:1 19 157 ms 156 ms 149 ms 2a01:348:0:4:0:3:1:1 20 151 ms 152 ms * 2a01:348:0:4:0:3:0:1 21 214 ms 159 ms 150 ms 2a01:348:0:4:0:3:1:1 22 151 ms 148 ms * 2a01:348:0:4:0:3:0:1 23 148 ms 186 ms 143 ms 2a01:348:0:4:0:3:1:1 24 145 ms 151 ms 151 ms 2a01:348:0:4:0:3:0:1 25 146 ms 151 ms 153 ms 2a01:348:0:4:0:3:1:1 26 146 ms 149 ms 149 ms 2a01:348:0:4:0:3:0:1 27 149 ms 149 ms 150 ms 2a01:348:0:4:0:3:1:1 28 158 ms 141 ms 174 ms 2a01:348:0:4:0:3:0:1 29 167 ms 157 ms 149 ms 2a01:348:0:4:0:3:1:1 30 227 ms 172 ms 194 ms 2a01:348:0:4:0:3:0:1 IP: Errno(8) Trace Route Failed, no response from target node.# Entry cached for another 33 seconds. core1.fmt1.he.net> traceroute ipv6 2a01:348:6:25e::1 Tracing the route to IPv6 node 2a01:348:6:25e::1 from 1 to 30 hops 1 4 ms <1 ms <1 ms 2001:470:0:2f::2 2 27 ms 53 ms 40 ms 2001:470:0:1b4::2 3 65 ms 68 ms 126 ms 2001:470:0:1af::1 4 69 ms 70 ms 74 ms 2001:470:0:1c6::1 5 149 ms 149 ms 149 ms 2001:470:0:128::2 6 150 ms 200 ms 148 ms 2001:7f8:4::999e:1 7 156 ms 146 ms 149 ms 2a01:348::49:1:1 8 * 151 ms 163 ms 2a01:348::27:1:1 9 198 ms * 181 ms 2a01:348:0:4:0:3:1:1 10 147 ms 149 ms 148 ms 2a01:348:0:4:0:3:0:1 11 149 ms 156 ms 146 ms 2a01:348:0:4:0:3:1:1 12 147 ms * 150 ms 2a01:348:0:4:0:3:0:1 13 161 ms 138 ms 166 ms 2a01:348:0:4:0:3:1:1 14 158 ms 180 ms 160 ms 2a01:348:0:4:0:3:0:1 15 159 ms 149 ms * 2a01:348:0:4:0:3:1:1 16 152 ms 162 ms 173 ms 2a01:348:0:4:0:3:0:1 17 146 ms 142 ms 149 ms 2a01:348:0:4:0:3:1:1 18 150 ms 149 ms 149 ms 2a01:348:0:4:0:3:0:1 19 149 ms 150 ms 150 ms 2a01:348:0:4:0:3:1:1 20 * 151 ms 166 ms 2a01:348:0:4:0:3:0:1 21 158 ms 149 ms 152 ms 2a01:348:0:4:0:3:1:1 22 151 ms 145 ms * 2a01:348:0:4:0:3:0:1 23 * 179 ms 149 ms 2a01:348:0:4:0:3:1:1 24 149 ms 150 ms 149 ms 2a01:348:0:4:0:3:0:1 25 228 ms * 204 ms 2a01:348:0:4:0:3:1:1 26 160 ms 140 ms * 2a01:348:0:4:0:3:0:1 27 166 ms 161 ms 151 ms 2a01:348:0:4:0:3:1:1 28 153 ms 148 ms 146 ms 2a01:348:0:4:0:3:0:1 29 149 ms 149 ms 151 ms 2a01:348:0:4:0:3:1:1 30 148 ms 150 ms * 2a01:348:0:4:0:3:0:1 IP: Errno(8) Trace Route Failed, no response from target node.# Entry cached for another 28 seconds. Both trace routes appear to be looping at glomb02