|
Ticket ID: SIXXS #13259157 Ticket Status: Resolved PoP: ausyd01 - OCCAID Inc. (Sydney)
Routing issue to Softlayer
![[au]](/s/countries/au.gif) Shadow Hawkins on Monday, 02 March 2015 06:42:19
Hi!
It appears that routing to Softlayer may be broken.
One such address is 2607:f0d0:1901:2::c
Another is softlayer.com itself on 2607:f0d0:1000:11:66:228:119:72
Here's a traceroute:
================================================================================================
traceroute to 2607:f0d0:1000:11:66:228:119:72 (2607:f0d0:1000:11:66:228:119:72), 30 hops max, 80 byte packets
1 gw-83.syd-01.au.sixxs.net (2001:4830:1200:52::1) 119.745 ms 119.618 ms 120.632 ms
2 ausyd01.sixxs.net (2001:4830:ac:1::2) 120.578 ms 121.452 ms 119.248 ms
3 sixxs.sydn01.occaid.net (2001:4830:ac:1::1) 119.197 ms 119.154 ms 121.236 ms
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * * FAIL!!
================================================================================================
Here's the general info:
================================================================================================
OS: Linux gateway.home.local 3.13.0-45-generic #74-Ubuntu SMP Tue Jan 13 19:36:28 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
================================================================================================
================================================================================================
Interface table:
================================================================================================
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:04:a7:0c:17:8d brd ff:ff:ff:ff:ff:ff
inet6 fe80::204:a7ff:fe0c:178d/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:04:a7:0c:17:8e brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc cbq state UP group default qlen 1000
link/ether 00:04:a7:0c:17:8f brd ff:ff:ff:ff:ff:ff
inet 10.0.0.253/24 brd 10.0.0.255 scope global eth2
valid_lft forever preferred_lft forever
inet 10.8.255.1/24 brd 10.8.255.255 scope global eth2:1
valid_lft forever preferred_lft forever
inet6 2001:4830:1200:8052::1/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::204:a7ff:fe0c:178f/64 scope link
valid_lft forever preferred_lft forever
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:04:a7:0c:17:90 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:04:a7:0c:17:91 brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:04:a7:0c:17:92 brd ff:ff:ff:ff:ff:ff
8: eth6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:04:a7:0c:17:93 brd ff:ff:ff:ff:ff:ff
9: eth7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:04:a7:0c:17:94 brd ff:ff:ff:ff:ff:ff
10: eth2.10@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc cbq state UP group default
link/ether 00:04:a7:0c:17:8f brd ff:ff:ff:ff:ff:ff
inet 10.0.255.253/24 brd 10.0.255.255 scope global eth2.10
valid_lft forever preferred_lft forever
inet6 fe80::204:a7ff:fe0c:178f/64 scope link
valid_lft forever preferred_lft forever
11: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default
link/sit 0.0.0.0 brd 0.0.0.0
12: sit255@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1280 qdisc noqueue state UNKNOWN group default
link/sit 0.0.0.0 peer 66.186.206.66
inet6 2001:4830:1200:52::2/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::a00:fffd/64 scope link
valid_lft forever preferred_lft forever
inet6 fe80::a08:ff01/64 scope link
valid_lft forever preferred_lft forever
inet6 fe80::a00:fd/64 scope link
valid_lft forever preferred_lft forever
13: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 100
link/none
inet 10.8.222.40 peer 10.8.222.41/32 scope global tun0
valid_lft forever preferred_lft forever
14: ppp0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc htb state UNKNOWN group default qlen 3
link/ppp
inet 202.89.168.162 peer 203.161.65.253/32 scope global ppp0
valid_lft forever preferred_lft forever
================================================================================================
================================================================================================
Routing table:
================================================================================================
default dev ppp0 scope link
10.0.0.0/24 dev eth2 proto kernel scope link src 10.0.0.253
10.0.255.0/24 dev eth2.10 proto kernel scope link src 10.0.255.253
10.8.222.41 dev tun0 proto kernel scope link src 10.8.222.40
10.8.255.0/24 dev eth2 proto kernel scope link src 10.8.255.1
172.20.184.0/24 via 10.8.222.41 dev tun0
203.161.65.253 dev ppp0 proto kernel scope link src 202.89.168.162
2001:4830:1200:52::1 dev sit255 metric 1024
2001:4830:1200:52::/64 dev sit255 proto kernel metric 256
2001:4830:1200:8052::/64 dev eth2 proto kernel metric 256
fe80::/64 dev sit255 proto kernel metric 256
fe80::/64 dev eth0 proto kernel metric 256
fe80::/64 dev eth2 proto kernel metric 256
fe80::/64 dev eth2.10 proto kernel metric 256
default via 2001:4830:1200:52::1 dev sit255 metric 1024
================================================================================================
================================================================================================
Traceroute to POP v4:
================================================================================================
traceroute to 66.186.206.66 (66.186.206.66), 30 hops max, 60 byte packets
1 Lo0.amnet.l1-45sgt-se800-01.wa.amnet.net.au (203.161.65.253) 29.075 ms 29.923 ms 31.589 ms
2 vl70.ten7-3.cr01.wa.amnet.net.au (203.161.65.41) 31.474 ms 33.098 ms 33.378 ms
3 te3-2.br02.wa.amcom.net.au (203.161.65.65) 33.335 ms 33.250 ms 33.197 ms
4 vocus.ix.waia.asn.au (198.32.212.55) 77.748 ms 78.184 ms 78.856 ms
5 ge-0-0-0-18.cor01.per02.wa.VOCUS.net.au (114.31.206.82) 80.521 ms 80.739 ms 81.680 ms
6 ten-0-2-0-7.cor03.syd03.nsw.VOCUS.net.au (114.31.206.126) 82.576 ms 78.941 ms 71.566 ms
7 ten-0-1-0.bdr02.syd03.nsw.VOCUS.net.au (114.31.192.61) 71.489 ms 71.439 ms 71.398 ms
8 as30071.cust.bdr02.syd03.nsw.VOCUS.net.au (114.31.200.93) 72.445 ms 73.138 ms 73.818 ms
9 66.186.206.66 (66.186.206.66) 75.239 ms 76.890 ms 77.049 ms
================================================================================================
================================================================================================
Traceroute to POP v6:
================================================================================================
traceroute to 2001:4830:ac:1::2 (2001:4830:ac:1::2) from 2001:4830:1200:52::2, 30 hops max, 24 byte packets
1 gw-83.syd-01.au.sixxs.net (2001:4830:1200:52::1) 86.629 ms * 69.806 ms
2 ausyd01.sixxs.net (2001:4830:ac:1::2) 69.649 ms 69.933 ms 69.953 ms
State change: remoteproblem
The state of this ticket has been changed to remoteproblem
Routing issue to Softlayer
Without an inbound traceroute, very little what can be said about what goes wrong.
Every few runs it will show some hops inside the softlayer network though:
traceroute to 2607:f0d0:1000:11:66:228:119:72 (2607:f0d0:1000:11:66:228:119:72), 30 hops max, 80 byte packets
1 sixxs.sydn01.occaid.net (2001:4830:ac:1::1) 0.238 ms 0.192 ms 0.244 ms
2 * * *
3 * * *
4 * * *
5 * * *
6 * * *
7 po1.bbr01.cs01.lax01.networklayer.com (2607:f0d0:2:2::26) 164.413 ms * *
8 * * *
...
Which makes it look like packetloss.
The Melbourne node has similar issues:
traceroute to 2607:f0d0:1000:11:66:228:119:72 (2607:f0d0:1000:11:66:228:119:72), 30 hops max, 80 byte packets
1 sixxs.bris01.occaid.net (2001:4830:ab:7::1) 1.512 ms 1.454 ms 1.429 ms
2 bbr01-g0-1.bris01.occaid.net (2001:4830:fd:641::1) 1.763 ms 1.746 ms 1.718 ms
3 bbr01-v264.sydn01.occaid.net (2001:4830:fd:640::2) 17.269 ms 17.271 ms 17.253 ms
4 ibr01-g0-1.sydn01.occaid.net (2001:4830:fd:644::2) 17.833 ms 17.818 ms 17.799 ms
5 * 36351.syd.equinix.com (2001:de8:6::3:6351:1) 130.283 ms *
6 * * *
7 2607.f0d0.0002.0002.0000.0000.0000.0147-static.v6reverse.softlayer.com (2607:f0d0:2:2::147) 148.176 ms * *
8 * * *
9 * * *
from eg usqas01, also OCCAID, it looks like:
traceroute to 2607:f0d0:1000:11:66:228:119:72 (2607:f0d0:1000:11:66:228:119:72), 30 hops max, 80 byte packets
1 iad0-b0-ge2.hotnic.net (2001:4810:0:100::1) 0.629 ms 0.616 ms 0.698 ms
2 * * *
3 te1-2.cer01.wdc01.washingtondc-datacenter.com (2001:504:0:2:0:3:6351:1) 1.972 ms 2.792 ms 2.789 ms
4 po3.bbr02.tl01.atl01.networklayer.com (2607:f0d0:2:2::19) 14.131 ms 14.148 ms 14.134 ms
5 po7.bbr02.eq01.dal01.networklayer.com (2607:f0d0:2:2::6) 33.749 ms 34.809 ms 33.721 ms
6 2607.f0d0.0002.0002.0000.0000.0000.0087-static.v6reverse.softlayer.com (2607:f0d0:2:2::87) 33.676 ms 33.120 ms po6.dar02.sr01.dal01.networklayer.com (2607:f0d0:2:2::51) 33.850 ms
7 po2.slr01.sr01.dal01.networklayer.com (2607:f0d0:1000:1::7:42) 33.080 ms po1.slr01.sr01.dal01.networklayer.com (2607:f0d0:1000:1::7:36) 34.053 ms po2.slr01.sr01.dal01.networklayer.com (2607:f0d0:1000:1::7:42) 32.776 ms
8 softlayer.com (2607:f0d0:1000:11:66:228:119:72) 32.400 ms 32.509 ms 32.467 ms
9 * * *
10 * * *
Note that the traceroute does not 'stop' at the IP address we target there... likely as it is responding with hoplimits instead of port unreachable or something.
I'll forward to OCCAID, maybe they see something wrong somewhere.
Routing issue to Softlayer
Shadow Hawkins on Monday, 02 March 2015 08:48:17
Here's a working route:
HOST: werri.pps.com.au 0
1. 2406:a000::1 1
2. 2406:a000:ffff:ffff::1 2
3. ge-0-0-0-107.a06.sydnau01.au.ra.gin.ntt.net 3
4. xe-0-1-0.r05.sydnau01.au.bb.gin.ntt.net 3
5. as-0.r02.lsanca03.us.bb.gin.ntt.net 142
6. xe-0-0-7.r21.lsanca03.us.bb.gin.ntt.net 140
7. ae-12.r22.snjsca04.us.bb.gin.ntt.net 153
8. ae-23.r01.snjsca04.us.bb.gin.ntt.net 156
9. te1-5.bbr01.eq01.sjc01.networklayer.com 164
10. po1.bbr02.eq01.sjc01.networklayer.com 165
11. ??? 0
12. po1.bbr01.cs01.lax01.networklayer.com 165
13. po5.bbr01.eq01.dal01.networklayer.com 222
14. 2607.f0d0.0002.0002.0000.0000.0000.0085-static.v6reverse.softlayer.com 219
15. po2.slr01.sr01.dal01.networklayer.com 218
16. softlayer.com 221
Routing issue to Softlayer
Here's a working route:
There are "working" routes, but that does not help for debugging this issue, especially as this is a completely different route using NTT as transit.
Note also that the softlayer -> $dest traceroutes are missing, which would be helpful to get insight what that path is doing.
Routing issue to Softlayer
Shadow Hawkins on Monday, 02 March 2015 09:46:05
Hopefully OCCAID can shed some light on the problem then :)
Many things use softlayer - thanks for the promptness.
Routing issue to Softlayer
Shadow Hawkins on Tuesday, 03 March 2015 09:45:28
It would appear someone has fixed something :)
Tracing route to softlayer.com [2607:f0d0:1000:11:66:228:119:72]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms net-vl0-10-edge1-router.home.local [2001:4830:1200:8052::1]
2 212 ms 158 ms 158 ms gw-83.syd-01.au.sixxs.net [2001:4830:1200:52::1]
3 219 ms 220 ms 204 ms ausyd01.sixxs.net [2001:4830:ac:1::2]
4 70 ms 86 ms 113 ms sixxs.sydn01.occaid.net [2001:4830:ac:1::1]
5 195 ms 170 ms 170 ms 36351.syd.equinix.com [2001:de8:6::3:6351:1]
6 96 ms 130 ms 141 ms 2607.f0d0.0002.0002.0000.0000.0000.0143-static.v6reverse.softlayer.com [2607:f0d0:2:2::143]
7 140 ms 114 ms 113 ms 2607.f0d0.0002.0002.0000.0000.0000.0147-static.v6reverse.softlayer.com [2607:f0d0:2:2::147]
8 112 ms 112 ms 137 ms 2607.f0d0.0002.0002.0000.0000.0000.0144-static.v6reverse.softlayer.com [2607:f0d0:2:2::144]
9 314 ms 277 ms 246 ms 2607.f0d0.0002.0002.0000.0000.0000.014d-static.v6reverse.softlayer.com [2607:f0d0:2:2::14d]
10 257 ms 271 ms 334 ms po1.bbr01.cs01.lax01.networklayer.com [2607:f0d0:2:2::26]
11 364 ms 322 ms 309 ms po5.bbr01.eq01.dal01.networklayer.com [2607:f0d0:2:2::c]
12 433 ms 384 ms 338 ms po6.dar01.sr01.dal01.networklayer.com [2607:f0d0:2:2::4f]
13 317 ms 370 ms 405 ms po1.slr01.sr01.dal01.networklayer.com [2607:f0d0:1000:1::7:36]
14 344 ms 303 ms 307 ms softlayer.com [2607:f0d0:1000:11:66:228:119:72]
Trace complete.
Routing issue to Softlayer
OCCAID contacting Softlayer and the latter fixing their Sydney router fixed the issue.
State change: resolved
The state of this ticket has been changed to resolved
|
![[ch]](/s/countries/ch.gif)
on Monday, 02 March 2015 07:15:46
Posting is only allowed when you are