Gnax network outage [Merged]

Portal Home > Knowledgebase > Articles Database > Gnax network outage [Merged]

Posted by BurtonHost, 03-25-2004, 07:55 AM
There are some problems accessing the network at the moment. Looks to be the abovenet link that is the problem. From the UK, times out here: 4 40 ms 29 ms 25 ms 251.ge6-0.mpr1.lhr1.uk.above.net [213.161.78.85] 5 23 ms 23 ms 25 ms so-4-1-0.cr1.lhr3.uk.above.net [208.184.231.174] 6 * * * Request timed out. 7 * * * Request timed out. -- Earlier before it went down totally, the result was: 4 24 ms 23 ms 22 ms 251.ge6-0.mpr1.lhr1.uk.above.net [213.161.78.85] 5 23 ms 25 ms 29 ms so-4-1-0.cr1.lhr3.uk.above.net [208.184.231.174] 6 94 ms 100 ms 98 ms so-7-0-0.cr1.dca2.us.above.net [64.125.31.186] 7 * * * Request timed out. 8 * * * Request timed out. 9 * * * Request timed out. -- Looks like an abovenet network issue.
Posted by hostbox, 03-25-2004, 07:58 AM
Hi, Maybe is just me but my Server and the Gnax Site are down for me The strangest thing is that I have tree servers there and the only one who is not working is my main one, and I can't log into the Support or APC Remote Reboot. Anyone has this problem? Mod, please move it for me if they're down Cheers
Posted by ViAdcK, 03-25-2004, 08:00 AM
Down for me also.. I've tried traces from theplanet, nac etc. and they all die.
Posted by hostbox, 03-25-2004, 08:03 AM
Damm, Now I lost my 100% Uptime
Posted by {NIRMANI}, 03-25-2004, 08:04 AM
Yes - It's down
Posted by hostbox, 03-25-2004, 08:06 AM
Anyone else is down with the Gnax Site? This server is up: 209.51.155.234 These Servers are down: 65.254.44.138 www.Mtxis.net
Posted by IceCell, 03-25-2004, 08:06 AM
confirmed from Central CA. Gnax is down.
Posted by Apoc, 03-25-2004, 08:07 AM
Matthew, They announced on their forum that there would be some core router replacement, however they also told me that this wouldn't cause any outages. I'm from the netherlands and I have the same problem. The network seems to be unreachable from Europe. Also from the US it appears to be down, I can't ping the network from any other box in the US. I also wonder what's going on, I hope Jeff can respond. I'm also having some issues when trying to call them, I can't seem to get though. The number I'm trying is 404.230.9150 (that's +1.404.230.9150 from Europe). It's odd, I hear someone talking to me but they can't seem to hear me..
Posted by Boost, 03-25-2004, 08:09 AM
now this is a real joke, what is going on all around we fix one place we get the other. down here too
Posted by Apoc, 03-25-2004, 08:09 AM
If anyone can get through please let me know if they have any news.
Posted by BurtonHost, 03-25-2004, 08:10 AM
I've been speaking on the phone numerous times this morning, no problems at all. Hoping they can either have abovenet fix the issue, or re-route traffic shortly.
Posted by freakysid, 03-25-2004, 08:12 AM
There is a forum for this: http://www.webhostingtalk.com/forumd...php?forumid=59
Posted by inteltechs, 03-25-2004, 08:12 AM
they are working on it.
Posted by David Anthon, 03-25-2004, 08:13 AM
My server is down too ... Please any body can call them Regards
Posted by Russ Foster, 03-25-2004, 08:13 AM
Getting routing from the UK here via telia -> national.net Rus
Posted by David Anthon, 03-25-2004, 08:15 AM
Tranxactglobal.com is down too the datacenter is DOWN!!!! Regards
Posted by Apoc, 03-25-2004, 08:15 AM
Last time I asked them to re-route something they were taking care of it rightaway, currently I just can't contact them though.
Posted by BurtonHost, 03-25-2004, 08:15 AM
One of our traces earlier this morning: 4 24 ms 23 ms 22 ms 251.ge6-0.mpr1.lhr1.uk.above.net [213.161.78.85] 5 23 ms 25 ms 29 ms so-4-1-0.cr1.lhr3.uk.above.net [208.184.231.174] 6 94 ms 100 ms 98 ms so-7-0-0.cr1.dca2.us.above.net [64.125.31.186] 7 * * * Request timed out. 8 * * * Request timed out. 9 * * * Request timed out. 10 * 127 ms 123 ms pos11-1.atl-e00.wvfiber.net [63.223.2.250] 11 155 ms 106 ms 106 ms 64.154.39.54 12 108 ms 109 ms 105 ms or1.atl1-s0.national-net.com [66.115.128.65] 13 129 ms 133 ms 135 ms gige.globalxng-telia-aleron-ibis7.gnax.net [209. 51.130.33] 14 150 ms 125 ms 127 ms l3-atl-3.gnax.net [209.51.131.78]
Posted by hostbox, 03-25-2004, 08:18 AM
I just though it was only me BTW; I asked to be moved if it's down for everyone, I'm not that stupid
Posted by Apoc, 03-25-2004, 08:19 AM
On what?
Posted by BurtonHost, 03-25-2004, 08:21 AM
the problem?
Posted by inteltechs, 03-25-2004, 08:21 AM
I just called them, one of the tech said that they noticed the problem and working on it.
Posted by Apoc, 03-25-2004, 08:21 AM
No kidding sherlock. I mean, if they are working on it, they have to know what the problem is. So what is the problem?
Posted by BurtonHost, 03-25-2004, 08:23 AM
A network problem? Rus has confirmed he can get through on a different route (via telia / n-net) and so far traces we have done suggest it is the abovenet link that has the problem.. timing out before it even reaches the network.
Posted by Apoc, 03-25-2004, 08:25 AM
Hehe yeah yeah I understand that, but I meant more like, what's going on with Abovenet then (asin, what did the datacenter say exactly)? There would be a router replacement (according to GNAX) this night, did anything go wrong there?
Posted by Chuggles, 03-25-2004, 08:27 AM
It's back up.
Posted by inteltechs, 03-25-2004, 08:27 AM
it's back online again
Posted by BurtonHost, 03-25-2004, 08:27 AM
Not sure.. but our access has just been restored. 5 23 ms 21 ms 21 ms so-4-1-0.cr1.lhr3.uk.above.net [208.184.231.174] 6 94 ms 96 ms 94 ms so-7-0-0.cr1.dca2.us.above.net [64.125.31.186] 7 110 ms 107 ms 113 ms pos0-0.pr1.atl4.us.above.net [64.125.28.230] 8 125 ms 121 ms 123 ms g102-abovenet-atl.bb.ibis7.net [209.249.11.102] 9 127 ms 133 ms 127 ms ge1-25-6.atl-c00.wvfiber.net [63.223.0.161] 10 159 ms 121 ms 121 ms pos11-1.atl-e00.wvfiber.net [63.223.2.250] 11 105 ms 109 ms 106 ms 64.154.39.54 12 107 ms 113 ms 106 ms or1.atl1-s0.national-net.com [66.115.128.65] 13 125 ms 127 ms 131 ms gige.globalxng-telia-aleron-ibis7.gnax.net [209. 51.130.33] 14 138 ms 127 ms 129 ms l3-atl-3.gnax.net [209.51.131.78]
Posted by David Anthon, 03-25-2004, 08:29 AM
It's backup .... Happy All
Posted by -Edward-, 03-25-2004, 08:31 AM
Think it was a world routing issue
Posted by rodeto, 03-25-2004, 08:31 AM
from the netherlands: 3 ge4-2.mpr2.ams1.nl.above.net (195.69.144.122) [AS 1200] 4 msec 0 msec 4 msec 4 pos2-0.cr2.ams2.nl.above.net (208.184.231.254) [AS 6461] 0 msec 4 msec 0 msec 5 so-4-0-0.cr2.lhr3.uk.above.net (64.125.31.157) [AS 6461] 8 msec 8 msec 12 msec 6 so-0-0-0.cr1.lhr3.uk.above.net (208.184.231.145) [AS 6461] 8 msec 8 msec 8 msec 7 so-7-0-0.cr1.dca2.us.above.net (64.125.31.186) [AS 6461] 84 msec 80 msec 84 msec 8 pos11-0.pr1.atl4.us.above.net (64.125.29.37) [AS 6461] 92 msec 92 msec 96 msec 9 64.124.17.5.gnax.net (64.124.17.5) [AS 6461] 92 msec 108 msec * 10 L3-ATL-3.gnax.net (209.51.131.78) [AS 16626] 92 msec 92 msec 92 msec
Posted by hostbox, 03-25-2004, 08:31 AM
Thank God! Anyone knows what happened ?
Posted by sailor, 03-25-2004, 08:55 AM
as we announced on our forum - abovenet announced to us that they were going to be replacing their core router and it would involve a 30 minute outage. this shuold be no big deal - other than some customers would be affected duing this time as the global route registrys update since it can take 30 minutes to get route updates to your provider - so even though it appears we are down - we are not - your provider just has not gotten route updates yet that the path is no longer accessible inbound. This was the majority of the issue for you guys. then the abovenet router came back - and one of the hsrp peers on their side only gave us parital tables - so this corrupted the session. I cleared it and brought the session down and back up again. then we got full routes from them finally. sorry if anyone was affected - we did however give everyone plenty of announcement on the maintenace this time from our upstream this time as you all have requested. the main issue is that a lot of people come in on abovnet since they have strong peers. I am adding 3 more gig interfaces that we have just completed contracts on that should heavily bolster our inbound routes and cut these issues down even further. they are - Verio , Telia (we are getting them direct soon) and PCCWBTN. we are not dropping any connections just adding. thanks again for your continued business.
Posted by Boost, 03-25-2004, 09:03 AM
sailor keep it up you are still the best, the golden DC
Posted by sailor, 03-25-2004, 09:11 AM
appears there were more issues on abovnet created as a result of their upgrades - jsut got this in the email. they were giving us errors also - it caused my internal routing to be somewaht freaky too. This was causing our routers to think they were still there anb to try to send them traffic - but they were munching it in both directions. I have completely cleared all sessions now - including internal sessions for good measure. Dear Valued Customer, mpr2.atl4 began experiencing memory problems this morning. Some bgp destinations were unreachable via atl4. We have just performed an emergency reboot and expect the router to be back in a few minutes. If you have any questions concerning this email, please contact our 24/7 AboveNet Client Services Center(CSC) at (877) ABOVENET (877-226-8363) or locally at(408) 350-6673 or internationally at 001 (408)350-6673. Thank you, AboveNet Client Services
Posted by Andrew, 03-25-2004, 09:22 AM
Jeff, that's not fair and you know it's not. We're supposed to check your forums daily to make sure there isn't going to be some scheduled outage? This isn't the way you've notified us in the past and if you would like to start doing that, could you please create an announcements forum that we could subscribe to? The way it stands now, we'd have to subscribe to the whole forum to be notified of anything like this. You guys are the best there is, as far as I'm concerned, but I would really like to know about scheduled downtime.
Posted by sailor, 03-25-2004, 09:33 AM
I could email off the forum. I get some complaints from time to time but - I guess that is the best bet at this point. the email that says - hey you need to follow this link for important in formation. I will do that. thanks!
Posted by Andrew, 03-25-2004, 09:36 AM
Cool thanks! I know you took heat for one of the other things, but that was rather silly. I like the notifications by email. The announcement forum by subscription would probably be a solution everyone would be happy with. That way people could subscribe if they want to know and remain blissful if they don't.
Posted by Aussie Bob, 03-25-2004, 10:40 AM
Yep, just subscribe to their announcement forum, with an off-network email address, and you'll be emailed when they make a post. Just hope they have that forum off their network!