Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > The Planet having major network connectivity issues? [ Thread Merged ]


The Planet having major network connectivity issues? [ Thread Merged ]




Posted by bubbafish, 05-03-2010, 01:22 AM
A lot of my Houston-based The Planet machines are having major network connectivity issues. Any one else?

Posted by Nick H, 05-03-2010, 01:29 AM
The outage affects the entire ThePlanet datacenter in Houston.

Posted by X-Max, 05-03-2010, 01:31 AM
count me in, i am too having tons of issues, lots of people not able to access server since last 30+ minutes.

#traceroute xxx.xxx.xxx
traceroute to xxx.xxx.xxx (174.123.xx.xx), 30 hops max, 38 byte packets
1 172.17.13.1 (172.17.13.1) 0.283 ms 0.195 ms 2.684 ms
2 74.200.242.62 (74.200.242.62) 1.596 ms 0.574 ms 0.398 ms
3 border5.tge8-1.layeredtech-6.chg.pnap.net (74.217.8.9) 0.566 ms 0.531 ms 0.546 ms
4 core5.te2-1-bbnet1.chg.pnap.net (64.94.32.14) 0.980 ms 0.550 ms 0.513 ms
5 12.94.24.9 (12.94.24.9) 0.966 ms 0.829 ms 0.807 ms
6 cr1.cgcil.ip.att.net (12.122.99.14) 1.988 ms cr2.cgcil.ip.att.net (12.122.132.246) 2.214 ms 2.145 ms
MPLS Label=17189 CoS=6 TTL=255 S=0
7 cgcil01jt.ip.att.net (12.122.84.49) 18.103 ms cgcil01jt.ip.att.net (12.122.86.41) 0.908 ms cgcil01jt.ip.att.net (12.122.84.49) 0.906 ms
8 192.205.34.78 (192.205.34.78) 1.088 ms 192.205.34.166 (192.205.34.166) 0.916 ms 192.205.34.78 (192.205.34.78) 0.926 ms
9 The-Planet-Dallas.TenGigabitEthernet6-2.ar5.DAL2.gblx.net (67.17.168.94) 49.638 ms 38.847 ms 44.095 ms
10 * * et1-1.ibr02.hstntx1.theplanet.com (70.87.253.54) 178.455 ms
11 * et5-4.ibr04.dllstx3.theplanet.com (70.87.253.53) 124.801 ms *
12 * et1-1.ibr02.hstntx1.theplanet.com (70.87.253.54) 162.330 ms *
13 * * *
14 * * *

Posted by X-Max, 05-03-2010, 01:33 AM
Nick, I am experiencing this for the first time in last 12 months, they are not unreliable.

Posted by X-Max, 05-03-2010, 01:34 AM
outage seems to be from only some places, because i am able to access server from other locations.

Posted by X-Max, 05-03-2010, 01:38 AM
Tracing route to xxxxx.xxxxx.com [174.123.xx.xx]
over a maximum of 30 hops:

1 3 ms 1 ms 2 ms homeportal [192.168.1.254]
2 18 ms 18 ms 17 ms dsl-servicio-l200.uninet.net.mx [200.38.193.226]

3 56 ms 53 ms 57 ms bb-la-grand-4-pos0-15-2-0.uninet.net.mx [201.125.8.29]
4 79 ms 58 ms 61 ms te-4-2.car2.Tustin1.Level3.net [4.53.178.25]
5 57 ms 58 ms 57 ms ae-2-4.bar2.Tustin1.Level3.net [4.69.132.226]
6 100 ms 92 ms 103 ms ae-10-10.ebr1.Dallas1.Level3.net [4.69.136.206]
7 88 ms 89 ms 89 ms ae-1-60.edge4.Dallas3.Level3.net [4.69.145.13]
8 92 ms 91 ms 89 ms THE-PLANET.edge4.Dallas3.Level3.net [4.59.32.30]

9 103 ms 198 ms 206 ms et1-1.ibr02.hstntx1.theplanet.com [70.87.253.54]
10 * 151 ms 198 ms et5-4.ibr04.dllstx3.theplanet.com [70.87.253.53]
11 * * * Request timed out.
12 353 ms * * et5-4.ibr04.dllstx3.theplanet.com [70.87.253.53]
13 * * * Request timed out.
14 * * * Request timed out.
15 * * * Request timed out.

Posted by Nick H, 05-03-2010, 01:43 AM
Quote:
Originally Posted by X-Max
Nick, I am experiencing this for the first time in last 12 months, they are not unreliable.
The place where I work has over 100 servers with ThePlanet and this is (at least) the 4th such outage we've had in the past 12 months.

Posted by Eleven2 Hosting, 05-03-2010, 01:52 AM
Possibly they had another explosion?

Posted by bubbafish, 05-03-2010, 01:54 AM
What is even more annoying is that The Planet's own support forums rarely acknowledge major network issues like this one. No posts this time again on their own forums.

Posted by Jedito, 05-03-2010, 01:56 AM
Nice, I was in the middle of the move of 300 accounts to their server :/

Posted by X-Max, 05-03-2010, 02:19 AM
it seems fine now

Posted by The Universes, 05-03-2010, 02:37 AM
Put in a support ticket with them and ask for a "RFQ". Lets us know what they say.

Posted by Nick H, 05-03-2010, 04:04 AM
Quote:
Originally Posted by The Universes
Put in a support ticket with them and ask for a "RFQ". Lets us know what they say.
You want an "RFO" (Reason For Outage not Request For Quote )

I should be hearing back from the CEO later today and will post here if I remember.

Posted by The Universes, 05-03-2010, 04:17 AM
Yes of course, RFO.

Posted by Todd Mitchell, 05-03-2010, 04:22 AM
We did experience a network issue this morning in Houston that affected both the H1 and H2 data centers. One of our four border routers in Houston experienced a routing issue that resulted in network connectivity issues for traffic in and out of Houston.

The network connectivity issues were sporadic as traffic is balanced between border routers. As we isolated the issue we removed from service the border router in question to fully restore network connectivity.

We are working with the border router vendor on a root cause analysis. In the meantime if your servers are still unavailable, please contact our support team.

Included below is our official statement regarding the service issue this evening.

Quote:
On May 2 at approximately 11:45 p.m. Central time, we experienced a network issue that affected connectivity within The Planet’s core network in Houston that may have prevented your servers from communicating to the Internet. Network connectivity service was fully restored in less than 85 minutes; however, your servers may have been impacted by this incident.

Please review your records to check for network connectivity loss during that timeframe. You may be entitled to receive a service credit based upon the terms of the Service Level Agreement (SLA) if your services were affected.

If your records indicate a loss of network connectivity, please contact The Planet Billing Care and we will review your account to validate you're eligibility for an SLA service credit. You may contact us through the Orbit ticketing system, Live Support chat, or by phone at 866-325-0045.

In our ongoing analysis of the occurrence, we determined that one of four border routers in Houston failed to properly maintain standard routing protocols. Traffic destined for this router may have experienced a failure to communicate within The Planet’s core network and to several Internet transit providers directly connected to this border router. We are working through a root cause analysis and we’ve isolated this router from our network to prevent further issues. We have escalated the issue to our vendor for further analysis.

We apologize for any inconvenience, and as always, thank you for placing your trust in The Planet.

Sincerely,

The Planet Support

Posted by Michaels75, 05-03-2010, 05:08 AM
me too having the issue with there network,how long is it been since you guys are facing the problem? for me it is about 3 hours since i m facing the issue

Posted by Nick H, 05-03-2010, 05:12 AM
LOL - ThePlanet has some f***ing nerve let me tell you.

I put in a ticket to request an SLA Credit (which should be around 30% just in case anyone is curious) - and they expect me to provide a list of all the affected servers. Some nerve!

Not only do they screw something up and cause our servers to be down for 2.5 hours - increasing our workload - they expect us to spend even more time cleaning up their mess to get credit.

Posted by pms, 05-03-2010, 09:17 AM
Any one else still seeing issues? Cannot reach a server in 174.133.xxx block at the planet, plus forums and orbit are now inaccessible.

Posted by Subsim, 05-03-2010, 09:19 AM
Now able to connect to Planet CP or main website, and my sites are offline. Can connect to other websites. Anyone else seeing this?

Posted by huck, 05-03-2010, 09:19 AM
We have about 40 servers down. Most of them behind:
c5.fd.5746.static.theplanet.com (70.87.253.197)

Posted by Dant27, 05-03-2010, 09:19 AM
Quote:
Originally Posted by pms
Any one else still seeing issues? Cannot reach a server in 174.133.xxx block at the planet, plus forums and orbit are now inaccessible.
Yeah, it just went down again for us about 15 minutes ago.

Posted by IH-Antonio, 05-03-2010, 09:19 AM
No ping reply from one of my servers at ThePlanet. Trying to open a ticket but their site is really slow.

Posted by Sectorlink, 05-03-2010, 09:20 AM
This does appear to be an issue. I am not able to get there on this end as well.

Posted by pms, 05-03-2010, 09:20 AM
Do they have an off-site status page? Or is it just that twitter deal?

Their forums are on 70.87.6.126 and orbit is 70.87.6.24, so they should notice pretty quick.

Posted by csc2ya, 05-03-2010, 09:22 AM
Getting the same thing here. I'm not hosted by them, but I often visit a site that is hosted by them (b3ta), and that's gone down. Their main site is also not loading or responding to pings.

Posted by IH-Antonio, 05-03-2010, 09:22 AM
Btw, I had a report of 11m downtime this morning (like 8 hours ago), but the server was online and running, so it must have been the network (IP not reachable)

Posted by under_gravity, 05-03-2010, 09:24 AM
Lot of our servers went down now as well...
they were up 5 min ago..

Posted by ArabCoders, 05-03-2010, 09:27 AM
174.133.xx my servers are down as well

Posted by Max Renn, 05-03-2010, 09:28 AM
Yep, my server is offline too.

Posted by eth00, 05-03-2010, 09:29 AM
Yeah we are down again and this time it seems like it might be worse, hopefully not...

Posted by inspiron, 05-03-2010, 09:30 AM
How to contact the support as the site is not working?

Posted by ArabCoders, 05-03-2010, 09:31 AM
does anyone able to contact them to know what happened this time? 'x

Posted by nettigritty, 05-03-2010, 09:32 AM
174.123.x.x
66.98.x.x
67.15.x.x

All down. This is not good.

Posted by Von Vonder, 05-03-2010, 09:32 AM
Their phone number is just a busy signal.

Posted by ALMGHARI, 05-03-2010, 09:33 AM
http://theplanet.com/
https://orbit.theplanet.com/
DOWN !!
And also my server is DOWN

Posted by X-Max, 05-03-2010, 09:34 AM
down again all server. very bad. i can't even reach their own web site.

Posted by ArabCoders, 05-03-2010, 09:34 AM
i managed to get to online tech support and there is nobody to answer...

"Thank you for choosing The Planet, please stand by while we connect you to the next available agent…
We're sorry, we're currently not available to take your chat. Please try your chat again later."

Posted by belindaj, 05-03-2010, 09:34 AM
My server is in the Dallas hub - so my sites aren't actually down - for me anyway in the KC area on Comcast. But I have customers in Wichita - who are unable to connect or receive email at all from their sites which are in the Dallas server, even though I can get to them fine.

I'd put in a ticket - but I can't access Orbit or The Planet websites

My first thought when I received my first customer call this morning (and after I verified that I can get to EVERYTHING they can't get to) was "whew - glad it's NOT actually my server" - followed by "crap - I'm still probably going to get calls and sure hope the issue doesn't expand past the Houston dc".

Posted by tinoandrijic, 05-03-2010, 09:38 AM
you can access Orbit with:

https://70.87.6.24/Login.aspx?url=/S...AddTicket.aspx

Posted by Von Vonder, 05-03-2010, 09:40 AM
Got on orbit for a second and was able to load the backbone bandwidth graph. Doesn't look like anyone is escaping this in Houston at least.

Posted by ArabCoders, 05-03-2010, 09:41 AM
they need to update us with their progress so we can tell our customers what going on, this is bad....

Posted by inspiron, 05-03-2010, 09:41 AM
Yes, its down for me too. Ping request says,

Pinging theplanet.com [70.87.6.117] with 32 bytes of data:

Request timed out.
Request timed out.
Request timed out.

Posted by Von Vonder, 05-03-2010, 09:42 AM
All my servers just came back online.
Uptime is 290 days so this was a network issue only... 08:41:32 up 290 days, 17:49, 1 user, load average: 0.35, 0.09, 0.07

Posted by nettigritty, 05-03-2010, 09:43 AM
1 server back online. many still off..

Posted by dialsoft, 05-03-2010, 09:44 AM
Im still unable to reach any planet property I am aware of including main site/forums/all my servers and even their phone number is a fast busy.

Posted by nettigritty, 05-03-2010, 09:44 AM
Quote:
Originally Posted by Von Vonder
All my servers just came back online.
Uptime is 290 days so this was a network issue only... 08:41:32 up 290 days, 17:49, 1 user, load average: 0.35, 0.09, 0.07
It's purely a network issue. we have multiple servers not reachable at the moment due to this network issue.

Posted by ALMGHARI, 05-03-2010, 09:45 AM
http://theplanet.com/
https://orbit.theplanet.com/
DOWN !!
And also my server is DOWN

Posted by X-Max, 05-03-2010, 09:46 AM
yep its gone.. completely down

Posted by ArabCoders, 05-03-2010, 09:49 AM
same my servers still down

Posted by HostMantis, 05-03-2010, 09:51 AM
Our servers are unreachable as well.

I hope they get this sorted out soon.

Posted by pms, 05-03-2010, 09:51 AM
orbit and the forums just came back for me (in Pittsburgh on Comcast)

Posted by Clutch, 05-03-2010, 09:52 AM
OUCH. Down for me... my dedicated server unreachable and can't get to The Planet or Orbit.

Posted by Dant27, 05-03-2010, 09:52 AM
All of our boxes are now back online (from the UK), still unable to reach theplanet.com and hostgator.com though.

Posted by softshop011, 05-03-2010, 09:52 AM
unreal, budget hosts are more stable than theplanet !@#

Posted by X-Max, 05-03-2010, 09:53 AM
orbit and theplanet web site still down from chicago.

Posted by Timu, 05-03-2010, 09:56 AM
Their own website is down.. They are not answering the phone calls.. I have been losing my clients
The worst day and time to be down..

Posted by ArabCoders, 05-03-2010, 09:57 AM
im able to access orbit and theplanet.com but unable to access my servers ;/

Posted by ymfm, 05-03-2010, 09:57 AM
I can now access all servers affected from Brazil, but still down for platinumservermanagement.

Posted by infracom2005, 05-03-2010, 09:59 AM
offline again

Posted by bubbafish, 05-03-2010, 10:00 AM
This one is a doozy... The Planet doesn't have good redundancy it seems.

Posted by Timu, 05-03-2010, 10:00 AM
Yeah theplanet.com is coming up slowly but no luck on my own server Thanks got I already moved most of my servers to another data center a couple years ago... The planet was a decent company while they were EV1...
They do not bother to let us know what is going on ..

Posted by The_Dominator, 05-03-2010, 10:01 AM
well if a router went down as they say - it will all come back soon - just need to refresh the ip tables -- hostgator down, the planet down - lots of big hosts are there with them -- crappy way to start a monday for all you effected --

Posted by X-Max, 05-03-2010, 10:01 AM
still down 45 minutes counting

Posted by spykee, 05-03-2010, 10:01 AM
yikes.. seems like the problem is not yet fix.

Posted by nettigritty, 05-03-2010, 10:01 AM
twitter.com/ThePlanet - they really could have used that to update customers.. lack of communication is bad.

Posted by SenseiSteve, 05-03-2010, 10:01 AM
Their website is still down from St. Louis

Posted by ArabCoders, 05-03-2010, 10:04 AM
my servers are back online

Posted by lbeachmike, 05-03-2010, 10:04 AM
The lack of communication and preparedness by The Planet is entirely inexplicable.

I spoke with them last night via chat when the following limited information was given to me by the support rep, because he said that his hands were cramping from so many support requests - in other words -

Quote:
Martin Osborne: At the moment we are experiencing some sort of network problem and this is interfering with traffic to and from the Houston datacenters. The NOC is aware of the problem and we are putting all of our available resources into solving the problem. We don't know the extent of the problem as of yet, and we don't have an estimate time for the fix. We apologize for the inconvenience this is causing and will update as soon as we know more. This particular issue appears to be network related, the servers themselves do not appear affected, they just cannopt reach the internet. Once network connectivity is restored, the servers should appear available then.
I strongly urged for them to put out an outage notification, which he said they would be doing, but I've seen nothing.

Instead, we cannot currently reach them via chat or phone and our servers in H1 are still down.

We should not collectively tolerate this level of service.

Thanks.

Mike

Posted by ALMGHARI, 05-03-2010, 10:05 AM
The server came back now

Posted by HostMantis, 05-03-2010, 10:05 AM
We are back up! Yay!!!!

Posted by infracom2005, 05-03-2010, 10:06 AM
my servers back up but are working very slow

Posted by tymiller, 05-03-2010, 10:06 AM
Just texted one of the DC Sr. Managers in Houston and all he said was, "network problem, dont yet have specifics."

Hopefully it will be back up soon.

Posted by nettigritty, 05-03-2010, 10:06 AM
Servers back online now. cant open orbit though.

Posted by handsonwebhosting, 05-03-2010, 10:07 AM
Quote:
Originally Posted by nettigritty
twitter.com/ThePlanet - they really could have used that to update customers.. lack of communication is bad.
This is what I"m most upset about too. They do have a twitter page, and when NOTHING is accessible, someone should at least be updating that page with what is going on.

They used to have an offsite page for updates before, but I no longer find that either (when the DC blew up a few years ago).

Posted by sirbruce, 05-03-2010, 10:07 AM
It was announced yesterday...



Dear Customer:

On May 2, 2010, between 12:00am and 7:00am CST, our Network Engineering team will implement a distribution layer in our Hstntx2 network. This is one of many improvements to our network which will provide increased capacity and reliability.

While we've scheduled a seven-hour maintenance window, we anticipate no downtime, however, there will be routing changes as traffic is rebalanced during the turn-up of the new routers.

If you have multiple servers in Hstntx2 and would like additional information about which servers are downstream from this maintenance, you can check your hardware objects in Orbit or contact us directly.

To check the location via Orbit, click "Network" and then "Bandwidth Utilization" (direct link). Servers with "hstntx2" in the location column will be downstream from the maintenance.

If you have any questions about the planned maintenance, please contact your sales representative or our Customer Care Department through our Orbit ticketing system, Live Support chat, or by phone at 866-325-0045.

Posted by ALMGHARI, 05-03-2010, 10:07 AM
SERVER (174.***.***.***)
UP at May 03 2010 04:03 PM (GMT +2:00)
Was down for: 01hr 00min

Posted by X-Max, 05-03-2010, 10:12 AM
all up now

Posted by IH-Antonio, 05-03-2010, 10:13 AM
Quote:
Originally Posted by sirbruce
It was announced yesterday...



Dear Customer:

On May 2, 2010, between 12:00am and 7:00am CST, our Network Engineering team will implement a distribution layer in our Hstntx2 network. This is one of many improvements to our network which will provide increased capacity and reliability.

While we've scheduled a seven-hour maintenance window, we anticipate no downtime, however, there will be routing changes as traffic is rebalanced during the turn-up of the new routers.

If you have multiple servers in Hstntx2 and would like additional information about which servers are downstream from this maintenance, you can check your hardware objects in Orbit or contact us directly.

To check the location via Orbit, click "Network" and then "Bandwidth Utilization" (direct link). Servers with "hstntx2" in the location column will be downstream from the maintenance.

If you have any questions about the planned maintenance, please contact your sales representative or our Customer Care Department through our Orbit ticketing system, Live Support chat, or by phone at 866-325-0045.
First of all, I didnt received such mail.

Second... Location: hstntx1

Posted by nettigritty, 05-03-2010, 10:14 AM
The update below says "we anticipate no downtime".


Quote:
Originally Posted by sirbruce
It was announced yesterday...



Dear Customer:

On May 2, 2010, between 12:00am and 7:00am CST, our Network Engineering team will implement a distribution layer in our Hstntx2 network. This is one of many improvements to our network which will provide increased capacity and reliability.

While we've scheduled a seven-hour maintenance window, we anticipate no downtime, however, there will be routing changes as traffic is rebalanced during the turn-up of the new routers.

If you have multiple servers in Hstntx2 and would like additional information about which servers are downstream from this maintenance, you can check your hardware objects in Orbit or contact us directly.

To check the location via Orbit, click "Network" and then "Bandwidth Utilization" (direct link). Servers with "hstntx2" in the location column will be downstream from the maintenance.

If you have any questions about the planned maintenance, please contact your sales representative or our Customer Care Department through our Orbit ticketing system, Live Support chat, or by phone at 866-325-0045.

Posted by Nick H, 05-03-2010, 10:21 AM
Quote:
Originally Posted by nettigritty
The update below says "we anticipate no downtime".
And this May 3rd, not May 2nd...

Posted by C4talyst, 05-03-2010, 10:24 AM
It's 10:23AM EST and I can't get to any of my machines at ThePlanet, their phones are down and I can't get to ThePlanet.com. I'm surprised there's not a more prominent thread on this.

Posted by Nick H, 05-03-2010, 10:26 AM
Quote:
Originally Posted by C4talyst
I'm surprised there's not a more prominent thread on this.
This isn't ThePlanet's support forum

Posted by HostMantis, 05-03-2010, 10:26 AM
Our servers are accessible, but the network seems sluggish...

Posted by bubbafish, 05-03-2010, 10:29 AM
Still down for us...

Posted by C4talyst, 05-03-2010, 10:29 AM
Quote:
Originally Posted by Nick H
This isn't ThePlanet's support forum
Obviously, but isn't this the largest webhost/ISP discussion forum, and isn't ThePlanet the largest dedicated server host in the world? I'm just surprised more here aren't affected by this.

Posted by sirbruce, 05-03-2010, 10:29 AM
It is working now, and ping is faster than before.

url xboxmaniac . es

Posted by netomatic, 05-03-2010, 10:39 AM
Still down for us.

Posted by KHazard, 05-03-2010, 11:00 AM
I'm sorry for the lack of updates here ... This morning, we experienced two distinct network issues, one around midnight CDT and the other around 8 a.m. CDT. We believe that all customer services have been completely restored, and our network team is investigating the root causes of both issues.

If I get any conclusive information about the cause of the outages, I'll try to post details on The Planet's twitter account (@theplanet), here and in The Planet Forums.

I'm sorry about the interruption and the lack of communication.

Posted by tomydurden, 05-03-2010, 11:14 AM
Quote:
Originally Posted by sirbruce
It was announced yesterday...



Dear Customer:

On May 2, 2010, between 12:00am and 7:00am CST, our Network Engineering team will implement a distribution layer in our Hstntx2 network. This is one of many improvements to our network which will provide increased capacity and reliability.

While we've scheduled a seven-hour maintenance window, we anticipate no downtime, however, there will be routing changes as traffic is rebalanced during the turn-up of the new routers.

If you have multiple servers in Hstntx2 and would like additional information about which servers are downstream from this maintenance, you can check your hardware objects in Orbit or contact us directly.

To check the location via Orbit, click "Network" and then "Bandwidth Utilization" (direct link). Servers with "hstntx2" in the location column will be downstream from the maintenance.

If you have any questions about the planned maintenance, please contact your sales representative or our Customer Care Department through our Orbit ticketing system, Live Support chat, or by phone at 866-325-0045.
FYI: The Change on May 02 isn't related to today's issues.

It looks like everything is normalizing at this time, however you guys may see some residual effects of the outage. Kevin Hazard and I are seeking more technical information from our networking group at this time.

Posted by Mekhu, 05-04-2010, 03:03 AM
We just went offline again. Hopefully it doesn't last long. We've had a tough day as is.

Back online... Minor hiccup.

Posted by beet, 05-04-2010, 03:03 AM
Cannot connect to servers... happened maybe 5 minutes ago

This is two days in a row.

Posted by Matt R, 05-04-2010, 03:06 AM
Encountering the same thing, yet again.

It's bouncing up and down, however.

Definitely another routing issue.

Posted by BalticServers Andrius, 05-04-2010, 03:20 AM
That's a shame for big company.



Was this answer helpful?

Add to Favourites Add to Favourites    Print this Article Print this Article

Also Read
WiredTree Down again! (Views: 1071)
Hosting50.com is down (Views: 1396)

Language: