securedservers. Unstable network even after upgrade.

Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > securedservers. Unstable network even after upgrade.

Posted by gretai-holdings, 04-01-2012, 11:54 PM
securedservers. Unstable network even after upgrade.

I feel thire network is really unstable before and even after theire network upgrade.

They have been experiencing network issue currently at least for 3 hours
and they still don`t know what is the problem.

anyone, Go to any online web-tool to check ping.
"just-ping.com" is good one. and try ping to thire IP.

You can see packet loss from some locations like below.
It is around 4am, April/2nd now.

Shanghai, China: Packets lost (20%)
Lille, France: Packets lost (40%)
Chicago, U.S.A.: Packets lost (30%)
Kuala Lumpur, Malaysia: Packets lost (20%)

3 hours already. I am really upset.
Posted by JordanJ, 04-01-2012, 11:59 PM
One of our carriers are currently having problems. This is 100% off our network. I will reply as soon as I have more info and an estimate from the carrier when they think the problem will be fixed.
Posted by gretai-holdings, 04-02-2012, 12:07 AM
thanks for quick reply.
If one of your carriers is the problem, isn`t it possible like
isolate that carrie from your network temporally and re-route the network?
This is 911 issue but not solved more than 3 hours and I am loosing customers.
Hope you can help in any way.
Posted by bijan588, 04-02-2012, 12:08 AM
I have been having the problem for hours.

Its annoying, but I think they are doing the best they can.

I am loosing customers too, 3 cancellations.
Posted by gretai-holdings, 04-02-2012, 12:19 AM
Even though this issue is 100% off your network, many of your customers are influenced by this issue.
So, securedservers should announce the current ongoing issue as soon as thet are aware of it.
They don`t announce/open anything about the network problem since customers create the ticket and ask them what is going on.
Securedservers, please tell us when you were aware of the issue.
Posted by JordanJ, 04-02-2012, 12:28 AM
Gretai,

Completely understand your concern - however its really hard for a provider to notify a customer every time one of our downstream carriers has any problem.

They have 1000s of routers and hundreds of nodes and often have issues.

On top of that, there is no way to know the scope of a network problem on their network.

In this instance, the issue is pretty small - for example, out of 50 worldwide nodes, only 2 show any packetloss or increased latency at all.

We are looking at things we can do to force traffic to come in other nodes, however most carrier router issues are resolved very quickly (far less than it has been now.)

Please know we have been in contact with the carrier, and will take any and all actions we can to better improve the level of service our customers are getting.
Posted by gretai-holdings, 04-02-2012, 12:41 AM
the issue is getting better now but 3 hours is really not acceptable.
Please tell us
what is the cause of the issue in detail
when did you aware of the issue
and if this can happen again and again.
Posted by JordanJ, 04-02-2012, 12:53 AM
This type of issue can happen to any provider and honestly there is nothing we or anyone can do to prevent it. That being said while issues do happen it rarely affects customers and when it does it's normally only a handful and only for a few minutes. An issue like this for this long is pretty abnormal.

If you ever feel an issue is not being taken care of, not only do you have the emails and phone numbers of all of the support assistant managers, managers, and director, but you can always email feedback@phoenixnap.com which emails all company executives directly. We respond quickly to every email there.

We are committed to providing out customers complete transparency however it's challenging when its the ISP thats causing the problem. Let me give us a use case as an example. Let's say Comcast has issues in Atlanta peering to level3 and you live in Atlanta and have Comcast. Your route that Comcast uses goes through level 3 to get to us. Any customers on Comcast in atlanta won't be able to get to us until the Comcast routers get fixed. The question becomes, how much notification can we provide to our customers? Do we tell the guy in France that there is a problem with an ISP we don't use but some end users might in a city he probably doesn't route through?

Like I said, it's really hard - and honestly the best thing for us to do is arm our support staff to be able to pass on info as they get it.
Posted by Steven, 04-02-2012, 03:36 AM
You could pull the provider out of bgp for example..
Posted by JordanJ, 04-02-2012, 04:56 AM
I'd this was a huge large scale issue that's obviously what we would do. But to remove a carrier due to a tiny issue affecting only a tiny tiny percent of customers on that specific network To have worse service specificly during the time bgp is figuring out how to route the packets now that that carrier is gone.

You have to trust that we have networking staff (not NOC staff but trained certifies engineers) working with the provider who are making the best possible choice for the most of our customers. Carriers having issues is a no win situation for us obviously but if it's Only affecting a couple people and the proidivder is saying it will be fixed soon - of course we would leave that isp in the mix.