Has burstnet just Burst?

Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > Has burstnet just Burst?

Posted by M5-Network, 03-05-2010, 10:58 PM
About 5 mins about the network started going nuts.

Bursts site isnt loading and my servers are going insane?
Posted by Nick H, 03-05-2010, 11:22 PM
My monitoring system is going insane too - all servers I monitor at Burst are going up and down constantly.

But HyperSpin didn't say they're down, so I'm sure they'll claim they weren't.
Posted by UnderHost, 03-05-2010, 11:27 PM
HyperSpin is hosted by Burst? ^^
Posted by Nick H, 03-05-2010, 11:39 PM

Quote:

Originally Posted by UnderHost

HyperSpin is hosted by Burst? ^^

No, they use HyperSpin to refute any claim that their network was offline

If HyperSpin says no then everyone else is wrong - including my own monitoring system apparently.
Posted by BudgetVPS, 03-05-2010, 11:43 PM
Every topic created that mentions BurstNET turns into a Let's slate them thread?

I've seen this happen many times, I'm sure they'll admit to network issues. Even on AIM they informed me they was looking into the issue.

Network is back to normal now
Posted by MattS, 03-05-2010, 11:46 PM
Yeah mine did the same thing, seems back to normal right now though.
Posted by M5-Network, 03-05-2010, 11:52 PM
-Marked for closure-
Posted by MattS, 03-05-2010, 11:54 PM
In case you're wondering:

Quote:

There was a small issue with a switch that required it to be restarted.

Posted by M5-Network, 03-05-2010, 11:55 PM
Ahh cheers Matt <3
Posted by M5-Network, 03-06-2010, 12:00 AM
Its going again :O
Posted by DMEHosting, 03-06-2010, 12:00 AM
10 servers that I've tested our showing 60-90% packet loss. I'm sure their network admins are looking into it and wouldn't expect this to go longer than 30-60 minutes.
Posted by Nick H, 03-06-2010, 12:00 AM

Quote:

Originally Posted by M5-Network

Its going again :O

Yep, great, my phone buzzing again.
Posted by JMele, 03-06-2010, 12:00 AM

Quote:

Originally Posted by M5-Network

Its going again :O

yup, im switching away from from burst.
Posted by JMele, 03-06-2010, 12:02 AM

Quote:

Originally Posted by DMEHosting

10 servers that I've tested our showing 60-90% packet loss. I'm sure their network admins are looking into it and wouldn't expect this to go longer than 30-60 minutes.

do you guys respond to tickets on the weekend? I opened a ticket about wanting to move locations because of this.
Posted by BA-Corey, 03-06-2010, 12:07 AM
Yea we are having issues also!

Someone in virginia is having 50% packet loss, and in grand saline tx. I'm not having any packet loss atm though.
Posted by twhiting9275, 03-06-2010, 12:08 AM

Quote:

Originally Posted by Nick H

No, they use HyperSpin to refute any claim that their network was offline

So very, very true. In every case, it's the world vs. Burst, and Burst always (always) insists they're right.
Seems up/down for me as well the past hour or so. Let's hope they get the problem fixed.
Posted by BA-Corey, 03-06-2010, 12:10 AM

Quote:

Originally Posted by linux-tech

So very, very true. In every case, it's the world vs. Burst, and Burst always (always) insists they're right.
Seems up/down for me as well the past hour or so. Let's hope they get the problem fixed.

yea we've had these types of issues at least once a month since we've been with them.
Posted by DMEHosting, 03-06-2010, 12:16 AM

Quote:

Originally Posted by JMele

do you guys respond to tickets on the weekend? I opened a ticket about wanting to move locations because of this.

Shoot me a PM with your ticket #. Will take a look.
Posted by M5-Network, 03-06-2010, 12:17 AM
stablized again

/sigh
Posted by KMyers, 03-06-2010, 12:27 AM
My servers at Burst are still showing some packet loss, especially to may management box.
Posted by M5-Network, 03-06-2010, 01:37 AM
Down again...

Who wants to play wack a server tech at blizz!

Rules are:
Rubber mallets only.
Must find the incompatant techies!
Must get servers back online!

--
The more you wack a techie the faster your server comes online!
Posted by JMele, 03-06-2010, 01:39 AM

Quote:

Originally Posted by M5-Network

Down again...

Who wants to play wack a server tech at blizz!

Rules are:
Rubber mallets only.
Must find the incompatant techies!
Must get servers back online!

--
The more you wack a techie the faster your server comes online!

mine are still up.. *knock on wood* but im still down, lets go.
Posted by RyanD, 03-06-2010, 02:04 AM
we've been seeing up/down alerts all night to servers we monitor at burst.
Posted by vghsoluciones, 03-06-2010, 01:05 PM
we have the problem that some countries do show our websites and other countries no worse they are not responding tikets 6 days ago, someone told him the same thing happens?
Posted by peruviantalk, 03-06-2010, 02:36 PM
Is always not THEIR fault. Is always something else.

This cycle never gets old. How much you want to be they still have their 99.9999% uptime after this
Posted by Negizmo, 03-06-2010, 08:13 PM
I'm surprised they haven't posted anything regarding this, they're usually pretty good about letting their customers know what goes on, aren't they?
Posted by todo1419, 03-06-2010, 08:25 PM
2 Of ours are down too, thanks burst!
Posted by Dan-CKS, 03-07-2010, 05:31 AM
same here too.
Posted by DMEHosting, 03-07-2010, 05:35 AM
Servers seem to be having packet loss again tonight. Maybe they are fixing an issue that was caused yesterday?
Posted by Dan-CKS, 03-07-2010, 05:39 AM

Quote:

Originally Posted by neo1211

I'm surprised they haven't posted anything regarding this, they're usually pretty good about letting their customers know what goes on, aren't they?

Perhaps they could be under attack, we just dont know. i have tried calling them several times however there is only an answer machine.
Posted by firefoxie, 03-07-2010, 07:11 AM
Does anybody know when will this issue be solved??
My hosting company emailed me and told they are down due to to burst net.
My sites are down.
Posted by Dan-CKS, 03-07-2010, 07:20 AM
No news as yet.
Posted by ASHCN, 03-07-2010, 07:30 AM
My servers keep coming up and then down as well...seems like network issues as ive been up and down for hours now!
Posted by ASHCN, 03-07-2010, 07:31 AM
My guess is that there is a DOS attack happening or a switch that has gone faulty..
Posted by Dan-CKS, 03-07-2010, 07:32 AM
Burst are usualy good at updates. however i have had nothing as yet.
Posted by RyanD, 03-07-2010, 07:35 AM
and the yo-yo continues........
Posted by MattS, 03-07-2010, 08:00 AM
Well the switch was part of the problem so we'll see I guess on Monday whether they replaced it or if they had something else happen on top of the switch issue.
Posted by Crothers, 03-07-2010, 08:08 AM
I wonder how they'll blame the customers this time.

Since its inevitable the blame will be placed somewhere else instead of on them.
Posted by M5-Network, 03-07-2010, 08:30 AM
Im just watching my 99.9% uptime go out the window here :/
Posted by RS Shamil, 03-07-2010, 08:33 AM
Hmm interesting. I don't have any issues with Burst myself.
Posted by Dan-CKS, 03-07-2010, 08:36 AM
A quick look on my ping monitor reveals.

http://hosted.shoutcast-host.co.uk/c4546a71.png
Posted by RealRoxy, 03-07-2010, 08:37 AM
Seems fixed now?
Posted by Dan-CKS, 03-07-2010, 08:41 AM
No, i am still having packet loss.
Posted by sharmaine1111, 03-07-2010, 09:12 AM

Quote:

Originally Posted by peruviantalk

This cycle never gets old. How much you want to be they still have their 99.9999% uptime after this

They don't guarantee 99.999% uptime, and never had. Their uptime guarantee is 99.5%

And I ping their websites, there is no packet loss at this time

I think they are under attack
Posted by sharmaine1111, 03-07-2010, 10:02 AM
I spoke too soon. Yes Burst is still experiencing packet loss
Posted by Isnhosting, 03-07-2010, 12:49 PM
brust net dropped our hardisk data loss its horrible
Posted by themuggle, 03-07-2010, 01:15 PM
Is hostnoc and burst the same company? If so i have another question~
Posted by SC-Jon, 03-07-2010, 01:20 PM

Quote:

Originally Posted by themuggle

Is hostnoc and burst the same company? If so i have another question~

Yes, hostnoc and burst are the same.
Posted by themuggle, 03-07-2010, 01:24 PM
Okay so i'm with burst through a reseller. For the past week or so.. or maybe for the past 10 days.. I've been having this issue where certain accounts just stop loading! Like there will be one or maybe two domains that won't work for me on the entire server, but i'm definitely not IP blocked through the csf firewall. And this has been happening to some of my clients as well. They can't view their domain but the rest of the accounts work! But at the same time if you view the domain through a proxy it works perfectly.

Does this have anything to do with burst's recent issues or do they even block IP's on a network level?
Posted by Dan-CKS, 03-07-2010, 01:33 PM

Quote:

Originally Posted by themuggle

Okay so i'm with burst through a reseller. For the past week or so.. or maybe for the past 10 days.. I've been having this issue where certain accounts just stop loading! Like there will be one or maybe two domains that won't work for me on the entire server, but i'm definitely not IP blocked through the csf firewall. And this has been happening to some of my clients as well. They can't view their domain but the rest of the accounts work! But at the same time if you view the domain through a proxy it works perfectly.

Does this have anything to do with burst's recent issues or do they even block IP's on a network level?

Get Rid of CSF, You will find your problem fixed
Posted by twhiting9275, 03-07-2010, 01:39 PM

Quote:

Originally Posted by M5-Network

Im just watching my 99.9% uptime go out the window here :/

I wouldn't count on Burst to refund for anything there. As others have said, if their own monitoring systems don't show it down, then it aint down.

Was up and down for a good little bit last night again, though.
Posted by BurstJoeM, 03-07-2010, 02:21 PM
Greetings,

This mornings issue has been resolved. We had a couple of issues occur over the last couple of days.

* A reoccurring +7Gb/s UDP flood targeting several related customers - The traffic appears to be originating mostly from china and is targeting a handful of customers. Unfortunately, this attack was creating a resource issue on 1 of our edge routers (GWY01.SCTN01). This resource issue was addressed this morning and we are working with 2 of our providers to mitigate the UDP flood if it happens again.

* This morning we began to experience an issue with one of our interface cards in GWY02.SCTN01. This card was replaced.

It appears that both issues came to a head at approximately the same time leading to a large number of reports packetloss and latency. Traffic coming in or out of GWY03.SCTN01 was not affected.

We apologize for any inconvenience.
Posted by Negizmo, 03-07-2010, 04:16 PM

Quote:

Originally Posted by BurstJoeM

Greetings,

This mornings issue has been resolved. We had a couple of issues occur over the last couple of days.
... <snip> ...

There we go, I'm just shocked that this wasn't let out sooner.. you guys seem to keep people in the now for the most part, not sure why that was different in this incident.
Posted by Crothers, 03-07-2010, 04:21 PM

Quote:

Originally Posted by BurstJoeM

Greetings,

This mornings issue has been resolved. We had a couple of issues occur over the last couple of days.

* A reoccurring +7Gb/s UDP flood targeting several related customers - The traffic appears to be originating mostly from china and is targeting a handful of customers. Unfortunately, this attack was creating a resource issue on 1 of our edge routers (GWY01.SCTN01). This resource issue was addressed this morning and we are working with 2 of our providers to mitigate the UDP flood if it happens again.

* This morning we began to experience an issue with one of our interface cards in GWY02.SCTN01. This card was replaced.

It appears that both issues came to a head at approximately the same time leading to a large number of reports packetloss and latency. Traffic coming in or out of GWY03.SCTN01 was not affected.

We apologize for any inconvenience.

7gig shouldn't be much for "one of the largest hosts" on WHT...

I also notice your HyperSpin says 100%, which we are all well aware is not the case... Perhaps that can be fixed as well while your working on mitigating that DDoS?

http://imgur.com/J67iG.png
Posted by RobM, 03-07-2010, 04:38 PM

Quote:

Originally Posted by Crothers

7gig shouldn't be much for "one of the largest hosts" on WHT...

I also notice your HyperSpin says 100%, which we are all well aware is not the case... Perhaps that can be fixed as well while your working on mitigating that DDoS?

http://imgur.com/J67iG.png

Like Joe said the whole network did not go offline.

Colo area was up 100% not even packet loss
Posted by BurstJoeM, 03-07-2010, 04:53 PM

Quote:

Originally Posted by Crothers

7gig shouldn't be much for "one of the largest hosts" on WHT...

I also notice your HyperSpin says 100%, which we are all well aware is not the case... Perhaps that can be fixed as well while your working on mitigating that DDoS?

http://imgur.com/J67iG.png

Generally 7Gb (it was actually more) isn't a problem, however since it was UDP based, the size of the attack wasnt the problem. It was packets per second (pps) that we ran into an issue with.

The attack impacted 3 of our 7 transit carriers. Apparently hyperspin's traffic traverses a transit carrier that was not impacted by this issue.
Posted by Crothers, 03-07-2010, 05:18 PM
Hyperspin doesn't use one location, it uses many. The odds that 100% of Hyperspins traffic came in on an unaffected route is slim. Something still seems broken there.

Just saying, its worth looking into... for the sake of legitimate uptime reports, and for the customers who may possibly require SLA credits.
Posted by themuggle, 03-07-2010, 05:35 PM

Quote:

Originally Posted by FirstNetServ

Get Rid of CSF, You will find your problem fixed

Would you suggest a good alternative?
Posted by Dan-CKS, 03-07-2010, 05:39 PM

Quote:

Originally Posted by themuggle

Would you suggest a good alternative?

No sorry

it may not be the best idea to remove csf but atleast it will solve all your problems.
Posted by BurstNET, 03-07-2010, 07:12 PM

Quote:

Originally Posted by Crothers

Hyperspin doesn't use one location, it uses many. The odds that 100% of Hyperspins traffic came in on an unaffected route is slim. Something still seems broken there.

Just saying, its worth looking into... for the sake of legitimate uptime reports, and for the customers who may possibly require SLA credits.

Our network was not down, just certain routes/circuits heavily congested, due to PPS-heavy DDOS traffic, appearing as latency and packet loss therefore. Hyperspin picked up nothing, as were were not down overall---so they must access from multiple routes to confirm accessibility.

As a competitor, I am not sure why you are commenting lately on threads regarding us. You have no first-hand information on us, nor familiar with our policies, or SLA for that matter. Please do not comment on things you know nothing about, and please respect WHT guidelines/rules on inappropriate posting. Our SLA does not cover DDOS attacks.
.
.
Posted by BurstNET, 03-07-2010, 07:15 PM

Quote:

Originally Posted by RobM

Like Joe said the whole network did not go offline.

Colo area was up 100% not even packet loss

Only a portion of routes were affected by this, not confined to specific service types. Unless you are referring to the card replacement this morning...which was quickly replaced.

Our network runs pretty darn well, even compared to "premium" providers charging 3-4x what we do, and if anyone thinks such DDOS does not happen from time to time on EVERY network, they are quite mistaken. Compared to most of the large providers discussed here on WHT, we probably have one of the least amount of network issues reported here in the past year and a half.

We run a rock-solid network, but DDOS affecting issues are inevitable, regardless of how much and what kind of protection is in place. It is not a matter of "if" they will occur , it is a matter of "when" they will occur. And even if we can stop an attack, that does not mean it will not affect service a little more upstream from us on one of our carrier's networks, and affect our traffic therefore. There is no such thing as 100% perfect network service, especially in the budget web hosting industry, it is just not going to happen from any hosting provider, let alone budget providers...that's a fact. Considering that, we've come pretty darn close with our new network though!
.
.
Posted by darkeden, 03-07-2010, 10:46 PM

Quote:

Originally Posted by BurstNET

Only a portion of routes were affected by this, not confined to specific service types. Unless you are referring to the card replacement this morning...which was quickly replaced.

Our network runs pretty darn well, even compared to "premium" providers charging 3-4x what we do, and if anyone thinks such DDOS does not happen from time to time on EVERY network, they are quite mistaken. Compared to most of the large providers discussed here on WHT, we probably have one of the least amount of network issues reported here in the past year and a half.

We run a rock-solid network, but DDOS affecting issues are inevitable, regardless of how much and what kind of protection is in place. It is not a matter of "if" they will occur , it is a matter of "when" they will occur. And even if we can stop an attack, that does not mean it will not affect service a little more upstream from us on one of our carrier's networks, and affect our traffic therefore. There is no such thing as 100% perfect network service, especially in the budget web hosting industry, it is just not going to happen from any hosting provider, let alone budget providers...that's a fact. Considering that, we've come pretty darn close with our new network though!
.
.

At-least you guys took care of it good work.... I do have to agree with you that for me at-least in the past year there have been almost no network issues after that upgrade you did.

And the uptime shows 100% because only some servers etc where affected for the people saying that its wrong I don't know many data centers that put an up-time checker on every single part of the network.
Posted by twhiting9275, 03-08-2010, 01:16 AM
And this continues for a third night this week. This problem is definitely not resolved by any means. Traceroutes show massive loss before, and after their network here, major dns lookup times, it's just a big ol' mess.
Posted by DMEHosting, 03-08-2010, 01:18 AM
They are probably getting another attack as there is more packet loss unfortunately. These things happen which is something all of us rather not deal with, but is unavoidable. When dealing with 7-10Gbps incoming traffic it can over load any switch/router and potentially ruin them.

Lots of attacks originating from China for some reason, time to contact Google on how to deal with this!!! lol
Posted by Nick H, 03-08-2010, 01:38 AM

Quote:

Originally Posted by linux-tech

And this continues for a third night this week. This problem is definitely not resolved by any means. Traceroutes show massive loss before, and after their network here, major dns lookup times, it's just a big ol' mess.

But this sort of thing happens at EVERY provider according to BurstNET - Daily outages/major packet loss due to attacks

BurstNET - FIX THIS - I'm tired of my monitoring system buzzing.
Posted by twhiting9275, 03-08-2010, 01:47 AM

Quote:

Originally Posted by DMEHosting

These things happen which is something all of us rather not deal with, but is unavoidable. When dealing with 7-10Gbps incoming traffic it can over load any switch/router and potentially ruin them.

Wanna know how many times Softlayer's been down for multiple days because of stunts like this? None. Theplanet? None. Oh, TP's been down for days before, but never because of issues like this.

In fact, I can't name a single major provider that's been hit like this for multiple days because they failed to keep their networks protected. That's what it comes down to, protecting your networks.

Quote:

Originally Posted by Nick H

But this sort of thing happens at EVERY provider according to BurstNET - Daily outages/major packet loss due to attacks

As someone who monitors servers across the globe, and responds to alerts for a living, let me tell you this is not the case. In fact, the only major provider I've ever seen this much of an issue, and this slow of a response time with is Burst. Of course, if you listen to them, the customer is always wrong, the network is never down, it's all in your head. That's why 3rd party monitoring services exist, to keep guys that'd try to tell you this scam on the level.
Posted by Nick H, 03-08-2010, 02:04 AM

Quote:

Originally Posted by linux-tech

As someone who monitors servers across the globe, and responds to alerts for a living, let me tell you this is not the case.

Same with me

I always know when BurstNET is down (or to appease BurstNET, not down, but having major issues within your facility, or whatever you wish to call it) because my email starts going off and my phone starts making all sorts of weird noises that it makes when it's receiving lots of messages at once. I'm personally sick of being woken up by it
Posted by DME-Geoff, 03-08-2010, 03:48 AM
Apparently the latest outage was related to a DDoS as well.
*sigh*
Posted by Crothers, 03-08-2010, 06:39 AM

Quote:

Originally Posted by BurstNET

Our network was not down, just certain routes/circuits heavily congested, due to PPS-heavy DDOS traffic, appearing as latency and packet loss therefore. Hyperspin picked up nothing, as were were not down overall---so they must access from multiple routes to confirm accessibility.

As a competitor, I am not sure why you are commenting lately on threads regarding us. You have no first-hand information on us, nor familiar with our policies, or SLA for that matter. Please do not comment on things you know nothing about, and please respect WHT guidelines/rules on inappropriate posting. Our SLA does not cover DDOS attacks.
.
.

Don't know where you get I'm a competitor, considering my job is software development and the company I work for doesn't sell anything at all... but its cool.

As I said earlier, what I'm upset about is the fact that your Hyperspin seems to be broken, which you danced around the topic.

It's showing 100% uptime, which is incorrect, false, and not legitimate as your customers know right now. I was simply wondering when that would be fixed.

So is their an ETA? Since your network was NOT up 100% in the past 48 hours?
Posted by M5-Network, 03-08-2010, 10:11 AM
Im going to get this closed before the flame war gets to over the top -.-

This thread was opened for information on what was happening with the latest downtime on burstnets network. This was not put in place for people to flame.
Posted by sharmaine1111, 03-08-2010, 10:45 AM
In fairness to Burst I have seen too many servers with packet loss with routes going through level3.net I posted another topic for that. Even WHT when doing trace route has packet loss, and again the packet loss in with level3
Posted by FastServ, 03-08-2010, 11:26 AM

Quote:

Originally Posted by sharmaine1111

In fairness to Burst I have seen too many servers with packet loss with routes going through level3.net I posted another topic for that. Even WHT when doing trace route has packet loss, and again the packet loss in with level3

Your assumption that level3 is at fault is wrong.
Posted by sharmaine1111, 03-08-2010, 11:38 AM

Quote:

Originally Posted by FastServ

Your assumption that level3 is at fault is wrong.

Oh really? Then check my reply here: http://www.webhostingtalk.com/showthread.php?t=932094

Make sure you know what youre talking about first before you make rude replies. If that's how rude you are with people, I will be very afraid to get a server from your company
Posted by twhiting9275, 03-08-2010, 11:52 AM
FastServ is correct here, the past few days had nothing to do with level3.

As he said, don't assume everyone is down just because burst is down. The server I've been watching through burst doesn't even go anywhere near level3, so your theory (and that's what it is, just a theory) is completely wrong.
Posted by sharmaine1111, 03-08-2010, 11:57 AM

Quote:

Originally Posted by linux-tech

FastServ is correct here, the past few days had nothing to do with level3.

As he said, don't assume everyone is down just because burst is down. The server I've been watching through burst doesn't even go anywhere near level3, so your theory (and that's what it is, just a theory) is completely wrong.

You even haven't checked my post about level3. I dont know how i can repeat this over and over again, but even before burst had packet losses ive seen packet losses for level3 a few days back, when i did traceroute some servers of hostgator, traceroute servers of other hosts, and recently, traceroute WHT. There are packet losses with level3.

I dont understand the whole point of you saying " The server I've been watching through burst doesn't even go anywhere near level3, so your theory (and that's what it is, just a theory) is completely wrong." I did not say anything that burst has packet losses because level3 does. Please do not put words into my mouth.

What i was saying was there are packet loss with level3. I DID NOT EVEN SAID THAT BURST PACKET LOSS HAS CONNECTION WITH LEVEL3. Duh. Read carefully PLEASE.
Posted by Coolraul, 03-08-2010, 12:32 PM
Closed by request of the OP. If you wish it reopened please let us know.