FutureHosting/RapidSwitch London problems

Portal Home > Knowledgebase > Industry Announcements > Web Hosting Main Forums > Providers and Network Outages and Updates > FutureHosting/RapidSwitch London problems

Posted by Quid5, 05-24-2010, 07:27 PM
Their own website down, and servers there all offline. Anybody confirm?
Posted by xeno007, 05-24-2010, 07:28 PM
Yes, I can confirm that. Since 5 minutes ago, all servers are down.
Also, their website doesn't open.
Posted by PCS-Jason, 05-24-2010, 07:35 PM
Yes RapidSwitch are currently down.
Posted by magdesign, 05-24-2010, 07:39 PM
Yep, all down for me too.

Why does this always seem to happen when you are right in the middle of something?!
Posted by albanach, 05-24-2010, 07:43 PM
Looks like their phones are down too. Nothing yet on their twitter feed either.

OK, these things happen. Hopefully things will be back a lot quicker this time and the communication will be better.

They will have learned, won't they?
Posted by xeno007, 05-24-2010, 07:45 PM
We will have to see about that, 20 minutes down and still nothing
Posted by PCS-Jason, 05-24-2010, 07:45 PM
There phones are down as they're VOIP which run on there network which in situations like this is beyond stupid.
Posted by StreamGBcom, 05-24-2010, 07:46 PM
i got 2 boxes down aswell, no email from them as yet to the reason.
Posted by VMhosts, 05-24-2010, 07:52 PM
All our systems are down, no phones at RapidSwitch either....
Posted by SlAiD, 05-24-2010, 07:54 PM
Hi,

Is my imagination or rapidswitch is AGAIN down?

I'm tired of every half year lose some hours because of the dacatenrter lazy work on backup connections.

Can anyone confirm this?

Message from FutureHosting support:

Date: 5/25/2010
Time: 12:20AM Local
Location: LON01
Services Affected: Core Network
Reason:

At approximately 12:20AM local time engineers were alerted to a loss of connectivity to multiple servers on in our London location. The issue appears to lie with our upstream provider's core network .

We do not have a current ETA for resolution; however, they are aware of the issue and we will update customer tickets as soon as additional information is available.

We apologize for the inconvenience and welcome feedback.

SL
Posted by mallee, 05-24-2010, 08:06 PM
I think it's not London specific. XenSmart in Maidenhead is unreachable too.
Posted by Armageddon21, 05-24-2010, 08:06 PM
This is stupid,

The worst part of this is not even the downtime but the complete failure in communication.

We need to know some kind of detail, ETA or at bare minimum that they are working on the problem.
Posted by Dan-CKS, 05-24-2010, 08:08 PM
Phone Lines are down.. They use there own in house voip Not good....

Edit: didnt see jasons post above..
Posted by albanach, 05-24-2010, 08:10 PM
You would think someone there would at least have an iPhone and an email account.

It looks like their DNS isn't geographically separated either, so neither nameserver is responsive. I guess unresponsive is part of the ethos there though.
Posted by EastCoast, 05-24-2010, 08:12 PM
See: http://www.webhostingtalk.com/showthread.php?t=950909

Mods please merge topic
Posted by VMhosts, 05-24-2010, 08:16 PM
The first time we got caught out by this we moved all our support desk and email systems to another provider, just so we can stay in contact with custoemrs if there was a problem. If the other provider had a problem then hopefully it wouldnt be at the same time as RapidSwitch

Why on earth dont RapidSwitch do this? No one has any idea what is going on, not very helpful, especially when you have customers calling you
Posted by kayz, 05-24-2010, 08:17 PM
The entire shebang is down, what stupidity.. their website is on the same network as their customers and so is their phone.. so no website and nobody to call.

Ridiculous!! This happened sometime ago too i remember, but it wasnt as this bad.

http://twitter.com/#search?q=rapidswitch
Posted by tsumaru, 05-24-2010, 08:18 PM
my coloed server is down also, zip on the communication. Surely they can afford a usb dongle to give us an update ¬_¬
Posted by Dan-CKS, 05-24-2010, 08:19 PM
Time to rapidly Switch People
Posted by Deehem, 05-24-2010, 08:19 PM
I only ever appear here should Rapidswitch fail, and yet again, they have. Christ, I'm so furious now.

Agreed on the complete lack of communication.
Posted by kayz, 05-24-2010, 08:22 PM

Quote:

Originally Posted by tsumaru

my coloed server is down also, zip on the communication. Surely they can afford a usb dongle to give us an update ¬_¬

Thats what im thinking... they make so much money yet they cant afford a few bucks to keep their own website and phone network seperate to their customers.

We should atleast know what is going on and be given some hope of the services being restored.

Seriously.. its like America.. you wonder how the country is so powerfull yet the people running it dont know where they are on the map.

I wonder which bright spark thought of keeping their main website and phone on the same network.
Posted by VMhosts, 05-24-2010, 08:26 PM

Quote:

Originally Posted by eG-Dan

I only ever appear here should Rapidswitch fail, and yet again, they have. Christ, I'm so furious now.

Agreed on the complete lack of communication.

Its nice they raised the prices as well to help pay for these outages - I think I may get in my car and drive down there, furious doesnt come close.....
Posted by tsumaru, 05-24-2010, 08:27 PM

Quote:

Originally Posted by kayz

Thats what im thinking... they make so much money yet they cant afford a few bucks to keep their own website and phone network seperate to their customers.

We should atleast know what is going on and be given some hope of the services being restored.

Seriously.. its like America.. you wonder how the country is so powerfull yet the people running it dont know where they are on the map.

I wonder which bright spark thought of keeping their main website and phone on the same network.

Even better, Pay Poundhost on the other side of town for a backup server... I might even colo another one ther and offer my services to Rapidswitch
Posted by albanach, 05-24-2010, 08:31 PM
This is a laugh:

twitter.com/iomartservice

Apparently those are real time updates on their data centre's, including RapidSwitch in Maidenhead. Last updated eighteen hours ago.

Others might want to try tweeting there and RapidSwitch
Posted by tsumaru, 05-24-2010, 08:31 PM
My server and teh rapidswitch website are back o.O
Posted by VMhosts, 05-24-2010, 08:31 PM
myservers page is now back
Posted by CretaForce, 05-24-2010, 08:31 PM
And back online.
Posted by PCS-Jason, 05-24-2010, 08:32 PM
Its now back for me.
Posted by yello, 05-24-2010, 08:32 PM
Up up up! My servers are up!
Posted by albanach, 05-24-2010, 08:35 PM
Looks like we're back up now.
Posted by Dan-CKS, 05-24-2010, 08:35 PM
Managed to get through on the phone, Apparently its a Severe Network problem and they have no information to give out yet...
Posted by kayz, 05-24-2010, 08:35 PM

Quote:

Originally Posted by VMhosts

Its nice they raised the prices as well to help pay for these outages - I think I may get in my car and drive down there, furious doesnt come close.....

Yes and i think you should take a message down their on our behalf too!

Quote:

Originally Posted by tsumaru

Even better, Pay Poundhost on the other side of town for a backup server... I might even colo another one ther and offer my services to Rapidswitch

Now thats an idea!
Posted by tsumaru, 05-24-2010, 08:37 PM
ok, ping alarms are set. im off for some ZZs have to be in the office at five hours ¬_¬
Posted by StreamGBcom, 05-24-2010, 08:38 PM
mine are back on now
Posted by VMhosts, 05-24-2010, 08:49 PM
Hi

There has been an unscheduled outage, which has affected some of your services:

Date: 25/05/2010
Time: 00:20
Outage Type: Connectivity Interruption
Duration: < 75 minutes

At approximately 00:20 a network issue was detected affecting connectivity to the North cluster in RapidSwitch Spectrum House.

Our on call engineers were contacted and the problem has been resolved. Service was restored at approximately 01:35. Unfortunately we do not currently have any further details available at this time but will issue them as and when they become available.

We would like to express our sincere apologies for the inconvenience this will have caused. If you are experiencing any problems as a result of this please do not hesitate to contact our support team immediately and we will look into this for you.

Regards,

The RapidSwitch Team
Posted by VMhosts, 05-24-2010, 08:49 PM
time for some sleep
Posted by albanach, 05-24-2010, 09:01 PM
Yes, bed time now.

Tomorrow however, RapidSwitch really have to get their act together.

Either their or their parent group's twitter page should have been updated. What's the point of a realtime status page that doesn't reflect what's really happening at that point in time.

I raised a ticket by email as soon as I spotted our servers were down. An hour later I get a reply - when the network was restored - asking me to validate the ticket using myservers. Why on earth demand tickets be validated using a service that goes down whenever your network does?

Customers need to know that the phone, email and support system will be moved to a diverse network. It should be possible for all that stuff to failover to an ADSL or even ISDN line if needed. At least enough to keep communications with the outside world running.
Posted by 1click2host, 05-24-2010, 09:15 PM
up after 1 hour down
Posted by xeno007, 05-24-2010, 09:30 PM

Quote:

Originally Posted by RS

It will have ranged from no effect, to an increase in latency and some packet loss, to a more substantial loss of connectivity.

WOW, what an ignorance. They were completely down, including their website, telephone, everything... and now using words like "it may have had no effect or increased latency and some packet loss"???
Wow, just wow.
Posted by 1click2host, 05-25-2010, 03:32 AM

Quote:

Originally Posted by xeno007

WOW, what an ignorance. They were completely down, including their website, telephone, everything... and now using words like "it may have had no effect or increased latency and some packet loss"???
Wow, just wow.

i think they mean after what happen not what happen !!!

Quote:

Our on call engineers were contacted and the traffic has been removed from the network. Normal service was restored at approximately 01:35. This problem will have affected different clients in different ways. It will have ranged from no effect, to an increase in latency and some packet loss, to a more substantial loss of connectivity.

Posted by CraigMesser, 05-25-2010, 03:49 PM
Anyone have any further details regarding this?
Posted by Dan-CKS, 05-25-2010, 04:53 PM
Everything is Fine now.

Here is what happened anyways:

At approximately 00:20 a network issue was detected affecting connectivity to the North cluster in RapidSwitch Spectrum House. The cause of this appears to have been a very large amount of malicious traffic directed at the RapidSwitch network. This traffic has unfortunately caused a routing issue within the RSH.North cluster.

hmm realy?? how come your Phone Lines, Website and Monitoring was offline....
Posted by BradQ, 05-25-2010, 09:50 PM

Quote:

Originally Posted by Armageddon21

This is stupid,

The worst part of this is not even the downtime but the complete failure in communication.

We need to know some kind of detail, ETA or at bare minimum that they are working on the problem.

People like you make me want to set myself on fire during any outage. Get over your self-righteous indigence and consider the following:

1) This is a full-network outage. When there are 10 posts on WHT confirming this, what on earth would make you think they aren't "working on the problem"? The techs fixing your issue aren't exactly focused on providing minutely updates.

2) If your network is completely down it's incredibly rare you'd have *any* ETA. The issue is usually desperately being worked on and is "fixed when it's fixed". It's rare the problem will be one with any predictable timeframe, even under circumstances with advance notice.

While the communication you get during an outage is important from a customer service perspective, it's more a function of the size of the company to have someone to dedicate to such posts than a commitment to prompt communication. Complaining that it's frustrating, fine, I buy it.

Mod note: Probably not exactly fair to be continuing this thread under FH's name. They just happen to be hosted there, and the outage is completely unrelated as I read it.